Uploading to Google Cloud Storage using Arc and Phoenix

In this post I will demonstrate how to upload to Google Cloud Storage using Arc and Arc.Ecto in Phoenix. There are several examples on how to use Arc in combination with S3 or local file uploads but the GCS integration is a bit more unknown. I assume you have an up to date elixir installation and know your way around Phoenix and GCS.

mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez
mix phx.new web --no-brunch

Follow instructions to configure your postgres credentials in config/dev.exs and create your database with mix ecto.create

We need to add arc_gcs and arc_ecto to our dependencies.

# mix.exs
      {:arc, "~> 0.8.0"},
      {:arc_ecto, "~> 0.7.0"},
      {:arc_gcs, "~> 0.0.2"}

Run mix deps.get to update your dependencies.

Next step is to add some configuration with the bucket name and a keyfile.

To create a keyfile you first need to create a service account, make sure it has the role of ‘Storage Object Creator’. After the service account is set you can create an account-key. This will give you the option to download a keyfile in json-format. This file has the credentials for ArcGCS to upload files. Note, you should never check this file into git. It should be treated as a secret, like API-keys and such.

Add configuration

#config/config.exs

config :arc,
  storage: Arc.Storage.GCS,
  bucket: "gcs-bucket-name"

config :goth,
  json: "/var/keyfile.json" |> Path.expand |> File.read!

Now, we can create an uploader. It is a separate file that handles some of the upload logic. If you know the carrierwave gem in ruby, it sort of works the same.

mix arc.g image_file

Phoenix 1.3 note

As of now the Arc package creates an uploader in web/uploaders, since Phoenix 1.3 the app you’re building is no longer a special case and is moved under lib. We’ll need to move the uploader there to.

mv web/uploaders lib/app_web/uploaders

Uploader

The uploader itself should look like this. We set the storage to Arc.Storage.GCS and use gcs_object_headers to set the mimetype. Furthermore, we enable the Arc.Ecto.Definition

# lib/app_web_uploaders/image_file.ex
defmodule App.ImageFile do
  use Arc.Definition

  # Include ecto support (requires package arc_ecto installed):
  use Arc.Ecto.Definition

  @versions [:original]

  def __storage, do: Arc.Storage.GCS

  def gcs_object_headers(:original, {file, _scope}) do
    [content_type: MIME.from_path(file.file_name)]
  end
end

Phoenix

We’re now going to prepare the Phoenix app for the uploader. Using the new contexts feature in Phoenix 1.3 we will generate an images table with two string fields, name and filename.

 mix phx.gen.html Assets Image images name:string filename:string

Follow the instructions and add

resources "/images", ImageController

to app/lib/app_web/router.ex.

The generator created an Image model. Next, we make it work together with the uploader we created. We add the use Arc.Ecto.Schema line and change the type of the :filename field to the uploader with App.ImageFile.Type.

Furthermore, in the changeset we cast the :filename field to an attachments.

# lib/app/assets/image
defmodule App.Assets.Image do
  use Ecto.Schema
  import Ecto.Changeset
  alias App.Assets.Image
  use Arc.Ecto.Schema

  schema "images" do
    field :filename, App.ImageFile.Type
    field :name, :string

    timestamps()
  end

  @doc false
  def changeset(%Image{} = image, attrs) do
    image
    |> cast(attrs, [:name, :filename])
    |> cast_attachments(attrs, [:filename])
    |> validate_required([:name, :filename])
  end
end

Our last step is to prepare the form for uploads. We first add [{:multipart, true}] to the form_for/4 call. We also change the :filename input field to use file_input instead of text_input like so:

<!-- lib/app_web/templates/image/form.eex -->
<%= form_for @changeset, @action, [{:multipart, true}], fn f -> %>
  <%= if @changeset.action do %>
    <div class="alert alert-danger">
      <p>Oops, something went wrong! Please check the errors below.</p>
    </div>
  <% end %>

  <div class="form-group">
    <%= label f, :name, class: "control-label" %>
    <%= text_input f, :name, class: "form-control" %>
    <%= error_tag f, :name %>
  </div>

  <div class="form-group">
    <%= label f, :filename, class: "control-label" %>
    <%= file_input f, :filename, class: "form-control" %>
    <%= error_tag f, :filename %>
  </div>

  <div class="form-group">
    <%= submit "Submit", class: "btn btn-primary" %>
  </div>
<% end %>

Remove any references to @image.filename in app/lib/app_web/templates/image/index.html.eex and app/lib/app_web/templates/image/view.html.eex as this will trigger an error like (Protocol.UndefinedError) protocol Phoenix.HTML.Safe not implemented for %{file_name...

Run mix ecto.migrate to run the generated migrations and mix phoenix.server. You should be able to upload files to Google Cloud Storage.

Menu

Phoenix 1.3 note

Uploader

Phoenix