Matt Pruitt

RestClient: A Phoenix LiveView Experiment

2020-09-27T00:00:00Z

Phoenix LiveView is a great new addition to the Phoenix ecosystem. I wanted to make a small sample project to see how close we can get to the single page application that’s achievable with JavaScript frameworks like Vue or React. I thought that a REST client would be a fun project, so I spent the weekend trying to see how much I could get done in a few hours of work. I’ve open sourced the project as an example for others to learn from. Check it out and let me know what you think!

Demo
Source

Use Ecto Embedded Schemas to Back Phoenix Forms

2020-09-21T00:00:00Z

Phoenix has been pushing separation of concerns through it’s defaults by generating project and project_web directories and defaulting generators to use contexts. Since we should be separating our web application logic from our core application logic, I have started using embedded ecto schemas to create form modules in my projects.

This enables us to to separate our frontend data model representations from our schema data model. One of the most common suggestions for implementing a proper user authentication system is to add a virtual password field to the schema. Now, any time we get a user from the database, we have an empty password field in our struct! Wouldn’t it be better if we could encrypt the password in the registration form and pass it directly to our schema’s changeset?

Once you’ve started separating these data models, it opens up a lot more options to accept the input the user expects but save it in our database like our data model expects.

In this example, we’re going to make a product creation form for an online store that accepts the price of the product as a decimal (e.g. “$49.99”) but saves it in our database as an integer (e.g. 4999).

Let’s start with the new product action. Let’s make a ProductForm module that gives us an empty form. We’ll use Ecto’s embedded_schema to give us an easy to use struct that isn’t backed by a database table.

defmodule StoreWeb.ProductForm do
  use Ecto.Schema

  import Ecto.Changeset

  @required [:name, :price]
  @attributes @required ++ [:description]
  @primary_key false

  embedded_schema do
    field :name, :string
    field :price, :string
    field :description, :string
  end

  @spec form :: Ecto.Changeset.t()
  def form, do: cast(%__MODULE__{}, %{}, @attributes)
end

And now, in our controller, we can pass our function down to the template to be rendered by form_for.

defmodule StoreWeb.ProductController do
  use StoreWeb, :controller

  @spec new(Plug.Conn.t(), map()) :: Plug.Conn.t()
  def new(conn, _params) do
    changeset = StoreWeb.ProductForm.form()

    render(conn, "new.html", changeset: changeset)
  end
end

The form works, but now we need to hook it up to our create action. Our store’s data API accepts a map of attributes to the create_product function. So let’s add a function to handle that.

defmodule StoreWeb.ProductForm do
  use Ecto.Schema

  import Ecto.Changeset

  @required [:name, :price]
  @attributes @required ++ [:description]
  @primary_key false

  embedded_schema do
    field :name, :string
    field :price, :string
    field :description, :string
  end

  @spec form :: Ecto.Changeset.t()
  def form, do: cast(%__MODULE__{}, %{}, @attributes)

  @spec form :: Ecto.Changeset.t()
  def form, do: form(%{})

  @spec form(map()) :: Ecto.Changeset.t()
  def form(attributes) do
    cast(%__MODULE__{}, attributes, @attributes)
  end

  @spec attributes(Ecto.Changeset.t()) :: {:ok, map()} | {:error, Ecto.Changeset.t()}
  def attributes(form) do
    applied = apply_action(form, :create)

    case applied do
      {:ok, struct} -> {:ok, Map.from_struct(struct)}
      other -> other
    end
  end
end

Now let’s plug this in to our create action.

defmodule StoreWeb.ProductController do
  use StoreWeb, :controller

  @spec create(Plug.Conn.t(), map) :: Plug.Conn.t()
  def create(conn, %{"product_form" => product_params}) do
    form = StoreWeb.ProductForm.form(product_params)

    with {:ok, attributes} <- StoreWeb.ProductForm.attributes(form),
         {:ok, product} <- Store.create_product(attributes) do
      conn
      |> put_flash(:info, "Product created successfully.")
      |> redirect(to: Routes.product_path(conn, :show, product))
    else
      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "new.html", changeset: changeset)
    end
  end
end

We’ve got a problem now: our attributes are passing the price as a string but our API is expecting an integer. Let’s fix this and add some more validations:

defmodule StoreWeb.ProductForm do
  # omitted for length

  @price_regex ~r/^\$?(?\d*)(\.?(?\d{1,2}))?$/

  @spec attributes(Ecto.Changeset.t()) :: {:ok, map()} | {:error, Ecto.Changeset.t()}
  def attributes(form) do
    applied =
      form
      |> validate_required(@required)
      |> validate_format(:price, @price_regex, message: "must be a price ($19.99)")
      |> apply_action(:create)

    case applied do
      {:ok, struct} ->
        attributes =
          struct
          |> Map.from_struct()
          |> price_to_int()

        {:ok, attributes}

      other ->
        other
    end
  end

  defp price_to_int(%{price: price} = attributes) do
    [cents, dollars] = Regex.run(@price_regex, price, capture: :all_names)

    int_dollars = if dollars == "", do: 0, else: String.to_integer(dollars) * 100
    int_cents = if cents == "", do: 0, else: String.to_integer(cents)
    int_price = int_dollars + int_cents

    %{attributes | price: int_price}
  end
end

Now we need to use the form in our edit and update actions. We’ll want to make sure to display our price as a decimal string again. Here’s our final form module.

defmodule StoreWeb.ProductForm do
  use Ecto.Schema

  import Ecto.Changeset

  @price_regex ~r/^\$?(?\d*)(\.?(?\d{1,2}))?$/

  @required [:name, :price]
  @attributes @required ++ [:description]
  @primary_key false

  embedded_schema do
    field :name, :string
    field :price, :string
    field :description, :string
  end

  @spec form :: Ecto.Changeset.t()
  def form, do: form(%{})

  @spec form(map() | %Store.Product{}) :: Ecto.Changeset.t()
  def form(%_{} = struct), do: form(struct, %{})

  def form(attributes) do
    form(%__MODULE__{}, attributes)
  end

  @spec form(%__MODULE__{} | %Store.Product{}, map()) :: Ecto.Changeset.t()
  def form(%__MODULE__{} = form, attributes) do
    form
    |> int_to_price()
    |> cast(attributes, @attributes)
  end

  def form(%_{} = struct, attributes) do
    merged_attributes =
      struct
      |> Map.from_struct()
      |> int_to_price()
      |> Map.merge(attributes)

    form(%__MODULE__{}, merged_attributes)
  end

  @spec attributes(Ecto.Changeset.t()) :: {:ok, map()} | {:error, Ecto.Changeset.t()}
  def attributes(form) do
    applied =
      form
      |> validate_required(@required)
      |> validate_format(:price, @price_regex, message: "must be a price ($19.99)")
      |> apply_action(:create)

    case applied do
      {:ok, struct} ->
        attributes =
          struct
          |> Map.from_struct()
          |> price_to_int()

        {:ok, attributes}

      other ->
        other
    end
  end

  defp price_to_int(%{price: price} = attributes) do
    [cents, dollars] = Regex.run(@price_regex, price, capture: :all_names)

    int_dollars = if dollars == "", do: 0, else: String.to_integer(dollars) * 100
    int_cents = if cents == "", do: 0, else: String.to_integer(cents)
    int_price = int_dollars + int_cents

    %{attributes | price: int_price}
  end

  defp int_to_price(%{price: int_price} = struct) when is_integer(int_price) do
    price =
      "$" <>
        to_string(Integer.floor_div(int_price, 100)) <>
        "." <> to_string(Integer.mod(int_price, 100))

    %{struct | price: price}
  end

  defp int_to_price(struct), do: struct
end

And here it is plugged in to our controller actions.

defmodule StoreWeb.ProductController do
  use StoreWeb, :controller

  @spec new(Plug.Conn.t(), map()) :: Plug.Conn.t()
  def new(conn, _params) do
    changeset = StoreWeb.ProductForm.form()

    render(conn, "new.html", changeset: changeset)
  end

  @spec create(Plug.Conn.t(), map) :: Plug.Conn.t()
  def create(conn, %{"product_form" => product_params}) do
    form = StoreWeb.ProductForm.form(product_params)

    with {:ok, attributes} <- StoreWeb.ProductForm.attributes(form),
         {:ok, product} <- Store.create_product(attributes) do
      conn
      |> put_flash(:info, "Product created successfully.")
      |> redirect(to: Routes.product_path(conn, :show, product))
    else
      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "new.html", changeset: changeset)
    end
  end

  @spec edit(Plug.Conn.t(), map) :: Plug.Conn.t()
  def edit(conn, %{"id" => id}) do
    product = Store.get_product!(id)
    changeset = StoreWeb.ProductForm.form(product)

    render(conn, "edit.html", product: product, changeset: changeset)
  end

  @spec update(Plug.Conn.t(), map) :: Plug.Conn.t()
  def update(conn, %{"id" => id, "product_form" => product_params}) do
    product = Store.get_product!(id)
    form = StoreWeb.ProductForm.form(product, product_params)

    with {:ok, attributes} <- StoreWeb.ProductForm.attributes(form),
         {:ok, product} <- Store.update_product(product, attributes) do
      conn
      |> put_flash(:info, "Product updated successfully.")
      |> redirect(to: Routes.product_path(conn, :show, product))
    else
      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "edit.html", product: product, changeset: changeset)
    end
  end
end

Monolith First in Elixir with Umbrella Projects

2017-03-22T00:00:00Z

Monolith First

Microservices are the hot new trend in the startup scene lately. If you’re building a new project should you start with a microservice architecture? Microservices tend to bring more overhead in devops, project setup, testing, and deployments.

The problems solved by services aren’t technical; they’re political. If you have a small team, you’re much better off starting with a monolith than a service architecture.

You shouldn’t start a new project with microservices, even if you’re sure your application will be big enough to make it worthwhile.

— Martin Fowler: MonolithFirst

Using umbrella projects you can get the benefits of monoliths and microservices.

Umbrella Projects

Elixir umbrella projects are a kind of meta application. It lets us split our code into multiple elixir applications and manage them at the top level using mix.

We can generate a new umbrella application with mix:

mix new --umbrella application_name

A typical umbrella project will look like this:

umbrella
|-- config
|-- mix.exs
|-+ apps
  |-+ web
    |-- config
    |-- mix.exs
    |-- lib
  |-+ data
    |-- config
    |-- mix.exs
    |-- lib
  |-+ business_logic
    |-- config
    |-- mix.exs
    |-- lib

The top level umbrella project has three internal projects: web, data, and business_logic.

Applications

Make a new application any time you would have made a new microservice. It should have a cohesive domain and own it’s own data storage (either database or an internal ETS store) .

In a typical web project I start with two applications: a web application (using Phoenix) for the HTTP interface, and a data application for the database interface. I’ll add new applications for business logic not related to either storage or web requests.

Releases

I use distillery to create releases when I’m ready to deploy. A simple release configuration will look like this:

release :umbrella do
  set version: "1.0.0"

  set applications: [
    web: :permanent,
    data: :permanent,
    business_logic: :permanent,
  ]
end

If I decide that the business logic application needs to be deployed on its own to more servers I can define two releases:

release :umbrella do
  set version: "2.0.0"

  set applications: [
    web: :permanent,
    data: :permanent,
  ]
end

release :business_logic do
  set version: "1.0.0"

  set applications: [
    business_logic: :permanent,
  ]
end

Now I can deploy umbrella and business_logic to different servers without changing any of the project structure.

Design Considerations

The one potential downside of umbrella applications is that they are treated like libraries. This means you still have to enforce decoupling at the module API level. With microservices each service only has access to its own code and is forced to communicate over an explicitly defined protocol.

Breaking the Monolith

As my team grows separate teams handle the business_logic and umbrella applications. It’s simple to extract the internal applications into a new umbrella project.

Because we have defined our module APIs, we can keep the current calls to the business logic service, but replace their implementation with something like an HTTP client.

Phoenix 1.3

Phoenix 1.3 includes a new umbrella generator:

mix phx.new --umbrella application_name

This creates a new project that is very close to the implementation detailed above. You can learn more by watching Chris McCord’s Lonestar ElixrConf 2017 Keynote.

Elixir Design Patterns - The Pipeline

2017-03-20T00:00:00Z

Functional programming is all about manipulating data and Elixir is no different. One of the main ways this is handled in Elixir is through the pattern that I call “The Pipeline”.

The Pipeline is defined by a collection of functions that take a data structure as an argument and return the same type of data structure. We can see an example of this in two of the central pieces of a Phoenix application Plug.Conn and Ecto.Changeset.

Plug.Conn

Plug.Conn is one of the central structs in Plug and represents the current connection. It includes both request and response data and is the main data structure involved in Phoenix controllers.

In a controller, we can build the response by passing the Conn through a series of functions. The following code adds the content type and a location header before finally sending the response with a created status and a json body.

import Plug.Conn

conn = %Plug.Conn{}

conn
|> put_resp_content_type("application/json")
|> put_resp_header("Location", "https://example.com/resource/1")
|> send_resp(:created, ~s({"id": 1})

Echo Changesets

Ecto.Changeset is one of my favorite features of an Elixir library. Because data is immutable in Elixir, we need a way to build up changes to our database objects before we save them. Several of the features of Rails are implemented on top of this simple feature: validations and before save hooks to name two.

The following example is also interesting because the first function in the pipeline, cast/3 converts a different data type into a changeset. We’ll then add a default status and validate the user’s age.

import Ecto.Changeset

user = %User{}

user
|> cast(%{name: "Frank", age: 10}, [:name, :age])
|> put_change(:status, "active")
|> validate_number(:age, greater_than: 13)

Building Our Own

Let’s see how we can build our own pipelines. Let’s build a tax engine: we’ll pass in an order object and calculate taxes based on the location.

defmodule Order do
  defstruct products: [], price: 0, taxes: 0, country: "", city: ""
end

Let’s add a function to add a product to the order. We’ll want to keep a running total of the price as we go.

def add_product(order, name, price) do
  products  = [name | order.products]
  new_price = order.price + price

  %{order | products: products, price: new_price}
end

We always want the main data structure to be the first argument to our functions: this enables us to use the pipe operator to build our pipelines.

Now that we have some products, we want to calculate the taxes.

def calculate_country_tax(%Order{country: "US"} = order) do
  taxes = order.taxes + order.price * 0.05

  %{order | taxes: taxes}
end

def calculate_country_tax(order), do: order

Notice how we have a default function definition that just returns the order argument. This means we only have to pattern match the options that matter.

San Francisco has a 10¢ flat tax for bags. Let’s write a function to calculate the city tax.

def calculate_city_tax(%Order{city: "San Francisco"} = order) do
  taxes = order.taxes + 0.1 + order.price * 0.07

  %{order | taxes: taxes}
end

def calculate_city_tax(order), do: order

Now we can put it all together:

import Order

order = %Order{country: "US", city: "San Francisco"}

order
|> add_product("laptop", 1000)
|> add_product("mouse", 50)
|> calculate_country_tax()
|> calculate_city_tax()

And once the pipeline has finished, we can see the following result:

%Order{city: "San Francisco", country: "US", price: 1050,
       products: ["mouse", "laptop"], taxes: 126.1}

How to Skip ActiveModel Callbacks

2017-01-14T00:00:00Z

Over the years, I’ve been increasingly opposed to the use of model callbacks. There are better ways to achieve the same result (such as service objects) and they always seem to get in the way.

At work, we’ve recently run into a issue when migrating some data where we didn’t want callbacks to run. The official method of skipping callbacks is to wrap your save in skip_callback and set_callback:

User.skip_callback(:save, :before, :send_welcome_email)
user.update_attributes(email: "user@example.com")
User.skip_callback(:save, :before, :send_welcome_email)

The biggest problems with this method is that it disables the callback globally and that it is not thread safe. It could have unintended side effects on the rest of the system. You also have to run this for each callback that you want to skip.

We have come up with a better solution that skips all callbacks only for the instance we are saving. This makes use of monkey patching, which is not ideal, but as long as you aren’t constantly updating your version of ActiveModel you should be ok.

module SkipCallbacks
  def run_callbacks(kind, *args, &block)
    yield(*args) if block_given?
  end
end

user.extend(SkipCallbacks)
user.update_attributes(email: "user@example.com")