An Opinionated Way of Organizing GraphQL APIs using Absinthe and Phoenix

REST has been the predominant architectural choice for API development on the web. GraphQL being the new kid in town has been continually gaining traction over the last few years as an alternative way to build APIs. GraphQL is - "A query language for your API". While Phoenix is the de-facto web framework for developing web applications in Elixir, Absinthe is the de-facto library in Elixir for developing GraphQL APIs. The objective of this article is to suggest a way of organizing GraphQL APIs in Elixir using Absinthe/Phoenix that is more practical and suitable for real-world projects.

To do that we have decided to take source code from the book - "Craft GraphQL APIs in Elixir with Absinthe" and refactor, in some cases re-write this code to achieve something that can be used in real-world large-scale projects. If you are an Elixir web application developer you should definitely read this book. This article is not a substitute for the materials presented in the book. It rather takes the concepts and codes presented in the book and organizes them from a different point of view which is practical for real-world projects.

You can get the source code from the book here - Source Code (zip file).

The final version of the code presented in this article is available here - https://github.com/imeraj/absinthe_graphql

Domain Model

Before delving into GraphQL concepts and code organization, we will first explain the domain model we are going to use from the book mentioned above. We are not going to explain each migration or schema as they can be easily explored from the completed source code repository. The app in the book is called plate_slate. We will continue to use the same name.

However, the generated Entity Relationship Diagram looks like this -

(above diagram was generated from completed source code [2] using ecto_erd hex)

Here, we have a few tables from 3 different contexts -

Accounts context contains - User schema which is used to model users (both customers and employees in the system). The role column is used to differentiate between employee and customer roles.
Menu context contains - Category, Item, and ItemTag schemas. Each menu item belongs to a category. Items can have many tags, likewise, tags belong to many items. This many to many association is implemented using the item_taggings join table using a join_thtough association.
Ordering context contains - Order and Item schema. This Item is an embedded schema used to represent items belonging to an Order. Each user can have many orders placed.

It should be possible to follow the organization of the schemas and contexts above exploring the codes presented in the below directory structure -

(Re)-organizing GraphQL APIs

In this section, we will refactor the book code and re-organize the GraphQL APIs. GraphQL has three main concepts for developing APIs -

Query - Used to query resource/data
Mutation - Used to create/update/delete resources in the system
Subscription - Used to listen for updates for resources throughout life-cycle

In this section, we will present how we reorganized the code to make it better suited for real-world projects by presenting example API code for each of queries, mutations, and subscriptions. We will also add authentication and authorizations to the APIs. In the end, our main schema.ex should be pretty neat and looking at the schema it should be possible to easily navigate the entire codebase.

Schema and Directory Structure

The main GraphQL schema file used by Absinthe is under lib/plate_slate_web/schema.ex.

The rest of our GraphQL code will reside under lib/late_slate_web/graphql having the below directory structure -

Here -

directives - contains the absinthe directive used to implement feature flags. (Explaining how feature flags work and are implemented using directives is out of the scope of this article but for the sake of completeness we included it in thecompleted code. Maybe this will be a topic for a future article :)).
input_types - some of our APIs (queries/mutations/subscriptions) will require custom input types and input-type definitions are stored under the input_types directory.
middlewares - middleware contains various custom middleware we use throughout the project. Particularly, we will use middleware for authorization purposes.
payloads - responses from APIs if complex are not placed in the resolvers. Instead, this separate directory contains all the necessary payloads returned by different APIs.
queries - contains only the queries
mutations - contains only the mutations
subscriptions - contains only the subscriptions
resolvers - contains custom resolvers for the different queries/mutations/subscriptions
types - contains various scalar types for example custom enum etc and various basic types used by APIs

In the following sections, we will present one from each category of queries, mutations, and subscriptions to present how the code is organized.

Queries

As noted in schema.ex, we have the below queries in our system which are not under any authentication or authorization meaning open to public.

  query do
    import_fields(:menu_query)
    import_fields(:search_query)
  end

Here -

menu_query - is used to query menu items along with categories
search_query - is used to search through menu items and categories

We will go over the menu_query in this section.

In the terminology of Absinthe each query is a field in the schema. Necessary types and fields are then imported into theschema file.

queries/menu_query.ex contains an implementation of menu_query and it looks as below -

defmodule PlateSlateWeb.Graphql.Queries.MenuQuery do
  @moduledoc false

  use Absinthe.Schema.Notation
  use Absinthe.Relay.Schema.Notation, :modern

  alias PlateSlateWeb.Graphql.Resolvers.MenuResolver

  @desc "Get a menu of items"
  object :menu_query do
    connection field :menu_items, node_type: :menu_item do
      arg(:filter, :menu_item_filter, description: "Filter for menu items")

      arg(:order, :sort_order,
        default_value: :asc,
        description: "Menu items sort order. Default ASC"
      )

      resolve(&MenuResolver.menu_items/3)
    end
  end
end

Here we used Absinthe Relay Connection for cursor pagination for menu items.

The resolver function for this query resides under resolvers directory in menu_resolver.ex and looks as below -

defmodule PlateSlateWeb.Graphql.Resolvers.MenuResolver do
  @moduledoc false
  alias PlateSlate.Menu

  def menu_items(_, args, _) do
    Absinthe.Relay.Connection.from_query(
      Menu.items_query(args),
      &PlateSlate.Repo.all/1,
      args
    )
  end
end

If you want to learn more about how Absinthe Relay works, you can refer to [3]. But this resolver returns a list of menu items.

This query also takes two arguments -

filter - implemented as a custom type menu_item_filter

Implemented as input type under input_types/menu_item_filter.ex as -

defmodule PlateSlateWeb.Graphql.InputTypes.MenuItemFilter do
  @moduledoc false

  use Absinthe.Schema.Notation

  @desc "Filtering options for the menu items list"
  input_object :menu_item_filter do
    @desc "Matching a name"
    field :name, :string

    @desc "Matching a category name"
    field :category, :string

    @desc "Matching a tag"
    field :tag, :string

    @desc "Priced above a value"
    field :price_above, :float

    @desc "Priced below a value"
    field :price_below, :float

    @desc "Added to the menu before this date"
    field :added_before, :date

    @desc "Added to the menu after this date"
    field :added_after, :date
  end
end

order - implemented in types/enums/sort_order_enum.ex taking asc and desc order values -

            defmodule PlateSlateWeb.Graphql.Types.Enums.SortOrderEnum do
              @moduledoc false

              use Absinthe.Schema.Notation

              @desc "Sorting order"
              enum :sort_order do
                value(:asc, description: "Ascending")
                value(:desc, description: "Descending")
              end
            end

Once we define the types and fields, we need to import those in the main schema.ex file using the appropriate import macro. For example, the sort_order enum and filter type are imported using the below imports in schema.ex -

  import_types(PlateSlateWeb.Graphql.Types.Enums.SortOrderEnum)
  import_types(PlateSlateWeb.Graphql.InputTypes.MenuItemFilter)

We can use this query for example to get the first 3 items sorted in ascending name order and using a filter with a price above 4.5 as below -

Note: To run the query locally you can visit - localhost:4000/graphiql

Mutations

We have the below mutations in our system -

mutation do
  import_fields(:signup_mutation)
  import_fields(:login_mutation)
  import_fields(:create_menu_item_mutation)
  import_fields(:place_order_mutation)
  import_fields(:ready_order_mutation)
  import_fields(:complete_order_mutation)
end

Here -

signup_mutation - is used to signup a user in the system as either an employee or a customer
login_mutation - is used to log in as an employee or customer
create_menu_item_mutation - is used to create a menu item in the system. Only allowed by employees
place_order_mutation - is used used to place an order
ready_order_mutation - is used to mark an order ready by an employee. Used to update our subscriptions described later below.
complete_order_mutation - is used to mark an order completed by an employee. Used to update our subscriptions described later below.

We will present create_menu_item_mutation here which is implemented under mutations/create_menu_item_mutation.ex as below -

defmodule PlateSlateWeb.Graphql.Mutations.CreateMenuItemMutation do
  @moduledoc false

  use Absinthe.Schema.Notation

  alias PlateSlateWeb.Graphql.Resolvers.CreateMenuItemResolver

  object :create_menu_item_mutation do
    @desc "Create menu item"
    field :create_menu_item, :create_menu_item_payload do
      arg(:input, non_null(:create_menu_item_input))
      middleware(PlateSlateWeb.Middlewares.Authorize, "employee")
      resolve(&CreateMenuItemResolver.create_menu_item/3)
    end
  end
end

Similar to our query, this mutation has a custom resolver defined in resolvers/create_menu_item_resolver.ex as below -

defmodule PlateSlateWeb.Graphql.Resolvers.CreateMenuItemResolver do
  @moduledoc false

  alias PlateSlate.Menu

  def create_menu_item(_, %{input: params}, _) do
    with {:ok, menu_item} <- Menu.create_item(params) do
      {:ok, menu_item}
    end
  end
end

Also, as seen from the above Mutation diagram, this mutation returns a payload of type create_menu_item_payload-

field :create_menu_item, :create_menu_item_payload do

create_menu_item_payload is defined in payloads/create_menu_item_payload as below -

defmodule PlateSlateWeb.Graphql.Payloads.CreateMenuItemPayload do
  @moduledoc false

  use Absinthe.Schema.Notation

  @desc "Menu Item payload"
  union :create_menu_item_payload do
    types([:menu_item, :errors])

    resolve_type(fn
      %PlateSlate.Menu.Item{}, _ ->
        :menu_item

      %{errors: _}, _ ->
        :errors

      _, _ ->
        nil
    end)
  end
end

This payload is a union of menu_item and errors types. So if the resolver succeeds it will return menu_item otherwise it will return errors type.

We follow this general convention in the mutations to always return the union of expected return types and errors so that clients can process errors if any.

We defined errors in types/error.ex as below -

defmodule PlateSlateWeb.Graphql.Types.ErrorType do
  use Absinthe.Schema.Notation

  @desc "mutation errors"
  object :errors do
    field :errors, list_of(:error)
  end

  @desc "mutation error type"
  object :error do
    field :key, non_null(:string)
    field :message, non_null(:string)
  end
end

We can use this mutation to create a menu item as an employee as below -

Note: This mutation requires a signed-up employee. So you will need to use the signup and login mutation and use Authorization header to include Bearer token to test it successfully.

If you have any validation errors, you will get the below response thanks to the way we set up the payload -

There is a middleware used to authorize only employees to run this mutation -

 middleware(PlateSlateWeb.Middlewares.Authorize, "employee")

We will explain middleware later but if you are not an employee or forgot to include proper Bearer token, you should receive unauthorized error -

Subscriptions

If you have got this far, understanding the subscription code should be easy. Like queries and mutations, our system has a few subscriptions -

subscription do
  import_fields(:new_order_subscription)
  import_fields(:update_order_subscription)
end

Here -

new_order_subscription - is used to monitor incoming new orders
update_order_subscription - is used to monitor status updates on an order by customers

We will explain new_order_subscription here which is implemented in subscriptions/new_order_subscription.ex as below -

defmodule PlateSlateWeb.Graphql.Mutations.NewOrderSubscription do
  @moduledoc false

  use Absinthe.Schema.Notation
  alias PlateSlateWeb.Graphql.Resolvers.NewOrderResolver

  object :new_order_subscription do
    @desc "New order subscription"
    field :new_order, :order do
      config(fn _args, %{context: context} ->
        case context[:current_user] do
          %{role: "customer", id: id} ->
            {:ok, topic: id}

          %{role: "employee"} ->
            {:ok, topic: "*"}

          _ ->
            {:error, "unauthorized"}
        end
      end)

      resolve(&NewOrderResolver.new_order/3)
    end
  end
end

Explaining how subscriptions work in Absinthe is out of the scope of this article. For that, we recommend you read the book mentioned in [1]. But to give an idea, you need to subscribe to a topic. Here customer can monitor only their orders. So, the topic included include id - {:ok, topic: id}. Employees can monitor any order. So the topic includes asterix - {:ok, topic: "*"} .

I will leave to go over the resolver code as an exercise at this point.

After running this subscription, we should see the message on the right side the below screenshot -

Note: Subscription runs over websocket. So to run this subscription locally, we need to hit this endpoint - ws://localhost:4000/socket. If you are using the subscription as an employee include an Authorization header with an employee Bearer token.

Now, we need to place an order using place_order_mutation as below -

Once the mutation succeeds, thanks to the below line of code in resolvers/place_order_resolver.ex which notifies subscribers of any new order by publishing to the right topics -

with {:ok, order} <- Ordering.create_order(params) do
  Absinthe.Subscription.publish(PlateSlateWeb.Endpoint, order,
    new_order: [order.customer_id, "*"]
  )

you should see the below update in our previously pending subscription window -

Middlewares

We used Absinthe middleware for various purposes. Three of our middlewares are located under plate_slate_web/graphql/middlewares directory -

Here -

authorize middleware - is used for authorization purposes
changeset_errors middleware - is used to convert and wrap changeset errors to key/value pairs
debug middleware - is used to print some additional debug information

authorize middleware is implemented as below -

defmodule PlateSlateWeb.Middlewares.Authorize do
  @moduledoc false

  @behaviour Absinthe.Middleware

  def call(resolution, role) do
    with %{current_user: current_user} <- resolution.context,
         true <- correct_role?(current_user, role) do
      resolution
    else
      _ -> Absinthe.Resolution.put_result(resolution, {:error, "unauthorized"})
    end
  end

  defp correct_role?(%{role: role}, role), do: true
  defp correct_role?(%{}, :any), do: true
  defp correct_role?(_, _), do: false
end

This particularly checks if the correct user (employee or customer) is accessing the API or resource. The behaviour we implemented is Absinthe.Middleware which has mandatory call callback. Once we have this implemented we can use it in mutations as below using middleware/2 macro -

middleware(PlateSlateWeb.Middlewares.Authorize, "employee")

Also, we have registered our other middlewares including the default ones using middleware/3 callback function in schema.ex -


def middleware(middleware, field, object) do
  middleware
  |> apply(:errors, field, object)
  |> apply(:debug, field, object)
end

defp apply(middleware, :errors, _field, %{identifier: :mutation}) do
  middleware ++ [Middlewares.ChangesetErrors]
end

defp apply(middleware, :debug, _field, _object) do
  if System.get_env("DEBUG"),
    do: [{Middlewares.Debug, :start}] ++ middleware,
    else: middleware
end

defp apply(middleware, _, _, _), do: middleware

To learn more about how middlewares work refer to [5].

Miscellaneous

For subscriptions, we need to configure and implement certain callbacks related to websockets. We did this in plate_slate_web/channels/user_socker.ex as below -

defmodule PlateSlateWeb.UserSocket do
  use Phoenix.Socket
  use Absinthe.Phoenix.Socket, schema: PlateSlateWeb.Schema

  @doc """
  For graphql subscriptions to work must implement below callbacks
  """

  @impl true
  def connect(params, socket, _connect_info) do
    socket =
      Absinthe.Phoenix.Socket.put_options(socket,
        context: %{current_user: find_current_user(params)}
      )

    {:ok, socket}
  end

  @impl true
  def id(_socket), do: nil

  defp find_current_user(params) do
    with "Bearer " <> token <- params["Authorization"],
         {:ok, user} <- PlateSlateWeb.Authentication.verify(token) do
      user
    else
      _ -> %{}
    end
  end
end

Explaining how websockets and Phoenix channels work is out of the scope of this article. But you can refer to [1] and [4] if you want to learn more details. But what is important from our API development perspective is that we verify the Bearer token and retrieve proper user in our system thereby setting current_user on the socket.

Completed Schema

Our final completed schema is simplified and neat. It looks like this (with slight editing to exclude the directive part) -

defmodule PlateSlateWeb.Schema do
  @moduledoc false

  use Absinthe.Schema
  use Absinthe.Relay.Schema, :modern

  alias PlateSlateWeb.Middlewares
  alias PlateSlateWeb.Dataloader

  import_types(PlateSlateWeb.Graphql.Types.Scalars.Date)
  import_types(PlateSlateWeb.Graphql.Types.Scalars.Decimal)

  import_types(PlateSlateWeb.Graphql.Types.Enums.SortOrderEnum)
  import_types(PlateSlateWeb.Graphql.Types.Enums.RoleEnum)

  import_types(PlateSlateWeb.Graphql.Types.ErrorType)
  import_types(PlateSlateWeb.Graphql.Types.UserType)
  import_types(PlateSlateWeb.Graphql.Types.MenuItemType)
  import_types(PlateSlateWeb.Graphql.Types.CategoryType)
  import_types(PlateSlateWeb.Graphql.Types.OrderType)
  import_types(PlateSlateWeb.Schema.Types.SessionType)

  import_types(PlateSlateWeb.Graphql.InputTypes.MenuItemFilter)
  import_types(PlateSlateWeb.Graphql.InputTypes.SignupInput)
  import_types(PlateSlateWeb.Graphql.InputTypes.LoginInput)
  import_types(PlateSlateWeb.Graphql.InputTypes.CreateMenuItemInput)
  import_types(PlateSlateWeb.Graphql.InputTypes.PlaceOrderInput)

  import_types(PlateSlateWeb.Graphql.Queries.MenuQuery)
  import_types(PlateSlateWeb.Graphql.Queries.SearchQuery)

  import_types(PlateSlateWeb.Graphql.Mutations.SignupMutation)
  import_types(PlateSlateWeb.Graphql.Mutations.LoginMutation)
  import_types(PlateSlateWeb.Graphql.Mutations.CreateMenuItemMutation)
  import_types(PlateSlateWeb.Graphql.Mutations.PlaceOrderMutation)
  import_types(PlateSlateWeb.Graphql.Mutations.ReadyOrderMutation)
  import_types(PlateSlateWeb.Graphql.Mutations.CompleteOrderMutation)

  import_types(PlateSlateWeb.Graphql.Mutations.NewOrderSubscription)
  import_types(PlateSlateWeb.Graphql.Mutations.UpdateOrderSubscription)

  import_types(PlateSlateWeb.Graphql.Payloads.SignupPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.LoginPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.SearchPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.CreateMenuItemPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.PlaceOrderPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.ReadyOrderPayload)
  import_types(PlateSlateWeb.Graphql.Payloads.CompleteOrderPayload)

  def context(ctx), do: Map.put(ctx, :loader, Dataloader.dataloader())

  def plugins, do: [Absinthe.Middleware.Dataloader | Absinthe.Plugin.defaults()]

  def middleware(middleware, field, object) do
    middleware
    |> apply(:errors, field, object)
    |> apply(:debug, field, object)
  end

  defp apply(middleware, :errors, _field, %{identifier: :mutation}) do
    middleware ++ [Middlewares.ChangesetErrors]
  end

  defp apply(middleware, :debug, _field, _object) do
    if System.get_env("DEBUG"),
      do: [{Middlewares.Debug, :start}] ++ middleware,
      else: middleware
  end

  defp apply(middleware, _, _, _), do: middleware

  query do
    import_fields(:menu_query)
    import_fields(:search_query)
  end

  mutation do
    import_fields(:signup_mutation)
    import_fields(:login_mutation)
    import_fields(:create_menu_item_mutation)
    import_fields(:place_order_mutation)
    import_fields(:ready_order_mutation)
    import_fields(:complete_order_mutation)
  end

  subscription do
    import_fields(:new_order_subscription)
    import_fields(:update_order_subscription)
  end
end

In the completed code, we have also used dataloader to avoid N+1 queries in our APIs as per the book code [1]. Since we are more focused on organizing codebase over explaining details, we skipped explaining how dataloader works in detail.

Unit Testing

We implemented some unit tests located under test directory -

These have been updated from the book code to fit the conventions used in the article.

Convention Used

In general, we have followed below conventions to organize the codebase better -

Keep schema as neat as possible only importing types and fields, setting up middleware and plugins and moving the definitions of queries, mutations, subscriptions, middlewares etc. into their own folders and files.
Top-level resolvers for mutations/queries/subscriptions usually have their own modules defined in separate files. If the resolver is too simple in some cases they may be inlined though instead of separate files.
In case of mutations, if the input is complex we always wrap them in input object which is custom defined type.
Each mutation returns a payload which is a union of the expected types and errors so that clients can use fragments to handle both happy paths as well as any error scenarios.
Field-level resolvers are inlined in the same file as the object they are part of. For example, the category resolver on MenuItemType is defined in types/menu_item_type.ex as below -

    field :category, :category do
      resolve(fn menu_item, _, %{context: %{loader: loader}} ->
        loader
        |> Dataloader.load(Menu, :category, menu_item)
        |> on_load(fn loader ->
           category = Dataloader.get(loader, Menu, :category, menu_item)
           {:ok, category}
        end)
      end)
    end

Conclusion

In this article, we took the sample code from Craft GraphQL APIs in Elixir with Absinthe book and organized it such that code is cleaner, easier to navigate and facilitates implementing new APIs faster. Due to the brevity of the article, some of the Absinthe and GraphQL concepts were not explained in detail but you can always refer to the official Absinthe website and also the book in [1] for more information. The only downside I can think of this approach is it creates several small small files but that's a small price to pay for the benefits it provides.

The full source code for the example in this article is here - https://github.com/imeraj/absinthe_graphql

References

Craft GraphQL APIs in Elixir with Absinthe - https://pragprog.com/titles/wwgraphql/craft-graphql-apis-in-elixir-with-absinthe/
Complete source code from the article example project - https://github.com/imeraj/absinthe_graphql
Absinthe Replay - https://hexdocs.pm/absinthe/relay.html
Real-Time Phoenix - https://pragprog.com/titles/sbsockets/real-time-phoenix
Absinthe Middleware - https://hexdocs.pm/absinthe/Absinthe.Middleware.html
Absinthe - https://hexdocs.pm/absinthe/overview.html

For more elaborate and in-depth future technical posts please follow me here or subscribe to my newsletter.

Tech Musings

Tech Musings

An Opinionated Way of Organizing GraphQL APIs using Absinthe and Phoenix

Domain Model

(Re)-organizing GraphQL APIs

Schema and Directory Structure

Queries

Mutations

Subscriptions

Middlewares

Miscellaneous

Completed Schema

Unit Testing

Convention Used

Conclusion

References

An Opinionated Way of Organizing GraphQL APIs using Absinthe and Phoenix

Domain Model

(Re)-organizing GraphQL APIs

Schema and Directory Structure

Queries

Mutations

Subscriptions

Middlewares

Miscellaneous

Completed Schema

Unit Testing

Convention Used

Conclusion

References

Did you find this article valuable?