Aggregate Environment Variables for Deployment with Elixir Scripting

Sometimes a script is all you need!

Cassava runs in multiple environments as part of our release cycle. Each deployment requires a suite of variables to control the runtime configuration.

Today I will discuss how we use Elixir scripting to manage our environment variables and why this decision was chosen over an alternate approach.

The env-cmd Approach

Before presenting my preferred approach, I want to discuss an alternative that might favor some.

Back in my Ruby on Rails days, I used the Foreman gem with ease to manage my environment variables.

As I explored alternatives to use with Elixir, I came across the env-cmd Node package which is discussed in depth on Digital Ocean’s blog.

First, add the node library to the devDependencies section of your package.json.

"env-cmd": "^10.0.1"

Then run the usual npm install step.

cd assets && npm install

Create a .env-cmdrc file at the root of your project with all the environments that you need to support. For example:

{
  "common": {
    "DB_USER": "common_username",
    "DB_PSSWD": "common_password"
  },
    "dev": {
  },
    "prod": {
    "DB_USER": "prod_abc",
    "DB_PSSWD": "prod_def"
  }
}

In this contrived example, a base foundation is defined on top of which dev and prod inherit. Variables specified in higher environments supersede those in lower ones.

Next, add a log_env.js file to the root of your project to generate the resultant .env from the merge variables.

let variables = [
    "DB_USER",
    "DB_PSSWD"
]
variables.forEach(variable => {
  console.log('export %s="%s"', variable, process.env[variable]);
})

Lastly, run the env-cmd command to target the desired environments.

assets/node_modules/.bin/env-cmd
  -r ./.env-cmdrc
  --environments common,dev node log_env.js

This command generates an output file you can place on a target machine.

export DB_USER="common_username"
export DB_PSSWD="common_password"

One of the downsides of this approach is that you must add an extra node dependency to your solution, which might not fit your taste, especially if you are running Phoenix 1.6 and above.

Another con of this approach is the requirement to keep the log_env.js in sync with the .env-cmdrc, as adding a new variable to the rc file is not enough to ensure its inclusion in the resulting environment file.

These drawbacks were further compounded by my need to concisely specify the Slack workspace in addition to the deployment target.

So, what is a guy supposed to do when the stars don’t align quite as you wish?

The answer is to roll your own solution!

Elixir Script Approach (recommended)

Elixir is not only an excellent framework for building distributed applications but also a first-class scripting language for one-off tasks.

With this in mind, let’s start assembling a .env.exs script.

defmodule Cassava.Env do

  @doc false
  def make(target, workspace), do: nil

  supported_targets = ["dev", "prod"]

  supported_workspaces = ["primary", "secondary"]

  [target, workspace] = System.argv()

  unless target not in supported_targets and
         workspace not in supported_workspaces do
    config =
      target
      |> String.to_existing_atom()
      |> Cassava.Env.make(String.to_existing_atom(workspace))
      |> List.keysort(0)

    File.open(".env", [:write], fn file ->
      config
      |> Enum.each(fn {key, value} ->
        IO.puts(file, "export #{String.upcase(Atom.to_string(key))}=\"#{value}\"")
      end)
    end)
end

We have a script that allows us to build a suite of environment variables for a given target and workspace. This script can be invoked in one of two ways:

elixir env.exs <target> <workspace>
env.exs <target> <workspace>

The second flavor is made possible by changing the mode of the script to an executable and adding a shebang, as seen in this fabulous Thoughtbot tutorial.

chmod +x .env.exs

Before proceeding, you will want to add this script to your .gitignore file to ensure that sensitive credentials are not leaked.

Our script does not do much at the moment so let’s address this by adding implementations for the make function.

@doc false
def make(:dev = target, workspace) do
  target_vars(:common)
  |> override(target, workspace)
end

def make(:prod = target, workspace) do
  target_vars(:common)
  |> override(target, workspace)
end

defp target_vars(target) do
  target
  |> case do
    :common -> common_vars()
    :dev -> dev_vars()
    :prod -> prod_vars()
  end
end

defp common_vars, do: [
  db_user: "common_username",
  db_psswd: "common_password",
]

defp dev_vars, do: []

defp prod_vars, do: [
  db_user: "prod_abc",
  db_psswd: "prod_def",
]

defp override(vars, target, workspace),
  do:
    vars
    |> Keyword.merge(target_vars(target))
    |> Keyword.merge(workspace_vars(workspace, target))


defp workspace_vars(workspace, target) do
  workspace
  |> case do
    :primary -> configure_primary_workspace(target)
    :secondary -> configure_secondary_workspace(target)
  end
end

defp configure_primary_workspace(:dev),
  do: [
    slack_client_id: "primary_dev_client_id",
    slack_client_secret: "primary_dev_client_secret",
  ]

defp configure_primary_workspace(:prod),
  do: [
    slack_client_id: "primary_prod_client_id",
    slack_client_secret: "primary_prod_client_secret",
  ]

defp configure_secondary_workspace(:dev), do: [...]

defp configure_secondary_workspace(:prod), do: [...]

Here we have the environment variables specific to each target and workspace. The target vars are composable as before. There is also a workspace specification in concert with the deployment target.

A sample command of the form .env.exs dev primary yields:

export DB_USER="common_username"
export DB_PSSWD="common_password"
export SLACK_CLIENT_ID="primary_dev_client_id"
export SLACK_CLIENT_SECRET="primary_dev_client_secret"

Summary

So there you have it.

An example of using the powerful scripting language available in Elixir to generate environment variables to suit a particular use case. Scripts are also excellent whenever you need to pre or post-process data.

Your feedback is welcomed as always.

Michael