fix phoenix/elixir recipe #1252

nbw · 2025-01-05T07:16:54Z

Summary

1️⃣ Lock file problem when deploying a Phoenix Elixir app
2️⃣ Improvements for the Phoenix/Elixir deploying recipe
3️⃣ Fixes issue with copied files for the start phase
3️⃣ Fixes issue with default values for ARGS in the docker file

👋 the FINAL dockerfile output is in the reference section ⬇️

1️⃣ Lock file problem

When deploying a Phoenix Elixir app to Railway there is a conflict with the lock file that cause the error: lock outdated: the lock is outdated compared to the options in your mix.exs[1]. This is because the we're copying too many files during multiple phases. The solution is to update the elixir dockerfile builder to only copy specific files.

https://elixirforum.com/t/docker-build-started-failing-unchecked-dependencies-for-environment-prod/28844/6

The Railway template for phoenix got around this problem by removing the lock file, which is not behaviour we want.

2️⃣ General updates to the Elixir build

run mix ecto.setup if ecto_sql is detected rather than postgrex
- postgrex is a postgres specific adapter, but there are other adapters (for mysql, etc). detecting ecto_sql is more generic
only copy the necessary folders and assets
build hex, rebar, and other deps using MIX_ENV=prod

Future considerations

The way we run elixir is with mix phx.server but actually creating a mix release is the better approach (it creates a binary that can be run). More details here about why release are better. In the future, we should move away from mix phx.server and use releases instead, but it's beyond the scope of this PR. Here's an example of an Elixir dockerfile that uses releases.

3️⃣ Other fixes

Issue: setting only_include_files in the start phase was ignored. Instead COPY . /app was always added to the generated dockerfile
- ✅ if only_include_files is set in the start phase, use it
Issue: default environment values aren't set in the Dockerfile
- ✅ set default values for docker ARGs. Example output: ARG ELIXIR_ERL_OPTIONS="+fnu" LANG="en_US.UTF-8" LANGUAGE="en_US:en" MIX_ENV="prod" NIXPACKS_METADATA="elixir"

References

[1] Lock file problem

[+] Building 113.3s (15/18)                                                 docker:orbstack
 => [internal] load build definition from Dockerfile                                   0.0s
 => => transferring dockerfile: 858B                                                   0.0s
 => [internal] load metadata for ghcr.io/railwayapp/nixpacks:ubuntu-1733184274         0.9s
 => [internal] load .dockerignore                                                      0.0s
 => => transferring context: 2B                                                        0.0s
 => [ 1/14] FROM ghcr.io/railwayapp/nixpacks:ubuntu-1733184274@sha256:91b1c0ad82b8a7a  0.0s
 => [internal] load build context                                                      0.6s
 => => transferring context: 82.02MB                                                   0.5s
 => CACHED [ 2/14] WORKDIR /app/                                                       0.0s
 => CACHED [ 3/14] COPY .nixpacks/nixpkgs-c5702bd28cbde41a191a9c2a00501f18941efbd0.ni  0.0s
 => [ 4/14] RUN nix-env -if .nixpacks/nixpkgs-c5702bd28cbde41a191a9c2a00501f18941ef  102.7s
 => [ 5/14] COPY . /app/.                                                              0.9s
 => [ 6/14] RUN  mix local.hex --force                                                 2.0s
 => [ 7/14] RUN  echo test                                                             0.2s
 => [ 8/14] RUN  mix local.rebar --force                                               1.7s
 => [ 9/14] RUN  mix deps.get --only prod                                              2.5s
 => [10/14] COPY . /app/.                                                              1.1s
 => ERROR [11/14] RUN  mix compile                                                     0.4s
------
 > [11/14] RUN  mix compile:
0.382 Unchecked dependencies for environment prod:
0.382 * heroicons (https://github.com/tailwindlabs/heroicons.git - v2.1.1)
0.382   lock outdated: the lock is outdated compared to the options in your mix.exs. To fetch locked version run "mix deps.get"
0.384 ** (Mix) Can't continue due to errors on dependencies
------
Dockerfile:26
--------------------
  24 |     # build phase
  25 |     COPY . /app/.
  26 | >>> RUN  mix compile
  27 |     RUN  mix assets.deploy
  28 |     RUN  mix ecto.setup
--------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c mix compile" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/orbstack/orbstack/uxra2k6hpddi2mzjzo7r97fjr
Error: Docker build failed

[2] Final Dockerfile

output of nixpacks build --dockerfile . on a phoenix elixir project that uses a database:

FROM ghcr.io/railwayapp/nixpacks:ubuntu-1733184274

ENTRYPOINT ["/bin/bash", "-l", "-c"]
WORKDIR /app/


COPY .nixpacks/nixpkgs-c5702bd28cbde41a191a9c2a00501f18941efbd0.nix .nixpacks/nixpkgs-c5702bd28cbde41a191a9c2a00501f18941efbd0.nix
RUN nix-env -if .nixpacks/nixpkgs-c5702bd28cbde41a191a9c2a00501f18941efbd0.nix && nix-collect-garbage -d


ARG ELIXIR_ERL_OPTIONS="+fnu" LANG="en_US.UTF-8" LANGUAGE="en_US:en" MIX_ENV="prod" NIXPACKS_METADATA="elixir"
ENV ELIXIR_ERL_OPTIONS=$ELIXIR_ERL_OPTIONS LANG=$LANG LANGUAGE=$LANGUAGE MIX_ENV=$MIX_ENV NIXPACKS_METADATA=$NIXPACKS_METADATA

# setup phase
# noop

# install phase
COPY config/config.exs /app/config/config.exs
COPY config/${MIX_ENV}.exs /app/config/${MIX_ENV}.exs
COPY mix.exs /app/mix.exs
COPY mix.lock /app/mix.lock
RUN  mix local.hex --force
RUN  mix local.rebar --force
RUN  mix deps.get --only prod

# build phase
COPY config/runtime.exs /app/config/runtime.exs
COPY assets /app/assets
COPY priv /app/priv
COPY config /app/config
COPY lib /app/lib
RUN  mix compile
RUN  mix assets.deploy





# start


CMD ["mix ecto.setup && mix phx.server"]

postgrex is just the adapter for postgres specifically. there are other ones for different database (mysql, etc.): https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.html#module-built-in-adapters

- only copy certain files in each phase so that there isn't a conflict with the lock file - move db migrations to happen at start time rather than bulding - add more ENV defaults like LANG and LANGUAGE

src/nixpacks/builder/docker/dockerfile_generation.rs

nbw · 2025-01-05T07:18:15Z

src/providers/elixir.rs

+
+        let mut start_phase = StartPhase::new(start_phase_cmd.to_string());
+        // Skip copying any additional files for the start phase
+        start_phase.only_include_files = Some(vec![]);


we don't want to copy any files before starting. if we do, the "lock" file problem occurs.

nbw · 2025-01-05T07:19:40Z

src/providers/elixir.rs

        if mix_exs_content.contains("assets.deploy") {
            build_phase.add_cmd("mix assets.deploy".to_string());
        }

-        if mix_exs_content.contains("postgrex") && mix_exs_content.contains("ecto") {


this won't work because the DATABASE_URL isn't available at this time. Railway's own docs suggesting adding mix ecto.setup to the start phase instead.

nbw · 2025-01-05T07:20:21Z

src/providers/elixir.rs

@@ -76,6 +96,8 @@ impl ElixirProvider {
    fn default_elixir_environment_variables() -> EnvironmentVariables {
        let var_map = vec![
            ("MIX_ENV", "prod"),
+            ("LANG", "en_US.UTF-8"),


LANG and LANGUAGE are required for runtime and good to set as defaults

nbw · 2025-01-05T10:10:44Z

src/nixpacks/builder/docker/dockerfile_generation.rs

@@ -148,7 +148,7 @@ impl DockerfileGenerator for BuildPlan {
                // Pull the variables in from docker `--build-arg`
                variables
                    .iter()
-                    .map(|var| var.0.to_string())
+                    .map(|var| format!("{}=\"{}\"", var.0.trim(), var.1.trim()))


This line changes the Dockerfile output from:

ARG ELIXIR_ERL_OPTIONS LANG LANGUAGE MIX_ENV NIXPACKS_METADATA

To:

ARG ELIXIR_ERL_OPTIONS="+fnu" LANG="en_US.UTF-8" LANGUAGE="en_US:en" MIX_ENV="prod" NIXPACKS_METADATA="elixir"

Looking at the docker arg docs and from my own testing, this simply sets a default value for the ARG but it can be overridden by setting a variable.

without this line, the rest of the dockerfile doesn't actually inherit values like MIX_ENV for the build phases and then we need to find a different workaround (like prefixing commands with MIX_ENV=prod mix local.rebar --force)

I believe we have intentionally not done this because then all variables will be logged in the build logs. The current way of injecting build variables prevents that.

but this is injecting default values of environment variables. isn't it okay for these ones to be logged? like MIX_ENV= prod

there shouldn't be any user specific secrets set as defaults afaik.

@coffee-cup is this the only discussion point that's blocking? I will comb through the codebase a bit and see if there are any defaults that we might handle differently. the alternative here is to add some flag to explicitly say "in this case, please fill in the defaults"

nbw · 2025-01-05T11:08:21Z

src/nixpacks/builder/docker/dockerfile_generation.rs

+            start_cmd=start_cmd,}
+        } else {
+            // Copy over app files if provided, otherwise copy all files
+            let copy_cmds = match &self.only_include_files {


START PHASE: if only_include_files is specified, then use it. Otherwise default to COPY . /app

I believe only_include_files was only used when the runtime image was different from the build image. So this makes the uses the same 👍

nbw · 2025-01-05T11:58:09Z

src/providers/gleam/mod.rs

-        let mut phase = StartPhase::new("./build/erlang-shipment/entrypoint.sh run");
-        phase.only_include_files = Some(vec!["build/erlang-shipment".into()]);
-        phase
+        StartPhase::new("./build/erlang-shipment/entrypoint.sh run")


failed build

the phase.only_include_files = Some(vec!["build/erlang-shipment".into()]); was never working to begin with (since it was fixed in this PR). We can remove the line to keep the previous behaviour.

the only_include_files breaks the build and was never working to begin with

nbw added 3 commits December 27, 2024 23:05

detect ecto_sql rather than postgrex to handle other databaes types

b0d4c16

postgrex is just the adapter for postgres specifically. there are other ones for different database (mysql, etc.): https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.html#module-built-in-adapters

fix: use only_include_files if provided during start phase

760bacd

updated Elixir config

a04e29a

- only copy certain files in each phase so that there isn't a conflict with the lock file - move db migrations to happen at start time rather than bulding - add more ENV defaults like LANG and LANGUAGE