RabbitMQ service fails to start: "Bus error: 10"

[ter0]

Describe the bug
Enabling the rabbitmq service and running devenv up throws an error.

Running macOS Sonoma.

To reproduce
Run devenv init, edit devenv.nix to contain the following:

$ cat devenv.nix
{ pkgs, ... }:

{
    services.rabbitmq.enable = true;
}

Run devenv up:

$ devenv up
Building shell ...
12:09:03 system     | rabbitmq.1 started (pid=79201)
12:09:04 rabbitmq.1 | /nix/store/xpnay05w17jag16k8npp0zp0bnj6ir4a-rabbitmq-server-3.12.4/bin/rabbitmq-server: line 151: 79205 Bus error: 10           start_rabbitmq_server "$@"
12:09:04 system     | rabbitmq.1 stopped (rc=0)

It definitely used to work a while ago, although I'm not sure what exactly has changed.

Version
Running the latest master.

$ devenv version
devenv: 0.6.3

[alexpearce]

TL;DR: Erlang's JIT compilation, enabled on most systems by default, is broken on Sonoma (erlang/otp#7687 (comment)). You need to use a RabbitMQ package built with an Erlang built with --disable-jit. For example:

rabbitmq = {
  enable = true;
  package = let
    erlang = pkgs.beam.interpreters.erlangR25.override {
      configureFlags = ["--disable-jit"];
    };
    beamPackages = pkgs.beam.packagesWith erlang;
    elixir = beamPackages.elixir;
  in pkgs.rabbitmq-server.override {
    inherit erlang elixir;
  };
  managementPlugin.enable = true;
};

Full investigation below.

---

I'm also experiencing this on macOS Sonoma

It definitely used to work a while ago, although I'm not sure what exactly has changed.

Same, my devenv definitely worked before. I think today is the first time I ran devenv up since installing Sonoma, so that could be the culprit (somehow).

Interestingly I can get a similar Bus error: 10 when building RabbitMQ. If I override the RabbitMQ package with one built with Erlang 26:

rabbitmq = {
  enable = true;
  package = pkgs.rabbitmq-server.override { erlang = pkgs.beam.interpreters.erlangR25; };
};

I get these logs from the RabbitMQ build:

make[3]: Leaving directory '/private/tmp/nix-build-rabbitmq-server-3.12.0.drv-0/rabbitmq-server-3.12.0/deps/stdout_formatter'
 GEN    escript/rabbitmqctl
/nix/store/ag6lcqchlfb3nsn71slrdr9c60a4gf88-bash-5.2-p15/bin/bash: line 5: 62460 Done                    echo y
     62461 Bus error: 10           | ERL_COMPILER_OPTIONS=deterministic mix make_all_in_src_archive
make[2]: *** [Makefile:102: escript/rabbitmqctl] Error 138
make[2]: Leaving directory '/private/tmp/nix-build-rabbitmq-server-3.12.0.drv-0/rabbitmq-server-3.12.0/deps/rabbitmq_cli'
make[1]: *** [../../erlang.mk:4512: deps] Error 2
make[1]: Leaving directory '/private/tmp/nix-build-rabbitmq-server-3.12.0.drv-0/rabbitmq-server-3.12.0/deps/rabbit'
make: *** [erlang.mk:4512: deps] Error 2

I get a successful build with Erlang 25 (which I think is the default version of Erlang in my pinned nixpkgs, so the one my rabbitmq-server used anyway). But I get the bus error when starting.

If I try to run the erl executable my RabbitMQ runs I can reproduce the bus error.

$ /nix/store/5fhn6agp344p074jlvbixpa8faw1vdj3-erlang-25.3.2.2/bin/erl
fish: Job 1, '/nix/store/5fhn6agp344p074jlvbi…' terminated by signal SIGBUS (Misaligned address error)

This thread points to erlang/otp#7687 (comment) which suggests that JIT compilation isn't compatible with Sonoma.

Try using an Erlang built with --disable-jit:

rabbitmq = {
  enable = true;
  package = pkgs.rabbitmq-server.override {
    erlang = pkgs.erlang.override {
      configureFlags = ["--disable-jit"];
    };
  };
};

This failed in the same way as the previous build. But I noticed what's failing is a mix call, and presumably the elixir package is using its own erlang which still has JIT enabled.

Try using a custom Elixir as well.

rabbitmq = {
  enable = true;
  package = let
    erlang = pkgs.beam.interpreters.erlangR25.override {
      configureFlags = ["--disable-jit"];
    };
    beamPackages = pkgs.beam.packagesWith erlang;
    elixir = beamPackages.elixir;
  in pkgs.rabbitmq-server.override {
    inherit erlang elixir;
  };
  managementPlugin.enable = true;
};

The RabbitMQ build succeeds and devenv up works!

[domenkozar]

Should we add the package override when using Darwin in to erlang itself? Ideally this would even be fixed in https://github.com/NixOS/nixpkgs