adding distillery as the release manager and ensuring the release starts

[InoMurko]

• I've added Distillery dependency that we can use as the OTP release process.
• I've removed all Mix.env() runtime calls, because Mix isn't available in a OTP release, making the release unable to start.

[hayesgm]

A few questions on environments and releases. Also, can you do a refresh rebase off of master?

[hayesgm]

Should we load cookie from somewhere else or just not include it at all?

[InoMurko]

At this moment I think the purpose of having the release built in CI is that we're ensuring it doesn't get broken. Any tarball release that might deploy the tarball would want to overwrite the cookie anyway.

Setting it to something that doesn't look generated might make more sense.

[InoMurko]

In the longer run, having acceptance tests that would run against a release would prove the release working as well.

[hayesgm]

We should add the release tar.gz as an artifact so it's stored by CircleCI

[InoMurko]

It would make sense to pack the artefacts + logs once we have acceptance test run against it.

[InoMurko]

Sorting issues with RocksDB...

[hayesgm]

Alright, looks good to me. Thanks!

[hayesgm]

@InoMurko I was taking a look at this last night-- it's definitely having a problem compiling the rox dependency, breaking during a rust build of a C++ artifact? But, the bigger question is actually: it works fine building test (MIX_ENV=test mix compile)-- is it not building that artifact in test or is the environment different?

[InoMurko]

Hey @hayesgm!
What happens is that rox wants to optimise the cargo build for a release. As per https://hexdocs.pm/rustler/basics.html and the mix.exs of the rox dependency:
defp rustler_crates do [ rox_nif: [path: "native/rox_nif", cargo: :system, default_features: false, features: [], mode: :release], ] end

What I think it happens is that the compilation eats the all available memory in the container executing the job. I've been aligning with @atoulme to increase the CI plan that would allow us to increase the memory size - it's only 4gb at the moment. That should solve it. I've tried compiling a release on a bigger linux instance and it worked.

[hayesgm]

I guess I'm a little disheartened that rust takes 4GB of memory to compile? I could imagine it taking longer, but that much memory to build a KV database?

[InoMurko]

That's my current take (with limited time at devcon and the output being internal compiler error: Killed with exit code 4). Tried also rustc nightly build and it comes up the same.

[InoMurko]

Managed to get this working by swapping the RocksDB dependency used with Erlang's impl https://gitlab.com/barrel-db/erlang-rocksdb .

That pointed me to the fact that db handles are not being closed. Probably because the previous :rox dependency didn't support that. This awesome work from Benoit does, we should implement that in a later stage.

Please review again @hayesgm @masonforest @atoulme and others.

[hayesgm]

Great work getting this building. Do you know if it's possible to build for multiple architectures? E.g. one for linux, one for macOS and one for Windows?

[InoMurko]

I’m not 100% for windows. Everything else should work.

[hayesgm]

On MacOS:

$ ./bin/mana
./erts-10.0.5/bin/erl: line 13: ./erts-10.0.5/bin/erlexec: cannot execute binary file
./erts-10.0.5/bin/erl: line 13: ./erts-10.0.5/bin/erlexec: Undefined error: 0
Unusable Erlang runtime system! This is likely due to being compiled for another system than the host is running

[InoMurko]

Oh I wasn’t sure what you meant. You took the release artifact from Circle? You would have to build it separately. Afaik, circle allows docker, mac but not windows.

[hayesgm]

Yeah, it'd be cool if we could build for macOS, for instance, in a docker container. But for now, this is great and we should merge to master!

[hayesgm]

looks great, thanks for getting this working.

[MattMSumner]

Once the credo change is reverted, this LGTM.