Compiling ppx_deriving with +musl+static

[anmonteiro]

I wanted to revive the conversation started in #187 wrt compiling ppx_deriving in a switch that has a statically linked OCaml compiler.

I understand this is not currently possible given that ppx_deriving uses Dynlink, which is not available in a statically linked compiler. Is there a way forward that removes the dynlink dependency from ppx_deriving, or is that out of the question / would entail significant re-architecting?

[ghost]

Just chiming in as I thought about this in the past. Removing Dynlink would make things difficult for the toplevel as currently this is the only way to use derivers in the toplevel. It could be changed but that's a non-trivial amount of work. /cc @emillon as we were discussing toplevels recently.

However, it seems to me that it would be easy to make Dynlink optional in ppx_deriving and not include support for it when the compiler doesn't support shared libraries.

[emillon]

Out of curiosity, how does ppxlib solve this? does it support the toplevel at all?

[ghost]

Well, it doesn't really solve it. When using ppx packages via #require in the toplevel, we are still using the old method of:

• applying ppx rewriters one after the other via separate processes
• one of these processes in the ppx_deriving driver that dynamically load the various [@@deriving] plugins

One solution would be to do like we were doing at the Camlp4 times and simply do the preprocessing in the toplevel process directly. Then we wouldn't need anything special for the toplevel.

Thinking about this again, now that the vast majority of ppx rewriters are compatible with the static driver model and in particular based on ocaml-migrate-parsetree (either directly or indirectly via ppxlib), it shouldn't be too difficult to implement. In fact the following code does exactly this: https://github.com/janestreet/toplevel_expect_test/blob/0d669c54bb2e61ec710423e7daf5ad83fcb28d8b/src/main.ml#L152

The only tedious part is the handling of findlib predicates. But with a bit of work on Dune and the toplevel, it should be manageable at least for Dune based ppx rewriters.

[anmonteiro]

Thanks for your input, @diml and @emillon. As I understand it this is something that's possible but hard to do without at least some amount of work.

My follow-up to that is: should this be a goal for ppx_deriving given the amount of work involved? I can certainly not gauge the amount of work so I'll trust your opinions.

[ghost]

Well, there is not much ppx_deriving can do at this point. It's more: some work needs to be done on the tooling to completely stop relying on the old way of using ppx, and once this is done it becomes safe for ppx_deriving to drop the dynlink code path.

Regarding whether it is worth doing this work, I personally think it is as it would make the system technically simpler overall (less code paths). However, it's not urgent given that things mostly work right now.

[vphantom]

(Quick note in case others like me end up here looking to build static binaries but without actually needing a statically-linked compiler: I found out today that to compile static binaries, one only needs to use a +musl switch along with -ccopt -static compiler flags. ppx_deriving seems quite happy with the Musl libc.)