Provide libdir, ld and ldflags into the distribution to be usable by dune

[dinosaure]

Some information are missed (about #50). This PR add some others file to be usable by dune and compile/link correctly unikernels. Into details, dune with or without duniverse is able to load some information available from files.

In your case, dune can loads files located into $(opam config var lib)/ocaml-freestanding/* (with expansion of %{lib:ocaml-freestanding:*}). Currently, only cflags and libs are provided. This PR adds ld, libdir and ldflags.

[dinosaure]

'The patch differs from the initial patch of @TheLortex where a sexp-expression is provided about libdir instead raw arguments. I did this choice to keep a consistency of provided files.

[TheLortex]

Hi, thank you for this PR. I'm not sure using a S-expression should work here.
Indeed there are several ways to import files in dune:

• %{read:<file>}, %{read-lines:<file>},%{read-strings:<file>}: reads the file as a plain text file, this comes from variable expansion (https://dune.readthedocs.io/en/stable/concepts.html#variables)
• (:include <file>) parses a S-expression containing a single string (https://dune.readthedocs.io/en/stable/concepts.html#ordered-set-language).

So they are not part of the same language, and precisely I'm not sure that (:include <file>) can be used instead of %{read:<file>} for rule actions.

As a reminder, here are how there different files are used for the mirage dunification project:

• S-exp cflags: https://github.com/TheLortex/mirage/blob/full-dune/lib/mirage.ml#L744
• S-exp libs: https://github.com/TheLortex/mirage/blob/full-dune/lib/mirage.ml#L784
• plain libdir: https://github.com/TheLortex/mirage/blob/full-dune/lib/mirage.ml#L652
• plain ld/ldflags: https://github.com/TheLortex/mirage/blob/full-dune/lib/mirage.ml#L669 / https://github.com/TheLortex/mirage/blob/full-dune/lib/mirage.ml#L694

[dinosaure]

So they are not part of the same language, and precisely I'm not sure that (:include ) can be used instead of %{read:} for rule actions.

My idea then, specially about the dunification of MirageOS is the do the translation between arguments provided by these files to s-expression by generated (by mirage tool) dune file. Again, I would like to keep a certain consistency between all files provided by ocaml-freestanding and avoid special layout needed only by dune.

An other solution is to explicitly provide cflags .sexp and libs .sexp to notice a special format of them by the extension.

[hannesm]

About this PR and #50, they make me slightly uneasy due to duplication of information (in .pc and flags/). In a Mirage3 world (ocamlbuild), pkg-config and .pc files are used to pass around cflags/ldflags/.. . Now, in a Mirage4 world (dune), the same information is embedded in separate files, and pkg-config is still around, called at installation time of solo5 + ocaml-freestanding. Is this correct?

Would another path be implement the required pkg-config parts as OCaml library?
Will the result be .pc file free, and not depending on pkg-config (highly appreciated since we already discovered some divergence between Ubuntu, other Linux, and OpenBSD (where we install a custom symbolic link IIRC))?

[dinosaure]

About this PR and #50, they make me slightly uneasy due to duplication of information (in .pc and flags/). In a Mirage3 world (ocamlbuild), pkg-config and .pc files are used to pass around cflags/ldflags/.. . Now, in a Mirage4 world (dune), the same information is embedded in separate files, and pkg-config is still around, called at installation time of solo5 + ocaml-freestanding. Is this correct?

It's correct, but according generation of these files, we still follow what is provided by ocaml-freestanding.pc. I mean, with this PR, we have two paths for same values.

Would another path be implement the required pkg-config parts as OCaml library?
Will the result be .pc file free, and not depending on pkg-config (highly appreciated since we already discovered some divergence between Ubuntu, other Linux, and OpenBSD (where we install a custom symbolic link IIRC))?

From what I know about libraries (like bigstringaf or digestif), they can get these files without pkg-config with, like I said, expansion of %{lib:ocaml-freestanding:*} - but the requirement is that these libraries must use dune.

However, in other side, I'm not sure that is easily possible to get off pkg-config from the mirage tool where some *.pc have the Requires field (which can not be imitated by the second way - the way implemented by this PR). Currently, I really would like to post-pone your concern after the dunification and keep a compatible eco-system first with these two ways.

Then, we can think and do the choice about the right direction (and surely properly delete pkg-config) with dune in our mind and our unikernels.

[dinosaure]

Another diff, between files provided in this PR and pkg-config is that flags provided by files should be used by ocamlopt (with -ccopt/-cclib, see this commit). And pkg-config provides flags for ld/cc.

About this specific question, I try to find an uniform way which concerns only (I think) mirage tool.

[mato]

Stepping back a bit to get the big picture -- the goal here is to somehow integrate dune, mirage and unikernels. If we can replace pkg-config with something different, then we should try and do that.

As far as I can tell, the dependencies on pkg-config in the whole Mirage(/Solo5) stack are:

• solo5-bindings-*
  • ocaml-freestanding
    • mirage-solo5
      • (indirectly) the mirage tool and generated unikernel build system
    • gmp-freestanding
    • zarith-freestanding
    • (unclear, indirect?) nocrypto / mirage-entropy

and we control all of these packages, therefore we can change the whole stack as we see fit.

Now, I built/inherited the current pkg-config solution, so I know what pieces of information are passed around and how. What I don't know is:

Q: What are the available mechanisms for a dune-based project/package X to query dependency Y for some key/value based metadata published by Y, which is then "expanded" by dune during build of package X?

In other words, are we limited to expansion of the content of "some file published by X"? How does the location of "some file" get determined? What formats of "some file" can dune parse? Can this mechanism also work in some sensible way without depending on dune? For example, "cross"-builds of C packages such as gmp-freestanding?

Without understanding what's available I can't offer any potential solutions / replacements for pkg-config.

[dinosaure]

I did mirage/mirage#1018 to not lose what we talk in this issue. Please continue to this issue instead this PR.

[mato]

@dinosaure As I understand it, this PR is currently out of date / "to be replaced by something else"? Or should it be considered for merging?

[dinosaure]

Out of date 👍 !