Promote appl2pine and fnlo2pine from example programs to subcommands in the CLI

[alecandido]

I'm not properly understanding what do you (@cschwan) mean by "proper programs" https://github.com/NNPDF/runcards/issues/123#issuecomment-1044133667. Therefore, I'll make some assumptions, and spell out my ideas, even when trivial, and you've already thought about them. This way, you will correct me where I'm wrong (you're for sure much more expert than me in C/C++ code compilation), and we'll have a common ground.

I'll assume that, as you say for fastNLO, you'd like to make them part of PineAPPL CLI, so either as source or compiled.

In any case, I'd use cargo as far as possible as build tool (not putting cargo inside another build tool), and make both fastNLO and APPLgrid optional (so scope them as features).

Distributing source

This is just the simpler version, but contains the whole complication of writing the thing itself.

From my point of view, what we want is to compile appl2pine and fnlo2pine in libraries, providing essentially one function each (in place of main), that will be consumed by PineAPPL CLI.

In rust should be not so unusual to consume C libraries, and I believe we can compile altogether in one step, without anything else than cargo. This should be doable with build scripts and the cc crate, as written on Cargo Book - Build Scripts.

What I don't know here is can cc replace all you are using from meson?.
meson for sure will be able to do much more than cc alone, but you don't use the full meson, and maybe there are other crates that can help (something like pkg_config, for example).

Afterwards, we should be able to bind it with extern "C" {}, but here you are the expert (you already went at least once from C to Rust, in lhapdf).

Distributing binaries

This is a pain in the neck, but still doable in CD (worst case compiling in containers). We are not even doing right now, apart from the python package (and you know it has been painful).

Essentially, it adds almost nothing to the former section, it only has to be decided if linking statically or dynamically with libraries, and writing the workflows.
If possible, I'd go for dynamical linking (to shrink the size of the binary), but it's further pain.

[cschwan]

@alecandido thanks, you pretty much spelled out what I was thinking. Let me extend a bit: From a user's point of view I think that the best would be to have a subcommand pineappl import <APPLgrid/fastNLO table> pineappl-out.pineappl.lz4 which is

1. gated behind a feature, as you said, if the user doesn't have all the dependencies,
2. not tailored specifically to either APPLgrid or fastNLO, but detects this automatically so that the user doesn't have to worry ("I want it converted to PineAPPL")

I'm not quite sure about the technical issues; the easiest is certainly fastNLO, because we can use it as is, whereas for APPLgrid we need to use a bundled and patched version, most likely.

Meson we certainly don't need, we only need to call applgrid-config and fnlo-tk-config for APPLgrid and fastNLO, respectively, to get an accurate description of how to compile and link with them. The complicated but interesting question is how much of the code I can write in Rust using cxx and how much in C++.

[alecandido]

I was not aware of cxx, that's good news. The more code we can have in Rust the better it is.

Is it possible to do it in steps? Currently, we have a working C++, and some C++ we'll be needed anyhow (because of the libraries).
Should we try to compile it as it is (mostly), and then incrementally move to Rust?

[cschwan]

That's a good idea and that is possible with cxx!

[cschwan]

So the plan in more detail is the following:

1. convert fnlo2pine from a program to functions still in C++,
2. use cxx to interface to them,
3. write a build.rs that puts the two things together,
4. add a fastnlo feature in pineappl_cli's Cargo.toml that enables to above and
5. add a subcommand import PineAPPL's CLI that uses all of the above and respects the fastnlo feature.

That I should be able to do quite quickly, and step 1) and 2) can be iterated upon to migrate more code to Rust.

[alecandido]

That's perfect!

The moment we have something more or less stable in fnlo2pine, we can consider reapplying to appl2pine (with the related further complications).

[cschwan]

PR #120 implements what I outlined in my last comment.

[alecandido]

A couple of interesting resources, in case we're really going for binary distribution at some point:

• cvmfs github action, community contributed, but might still turn to be useful
• root containers, officially provided by the ROOT team