New depexts

[AltGr]

This closes #2919

This replaces the current obscure and verbose depexts format by lists of
system package names and filters based on opam variables, e.g. :

depexts: [
  ["libssl-dev"] { os-distribution = "debian" | os-distribution = "ubuntu" }
  ["openssl-devel"] { os-distribution = "centos" & os-version >= "7.2" }
]

The old format still parses, and is rewritten by the package migration
function (older packages in 2.0 format won't be rewritten and may get
broken depexts, as we don't want to retain compatibility with beta
format versions)

All tags currently present in opam-repository are supported, except
for source, which is discarded (they were in the same namespace and
now need to be classified between arch, os, etc.)

Note that the rewrite needs to know about the variables defined and
their values. This is based on #3058 and
https://gist.github.com/AltGr/5bfc8cea6f01e74b95de79ceaba39369, version
4, but may still change.

Opam itself does not define these variables at the moment (except for
arch and os, but not with the same normalisation yet), and the
command-line format changes too, which means that current opam-depext
will be broken by the changes. The new format is

opam list --external --vars arch=x86_64,os=linux,os-distribution=debian

instead of

opam list --external=x86_64,linux,debian

which was more ambiguous and less generic, if shorter. The idea is that
the variables will later be built in opam, so the user won't need to
declare them for normal usage.

[dbuenzli]

On aspect that doesn't seem to be handled that was present in #2919 is the ability to specify a version constraint for the package (to be handled by the sys package managers of course).

[AltGr]

Well, I don't see how we could better handle sys package versions. What is already possible is, if the package manager supports it, to give a version within the sys-package string: for example, you could use

"mariadb-client=10.1.26-0+deb9u1" {os-distribution = "debian"}

I don't think it's worth having more built-in, as it would be difficult to map to the package managers, may not even be supported in most of the cases, and most PMs only have one version per package anyway.

[dbuenzli]

This occurs quite often and is painful for CI, I still think it would be better to abstract this into depext functionality as a seperate field. This has the following advantage:

1. The person specifiying the depext does not need to know about the specifics of all the system package managers syntax. The tool responsible for installing the depext can perform a best-effort job to satisfy the request and/or warn if it can't.
2. The tool responsible for installing depext can install the package and may be able to check that a specific version was installed and warn/error otherwise.

[avsm]

Why can't this versioning be handled by os-version constraints?

e.g.

"conf-mariadb" {os-distribution = "debian" & os-version >= "8"}

The common case is that an upstream package is present in (e.g.) Ubuntu 17.04 and that needs to be encoded. The precise version would be nice, but not necessary for us to track.

[AltGr]

I think using os-version is OK for now, and if we want finer support and manage to implement it for the various PMs managed by opam-depext, we could keep the same syntax but parse e.g. "mariadb-client>=10.1" — so that this could be done without modifying opam itself.

[dbuenzli]

Why can't this versioning be handled by os-version constraints?

I'm not sure about that, e.g. think of homebrew which has nothing to do with the actual operating system version.

But indeed the depext declaration may not be the best place to specify this (if only because it wouldn't be DRY), this may need solving at the conf-* layer where being able to ask for specific version still remains an open question AFAIR.

[AltGr]

There is now a matching version of opam-depext here: ocaml-opam/opam-depext#81

[avsm]

I'm not sure about that, e.g. think of homebrew which has nothing to do with the actual operating system version.

Agreed, but homebrew is something of a special case since users are expected 'brew update' and keep on the master to solve compatibility problems. The significant version on osx is the base macos revision, which can be tracked by os-version.

I'm happy with the existing scheme in this PR; it's a huge improvement :-)

[samoht]

That looks great to me too.