Use the vcs to get a version number for pinned package?

[ghost]

Following the discussion on dbuenzli/topkg#111, I'd like to suggest to change the way opam assigns version numbers to pinned packages.

AFAIU, currently opam chooses the version for pinned packages as follow:

1. use the version specified on the command line, if any
2. use the version: field in the opam file of the pinned package, if any
3. use the latest version available in the package repository, if a package with the same name already exists
4. ~unknown with opam1 and ~dev (+dev?) with opam2

This can lead to inconsistencies between what opam thinks the version is and what other tools such as ocamlfind think the version is. I think especially rule 3 is problematic. For instance:

$ opam pin add --dev react
[...]
$ opam info react -f version
1.2.1
$ ocamlfind list|grep react
react               (version: 1.2.0-18-gd9c6338)
react.top           (version: 1.2.0-18-gd9c6338)
$ grep Release .opam/4.04.0/doc/react/README.md 
Release v1.2.0-18-gd9c6338

I suggest to change rule 3 to query the version from the vcs, when there is one, in a similar fashion to what topkg does for git and mercurial.

Alternatively, deleting rule 3 entirely would already be an improvement IMO.

[dbuenzli]

assigns version numbers to pinned packages.

The discussion should be broadened to dev packages (those for which %{dev}% is true in opam v2). This is pinned packages + packages from a repo with a VCS url. I have no idea how opam assigns version number to the latter.

[AltGr]

Packages from a repository, including dev packages, need to be defined with a version, due to the repository structure. The opam file from the source is not used in this case.

The rationale behind rule 3 is that you can plug in a direct replacement for a given package. In particular, all packages don't have to include an opam file, and in that case, it corresponds to the package definition opam is going to use (only altering the source).

Inferring a version from the VCS would be quite nice. The issue I might have with it is that it won't work everywhere (people may not tag their releases, for example, or may tag them with a v prefix...) I don't generally like to make assumption about the project sources. Is there a way to have it, but make sure it'll never give wrong or weird results ?

[ghost]

Sorry for the late reply, I had forgotten about this ticket. If we don't want to make assumption about project sources, we can make it an explicit opt-in. For instance by adding a variable to refer the version from the VCS. Then a user might write in their opam file:

version: version_from_vcs

And tools such as topkg or dune-release could update or remove this variable as needed when preparing a release. We could also use the same syntax as these, i.e. you'd write:

version: "%%VERSION%%"
# or, to drop the v prefix:
version: "%%VERSION_NUM%%"

[ghost]

Another idea: when pinned, many packages require a preparation step. For instance some Dune packages have ["jbuilder" "subst"] {pinned} in their build instructions in order to substitute %%VERSION%% and other special strings. One possibility would be to make this step more explicit and opam would only determine the version after applying this step. This is fairly generic and would allow packages to prepare their sources as needed.

[avsm]

It would be preferable to use opam's builtin version control logic if possible, so I prefer @diml's first idea of creating a %%VERSION%% substitution and tagging the package as "vcs-aware". That also works more gracefully with the topkg/dune-release workflows that already exist.