[opam 2.1~alpha3] Switch invariant is trying to be too clever

[kit-ty-kate]

While testing things for #4332 I created a switch supposedly to install ocaml-base-compiler.4.11.0, however the switch invariant seems a bit unsure when a system install with the same version is already installed:

$ opam switch create 4.11 4.11.0

<><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml-base-compiler" {= "4.11.0"} | "ocaml-system" {= "4.11.0"}]

I believe the same performance issue and undesired recompilation can happen in this situation too (see #4332)
Also in this case, it creates a shift from previous versions of opam where users would expect this switch to be a non-system switch and stay this way.

[AltGr]

Well, the idea is that the invariant is exactly what you specified, no more or less; but indeed it changes from 2.0 where the choice was made and fixed at switch creation time. If we put aside the question of the additional stress put on the solver (we'll discuss it in #4322), the idea is that anyway the solver is set to minimise the changes, so once a compiler is chosen, it won't be changed anyway except if:

• asked explicitely
• explicitely installing a package that requires the other variant (shouldn't happen in practice)
• the system compiler was removed or changed on the system, which would trigger a switch from ocaml-system to ocaml-base-compiler

these 3 cases are generalisations, and would have been errors in 2.0 so, solver stress apart, this sounds OK to me.

Also in this case, it creates a shift from previous versions of opam where users would expect this switch to be a non-system switch and stay this way.

This is not entiterly true: in 2.0, opam switch create 4.11.0 would:

1. if there is no system compiler at version 4.11.0, install ocaml-base-compiler as there is no other available option on the repo
2. if there is, error out telling it can't decide between two possible choice of compiler at version 4.11.0

In case1., the newer opam will do the same (but leave the door open for a change later on, if required)
In case 2., it will defer the two choices to the solver instead of erroring out (however, this is not really optimal because the choice the solver will make is not predictable)

[kit-ty-kate]

I was going to close this issue, however I found a related bug, where if the compiler package has been pinned before the upgrade to opam 2.1, the invariant will be set to "any versions of the compiler package" because it only sees one.

Test case:

FROM ocaml/opam:debian-testing-ocaml-4.10@sha256:c56a39cb0f02c6bcf40ee628ed85d92292f367ff7e622487b73736ec2064338e
RUN opam-2.1 switch 4.10
RUN opam-2.1 switch invariant
RUN opam-2.1 pin remove ocaml-base-compiler

Result:

Step 1/4 : FROM ocaml/opam:debian-testing-ocaml-4.10@sha256:c56a39cb0f02c6bcf40ee628ed85d92292f367ff7e622487b73736ec2064338e
 ---> 312387436a3d
Step 2/4 : RUN opam-2.1 switch 4.10
 ---> Running in 0236ffd3390c
This development version of opam requires an update to the layout of /home/opam/.opam from version 2.0 to version 2.1, which can't be reverted.
You may want to back it up before going further.
Continue? [Y/n] y
Format upgrade done.
# Run eval $(opam env) to update the current shell environment
Removing intermediate container 0236ffd3390c
 ---> d745cd374da9
Step 3/4 : RUN opam-2.1 switch invariant
 ---> Running in ac195a88c35a
["ocaml-base-compiler"]
Removing intermediate container ac195a88c35a
 ---> 01aabb91ee23
Step 4/4 : RUN opam-2.1 pin remove ocaml-base-compiler
 ---> Running in 5fdee2041c95
Ok, ocaml-base-compiler is no longer pinned to https://github.com/ocaml/ocaml/archive/4.10.2.tar.gz (version 4.10.2)
The following actions will be performed:
  - upgrade   ocaml-base-compiler 4.10.2 to 4.11.1
  - recompile ocaml-config        1                [uses ocaml-base-compiler]
  - upgrade   ocaml               4.10.2 to 4.11.1 [uses ocaml-base-compiler]
  - recompile opam-depext         1.1.5            [uses ocaml]
===== 2 to recompile | 2 to upgrade =====

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> retrieved opam-depext.1.1.5  (cached)
^C

[kit-ty-kate]

Superseded by #4501