opam upgrade --fixup doesn't always upgrade as much as possible

[Khady]

I'm not too sure how to word this issue. And details of the setup are old so I lost some context. But basically we have a CI which is stateful. We don't wipe the switch in between builds as it's very costly to start from scratch. So instead we run opam update && opam upgrade when necessary. More exactly we have

opam update

opam install ahrefs-all-deps

# with the --fixup option, the packages are not upgraded. Keeping the fixup first to fix a broken status if needed.
opam upgrade --fixup
opam upgrade

Originally we only had opam upgrade. But once in a while there's some kind of conflict between what is currently installed and what is required either by ahrefs-all-deps or by an upgrade. So we have to introduce the opam upgrade --fixup call. And unfortunately this upgrade --fixup is not enough on its own. We found out that it indeed fixes the state of the switch. But that it doesn't upgrade all packages as much as the subsequent opam upgrade call will.

It wouldn't really matter if opam upgrade was faster. But now it means losing about 15s to every opam upgrade call.

Opam 2.1.2

[rr0gi]

We don't wipe the switch in between builds as it's very costly to start from scratch

plus it simulates what the developers see when the dependency change happens

[kit-ty-kate]

Does this also happen with 2.1.5 and 2.2~beta2 ?

[Khady]

Haven’t used 2.2 enough at this point to answer this question, sorry.

[AltGr]

This works as intended 😬 . The issue is, I guess, that the fixup command isn't well named: there is no natural reason (from an interface point of view) that it should be a flag of opam upgrade.

In other words, the intended feature (disregarding the actual CLI) is to repair the switch by whatever means possible, with as little changes as possible. So I can see several possible paths from here:

• Make fixup its own command or move it to a better place
• Or at least make the doc more explicit about this

I am fully aware, though, that in this form it doesn't really solve your original issue, unless you resort to an ugly opam upgrade || ( opam upgrade --fixup && opam upgrade) ; or run fixup but with a tweak to use the upgrade criteria. Implementation-wise, it shouldn't be difficult to provide that last option: again, it's a problem of CLI, I am not sure how to fit that in.

[Khady]

or run fixup but with a tweak to use the upgrade criteria

is it something that I could already achieve with opam upgrade --fixup --criteria=...?

[kit-ty-kate]

I haven't tried (and it would be hard to test), but if that works as @AltGr hinted you can try:

opam upgrade --fixup '--criteria=-removed,-count[avoid-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new'

where the criteria used is the default criteria for upgrades with mccs (the default solver).

[Khady]

I have an easy reproduction on my side actually. And it seems to work. Thanks!

[kit-ty-kate]

In that case maybe we could change this to be the default for opam upgrade --fixup and create a new opam fixup command with the old behaviour. I think it would make more sense. What do you think @AltGr?