opam update issues

[leoheck]

When I try to upgrade or (re)install packages. OPAM says it is been a while since I don't upgrade OPAM.

So, it tells me to use opam update for every package install I run

Using the opam update opam removes all my packages (123 packages) and then it reinstalls everything again, in the same version. But this takes a lot of time. So it looks like it is something useless and time-consuming.

So, what this update does? Why do I have to update opam? And why does it remove all my existing packages to put them again?

[dra27]

Which version of opam?

[leoheck]

I feel it happens with all versions I ever used.
But here I have 2 machines. One is 2.1.2 and another one is 2.0.5

[rjbou]

It's the update of opam repository, to permit you to have last version of packages.
Do all packages have same version after upgrade ? Usually it is when a low level package is upgraded (ocaml / dune) that it triggers such big reinstall.
Does this happen when you launch 2 opam update/upgrade in a very short period of time?

[leoheck]

I see. As I could observe from many times I've done that all the packages were being upgraded to the latest version. How I do this opam update opam install app1 opam install app2 opam install app3 opam install app4 Apps that are already installed should not be reinstalled unless a new version exists. I believe Opam update should not remove and reinstall packages but it should only update Opans knowledge about new package versions available. This behavior is observed in other package managers.

…

On Mon, Apr 11, 2022, 13:12 R. Boujbel ***@***.***> wrote: It's the update of opam repository, to permit you to have last version of packages. Do *all* packages have same version after upgrade ? Usually it is when a low level package is upgraded (ocaml / dune) that it triggers such big reinstall. Does this happen when you launch 2 opam update/upgrade in a very short period of time? — Reply to this email directly, view it on GitHub <#5114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJX7YE2ZN547EO7WW4DVF3VERFO7ANCNFSM5S5ANXHQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rjbou]

If at least a dependency of app1 is updated, or have a new version, opam will update them and recompile ; not only if app1 have a new version.
To not launch opam install app-n for all you app packages, you can just run opam upgrade it will upgrade all packages of your switch.

[leoheck]

I may not have these apps installed. It is a script to install those apps. A new app may have been added to the list. If you run the script again, it will install extra things that may have been added and it should upgrade existing packages if there are newer versions of them. What is the goal of "opam update"? Why does it have to remove my apps? Why is it reinstalling the exact same version of the tools? This is the important part that has to be addressed here in this issue report.

…

On Mon, Apr 11, 2022, 13:24 R. Boujbel ***@***.***> wrote: If at least a dependency of app1 is updated, or have a new version, opam will update them and recompile ; not only if app1 have a new version. To not launch opam install app-n for all you app packages, you can just run opam upgrade it will upgrade all packages of your switch. — Reply to this email directly, view it on GitHub <#5114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJX7YE4GKKHDYYM26UA6WTVERG2VANCNFSM5S5ANXHQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rjbou]

As I pointed earlier, the goal of opam update is to update the opam repository, to permit you to have last version of packages. It doesn't update your packages. Given the new / updated packages in repository, opam installs new versions of already installed packages (with opam install or opam upgrade call).
Let's take two installed packages: a.1 and b.1, with a that depends on b. If after an update, there is a new version of b, b.2, opam will upgrade b.1 -> b.2, and reinstall a.1 as its dependency b was reinstalled.

There is a need of more information, steps to reproduce (script/dockerfile), to be able to dig on an issue.

[leoheck]

This is the install script I have

#!/bin/bash
# update anything that was already installed, if something is already installed
opam update

# required packages for building the application I want to build myself
opam install -y digestif
opam install -y lwt
opam install -y lwt_ppx
opam install -y cmdliner
opam install -y base64
opam install -y sha
opam install -y tyxml
opam install -y git-unix

The Required Packages is a list of packages that my system should have. This list may have new packages/items if I want to have other packages installed. I will run this bash script on a new computer or on an existing computer if something needs to be upgraded.

Is there a way or flag to force not removing existing packages?
This is the only information I have, what else do you need?
I have no idea how Opam works. But removing the packages to reinstall them again seems not right for me.

[rjbou]

In a switch with an outdated repository, having digestif and sha installed, i have

$ opam update

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
[default] synchronised from [...]
Now run 'opam upgrade' to apply any package updates.
$ opam install -y sha
[NOTE] Package sha is already installed (current version is 1.15.2).
$ opam install -y digestif
[NOTE] Package digestif is already installed (current version is 1.1.0).
$ opam upgrade
The following actions will be performed:
=== recompile 1 package
  ↻ eqaf       0.8            [uses cstruct]
=== upgrade 2 packages
  ↗ cstruct    6.0.1 to 6.1.0
  ↗ digestif   1.1.0 to 1.1.2
=== install 3 packages
  ∗ fmt        0.9.0          [required by cstruct]
  ∗ ocamlbuild 0.14.1         [required by fmt]
  ∗ topkg      1.0.5          [required by fmt]

Proceed with ↻ 1 recompilation, ↗ 2 upgrades and ∗ 3 installations? [Y/n] n

As you can see, installed packages doesn't change, and they are upgraded only if it is required explicitly.

Are you sure that you use always the same switch?

If you need help about opam usage, you can find help in ocaml forum

[kit-ty-kate]

@leoheck is #4647 your issue? If so this will be fixed in the next opam release.

[leoheck]

@rjbou this log window looks a bit different and it looks better indeed. Let me debug a bit deeper to see what else I can add here.
@kit-ty-kate not sure, maybe. How do I test or identify if this is the case? Do you have any idea?

[kit-ty-kate]

well, if your issue doesn’t happen between two subsequent (like a couple of minutes apart) calls to your script above for example, then this might be it.

[rjbou]

To debug, you can add --debug to opam update & opam install commands, and redirect stderr in a log file.

[kit-ty-kate]

@leoheck Could you try to use #5118 as your local opam and see if you still get the same issue?