Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"--skip-update" has no effect #4363

Open
RalfJung opened this issue Sep 15, 2020 · 6 comments
Open

"--skip-update" has no effect #4363

RalfJung opened this issue Sep 15, 2020 · 6 comments

Comments

@RalfJung
Copy link

If your issue concerns a package not building, please report to
https://github.com/ocaml/opam-repository/issues or to the package maintainer
unless you are confident it is an issue in the opam tool itself.

I have an opam package installed whose git URL is now dead. opam uninstall coq-tlc hangs forever, trying to download the latest sources:

+ /usr/bin/git "fetch" "-q" "https://gitlab.inria.fr/charguer/tlc.git" "--update-shallow" "+9205154a:refs/remotes/opam-ref-9205154a" (CWD=/home/r/.opam/coq-8.10/.opam-switch/sources/coq-tlc)
^C[ERROR] User interruption
'opam uninstall -yvv coq-stdpp coq-tlc coq-ltac2' failed.

So I tried to add --skip-updates, but that does not help either, it still updates the package! Looks like it is impossible to uninstall or update this package, so I will have to delete the switch and start over again?

$ opam uninstall --skip-updates coq-stdpp coq-tlc coq-ltac2 -yvvv
+ /usr/bin/lsb_release "-s" "-r"
- testing
+ /usr/bin/ocamlc "-vnum"
- 4.08.1
The following actions will be performed:
  ⊘ remove    coq-spygame-builddep ~dev*                     [uses coq-tlc]
  ∗ install   ocaml-system         4.08.1                    [required by ocaml]
  ⊘ remove    coq-ltac2            0.3
  ⊘ remove    coq-tlc              dev*
  ⊘ remove    coq-iris             dev.2020-07-15.0.6b0e19f5 [uses coq-stdpp]
  ↻ recompile ocaml-config         1                         [uses ocaml-system]
  ⊘ remove    coq-stdpp            dev.2020-07-02.1.c8129a37
  ↻ recompile ocaml                4.08.1                    [uses coq-ltac2, coq-tlc]
  ↻ recompile ocamlfind            1.8.1                     [uses ocaml]
  ↻ recompile num                  1.3                       [uses ocaml]
  ↻ recompile coq                  8.10.2*                   [uses coq-stdpp, coq-ltac2, coq-tlc]
===== ∗ 1   ↻ 5   ⊘ 5 =====

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[coq.8.10.2] found in cache
[num.1.3] found in cache
+ /usr/bin/git "remote" "set-url" "origin" "https://gitlab.inria.fr/charguer/tlc.git" (CWD=/home/r/.opam/coq-8.10/.opam-switch/sources/coq-tlc)
+ /bin/tar "xfz" "/home/r/.opam/download-cache/md5/f0/f074e12325e84ebc883b37e5db10403d" "-C" "/tmp/opam-351618-1872ec"
+ /bin/cp "-PRp" "/tmp/opam-351618-1872ec/num-1.3/.gitignore" "/tmp/opam-351618-1872ec/num-1.3/Changelog" "/tmp/opam-351618-1872ec/num-1.3/LICENSE" "/tmp/opam-351618-1872ec/num-1.3/Makefile" "/tmp/opam-351618-1872ec/num-1.3/README.md" "/tmp/opam-351618-1872ec/num-1.3/dune-project" "/tmp/opam-351618-1872ec/num-1.3/num.opam" "/tmp/opam-351618-1872ec/num-1.3/src" "/tmp/opam-351618-1872ec/num-1.3/test" "/tmp/opam-351618-1872ec/num-1.3/toplevel" "/home/r/.opam/coq-8.10/.opam-switch/sources/num.1.3"
[ocamlfind.1.8.1] found in cache
+ /bin/tar "xfz" "/home/r/.opam/download-cache/md5/18/18ca650982c15536616dea0e422cbd8c" "-C" "/tmp/opam-351618-d8a4ae"
+ /bin/cp "-PRp" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/INSTALL" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/LICENSE" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/Makefile" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/Makefile.config.pattern" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/configure" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/doc" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/findlib.conf.in" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/itest" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/itest-aux" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/mini" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/site-lib-src" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/src" "/tmp/opam-351618-d8a4ae/findlib-1.8.1/tools" "/home/r/.opam/coq-8.10/.opam-switch/sources/ocamlfind.1.8.1"
+ /bin/tar "xfz" "/home/r/.opam/download-cache/sha512/80/80df91b64efc9907480388ec479362ee21067c64436da720989d6d1645ffc2f2230ae5c13069c55842da3baa7facbd143c2190d1d64d8c87935802000a02156f" "-C" "/tmp/opam-351618-9b8556"
+ /bin/cp "-PRp" "/tmp/opam-351618-9b8556/coq-8.10.2/.github" "/tmp/opam-351618-9b8556/coq-8.10.2/.gitlab-ci.yml" "/tmp/opam-351618-9b8556/coq-8.10.2/.merlin.in" "/tmp/opam-351618-9b8556/coq-8.10.2/.ocamlinit" "/tmp/opam-351618-9b8556/coq-8.10.2/CODE_OF_CONDUCT.md" "/tmp/opam-351618-9b8556/coq-8.10.2/CONTRIBUTING.md" "/tmp/opam-351618-9b8556/coq-8.10.2/CREDITS" "/tmp/opam-351618-9b8556/coq-8.10.2/INSTALL" "/tmp/opam-351618-9b8556/coq-8.10.2/LICENSE" "/tmp/opam-351618-9b8556/coq-8.10.2/META.coq.in" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.build" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.checker" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.ci" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.common" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.dev" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.doc" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.dune" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.ide" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.install" "/tmp/opam-351618-9b8556/coq-8.10.2/Makefile.vofiles" "/tmp/opam-351618-9b8556/coq-8.10.2/README.md" "/tmp/opam-351618-9b8556/coq-8.10.2/azure-pipelines.yml" "/tmp/opam-351618-9b8556/coq-8.10.2/checker" "/tmp/opam-351618-9b8556/coq-8.10.2/clib" "/tmp/opam-351618-9b8556/coq-8.10.2/config" "/tmp/opam-351618-9b8556/coq-8.10.2/configure" "/tmp/opam-351618-9b8556/coq-8.10.2/configure.ml" "/tmp/opam-351618-9b8556/coq-8.10.2/coq-refman.opam" "/tmp/opam-351618-9b8556/coq-8.10.2/coq.opam" "/tmp/opam-351618-9b8556/coq-8.10.2/coqide-server.opam" "/tmp/opam-351618-9b8556/coq-8.10.2/coqide.opam" "/tmp/opam-351618-9b8556/coq-8.10.2/coqpp" "/tmp/opam-351618-9b8556/coq-8.10.2/default.nix" "/tmp/opam-351618-9b8556/coq-8.10.2/dev" "/tmp/opam-351618-9b8556/coq-8.10.2/doc" "/tmp/opam-351618-9b8556/coq-8.10.2/dune" "/tmp/opam-351618-9b8556/coq-8.10.2/dune-project" "/tmp/opam-351618-9b8556/coq-8.10.2/engine" "/tmp/opam-351618-9b8556/coq-8.10.2/gramlib" "/tmp/opam-351618-9b8556/coq-8.10.2/ide" "/tmp/opam-351618-9b8556/coq-8.10.2/install.sh" "/tmp/opam-351618-9b8556/coq-8.10.2/interp" "/tmp/opam-351618-9b8556/coq-8.10.2/kernel" "/tmp/opam-351618-9b8556/coq-8.10.2/lib" "/tmp/opam-351618-9b8556/coq-8.10.2/library" "/tmp/opam-351618-9b8556/coq-8.10.2/man" "/tmp/opam-351618-9b8556/coq-8.10.2/parsing" "/tmp/opam-351618-9b8556/coq-8.10.2/plugins" "/tmp/opam-351618-9b8556/coq-8.10.2/pretyping" "/tmp/opam-351618-9b8556/coq-8.10.2/printing" "/tmp/opam-351618-9b8556/coq-8.10.2/proofs" "/tmp/opam-351618-9b8556/coq-8.10.2/shell.nix" "/tmp/opam-351618-9b8556/coq-8.10.2/stm" "/tmp/opam-351618-9b8556/coq-8.10.2/tactics" "/tmp/opam-351618-9b8556/coq-8.10.2/test-suite" "/tmp/opam-351618-9b8556/coq-8.10.2/theories" "/tmp/opam-351618-9b8556/coq-8.10.2/tools" "/tmp/opam-351618-9b8556/coq-8.10.2/topbin" "/tmp/opam-351618-9b8556/coq-8.10.2/toplevel" "/tmp/opam-351618-9b8556/coq-8.10.2/vernac" "/home/r/.opam/coq-8.10/.opam-switch/sources/coq"
+ /usr/bin/git "fetch" "-q" "https://gitlab.inria.fr/charguer/tlc.git" "--update-shallow" "+9205154a:refs/remotes/opam-ref-9205154a" (CWD=/home/r/.opam/coq-8.10/.opam-switch/sources/coq-tlc)
# opam config report
# opam-version      2.0.7 
# self-upgrade      no
# system            arch=x86_64 os=linux os-distribution=debian os-version=testing
# solver            builtin-mccs+glpk
# install-criteria  -removed,-count[version-lag,request],-count[version-lag,changed],-changed
# upgrade-criteria  -removed,-count[version-lag,solution],-new
# jobs              8
# repositories      4 (http), 3 (local), 2 (version-controlled) (default repo at 4fb8d40b)
# pinned            1 (git), 3 (rsync), 1 (version)
# current-switch    coq-8.10
@AltGr
Copy link
Member

AltGr commented Sep 16, 2020

Hm, it's weird, for any pinned package opam is supposed to have a cache of its latest sources at $OPAM_SWITCH_PREFIX/.opam-switch/sources/$PKGNAME ; these sources are indeed updated from upstream before install/reinstall/upgrade actions when the package is listed on the command-line, unless you specified --skip-update.

For removals, however, these updates are never necessary, so indeed the option shouldn't do anything.

I only tested on 2.1.0~beta2, but here is what I get after removing a pinned package's upstream:

$ opam upgrade ocp-index

<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] Could not synchronize $OPAM_SWITCH_PREFIX/.opam-switch/sources/ocp-index from "git+file:///tmp/ocp-index.x#master":
        "/usr/bin/git fetch -q" exited with code 128
[ocp-index.1.2.1] fetching sources failed: git+file:///tmp/ocp-index.x#master

Already up-to-date.
Nothing to do.
$ opam upgrade ocp-index --skip
Already up-to-date.
Nothing to do.
$ opam remove ocp-index        
The following actions will be performed:
  ⊘ remove ocp-browser 1.2.1* [uses ocp-index]
  ⊘ remove ocp-index   1.2.1*
===== ⊘ 2 =====
Do you want to continue? [Y/n] y

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
⊘ removed   ocp-browser.1.2.1
⊘ removed   ocp-index.1.2.1
Done.

So there is no issue with either upgrade --skip-updates or remove ; with just upgrade, the download fails, but the command can still continue (useful for reinstall, for example). Of course, in your case it hangs rather than failing, which is more annoying...

NB: have you tried <enter> or ctrl-d when the downloading hangs ? We've had cases of downloads hanging that were actually silently waiting for input.

To conclude, at this point I don't know yet if this is fixed in 2.1 or if I just couldn't reproduce.

@RalfJung
Copy link
Author

RalfJung commented Sep 16, 2020

I have since worked around it by killing the hanging git-fetch-pack (or so) process. Then opam just continued, warning me that deinstallation might use outdated information because the update failed.

Note that the deinstallation did update many things because this is a system switch and the system compiler changed, so any command will rebuild all the things. (I always struggle hard to make it only rebuild the parts I intend to keep, but found it very hard to combine "rebuild" and "upgrade" and "remove stuff I do not need any more" in a single command so usually I just give up on the "upgrade" part.) Maybe that is related?

@AltGr
Copy link
Member

AltGr commented Sep 16, 2020

Ah, it might be related, indeed... I'll see if I can reproduce with that extra information.

I always struggle hard to make it only rebuild the parts I intend to keep, but found it very hard to combine "rebuild" and "upgrade" and "remove stuff I do not need any more" in a single command so usually I just give up on the "upgrade" part.

Indeed, I don't think I have an easy solution at the moment, there is no compound install/remove or upgrade/remove command. We have been discussing a more generic install command where you could specify a formula and conflicts, but nothing is planned at the moment.
Upgrading or not can be triggered using the solver criteria, though, maybe you could find a workaround with something like opam remove -a --criteria="-removed,-count[hidden-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new" ("upgrade criteria" as extracted from `opam config report, they will differ for 2.0.7).

Maybe the most useful solution would be to add a flag that does something similar to remove -a, but could be applied to install/reinstall/upgrade commands? opam upgrade --autoremove ?

Also note that you can do opam reinstall --forget-pending, but then it will be you responsibility to reinstall everything properly, so hardly a manageable solution.

@RalfJung
Copy link
Author

Maybe the most useful solution would be to add a flag that does something similar to remove -a, but could be applied to install/reinstall/upgrade commands? opam upgrade --autoremove ?

These are not usually autoinstalled packages that I want to get rid of.

What usually happens is that I want to update some library, and then realize the system switch needs rebuilding. (All my switches are system switches. Maybe that is the mistake.^^ But it keeps them smaller.) If I now just opam upgrade it'll rebuild the old version of that library (the library in question is pinned as it is the build-dependency of what I use the switch for); if I just update the pin it'll rebuild some other library that I do not even need any more. These all take a few minutes each to build so that's quite annoying. So I want to uninstall all libraries, upgrade, and rely on the build-dependency-installation to pull them in any more. But I cannot upgrade and uninstall in one step so there are still pointless rebuilds...

@AltGr
Copy link
Member

AltGr commented Oct 7, 2020

Maybe the most useful solution would be to add a flag that does something similar to remove -a, but could be applied to install/reinstall/upgrade commands? opam upgrade --autoremove ?

These are not usually autoinstalled packages that I want to get rid of.

Ah, actually the opposite could do the trick then ? opam remove PKGS --upgrade-the-rest
That would be a shortcut for opam remove PKGS --criteria=<upgrade-criteria>, where <upgrade-criteria> can be extracted from opam config report. Can you check if that works for you ?

@RalfJung
Copy link
Author

RalfJung commented Oct 7, 2020

opam remove PKGS --upgrade-the-rest

Yeah, something like that... though then I still need two transactions if I also want to install something.

aptitude install also supports mixing in deinstallation with aptitude install package1+ package2- (installs package1 and uninstalls package2). If opam install and opam upgrade supported something similar, that would be great:
opam upgrade -a package1+ package2-.
I recommend reading man aptitude for some further inspiration for useful flags to be used with install.

That would be a shortcut for opam remove PKGS --criteria=, where can be extracted from opam config report. Can you check if that works for you ?

I will try to remember looking this up the next time the problem comes up (aka the next time debian upgrades its ocaml). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants