Leftover files cause installation issues

[cpitclaudel]

I don't have a good repro for this, but when running complex installations, I sometimes end up in a broken state with leftover files, like this:

$ opam install --restore
Packages to be restored: { batteries, coq, coq-reduction-effects, coqide, core, core_bench, js_of_ocaml, js_of_ocaml-ppx, patdiff, pcap-format }
The following actions will be performed:
  ∗ install js_of_ocaml-compiler  3.9.1   [required by js_of_ocaml]
  ∗ install num                   1.4     [required by batteries, coq]
  ∗ install js_of_ocaml           3.9.0
  ∗ install sexplib               v0.14.0 [required by core]
  ∗ install coq                   8.13.2
  ∗ install batteries             3.3.0
  ∗ install js_of_ocaml-ppx       3.9.0
  ∗ install ppx_cstruct           6.0.0   [required by pcap-format]
  ∗ install core_kernel           v0.14.1 [required by core_bench, core, patdiff]
  ∗ install coqide                8.13.2
  ∗ install coq-reduction-effects 0.1.2
  ∗ install pcap-format           0.5.2
  ∗ install timezone              v0.14.0 [required by core]
  ∗ install patience_diff         v0.14.0 [required by patdiff]
  ∗ install core                  v0.14.1
  ∗ install textutils             v0.14.0 [required by core_bench]
  ∗ install patdiff               v0.14.0
  ∗ install core_bench            v0.14.0
===== ∗ 18 =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[coq-reduction-effects.0.1.2] downloaded from https://github.com/coq-community/reduction-effects/archive/v0.1.2.tar.gz
[batteries.3.3.0] downloaded from cache at https://opam.ocaml.org/cache
[core.v0.14.1] downloaded from cache at https://opam.ocaml.org/cache
[core_bench.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache
[coq.8.13.2] downloaded from cache at https://opam.ocaml.org/cache
[core_kernel.v0.14.1] downloaded from cache at https://opam.ocaml.org/cache
[js_of_ocaml.3.9.0] downloaded from cache at https://opam.ocaml.org/cache
[js_of_ocaml-ppx.3.9.0] found in cache
[js_of_ocaml-compiler.3.9.1] downloaded from cache at https://opam.ocaml.org/cache
[num.1.4] downloaded from cache at https://opam.ocaml.org/cache
[patience_diff.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache
[coqide.8.13.2] downloaded from cache at https://opam.ocaml.org/cache
[patdiff.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache
[pcap-format.0.5.2] downloaded from cache at https://opam.ocaml.org/cache
[textutils.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache
[sexplib.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache
[ppx_cstruct.6.0.0] downloaded from cache at https://opam.ocaml.org/cache
[timezone.v0.14.0] downloaded from cache at https://opam.ocaml.org/cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The installation of num failed at "make install".
[ERROR] The compilation of js_of_ocaml-compiler failed at "/home/clement/.opam/opam-init/hooks/sandbox.sh build dune
        build -p js_of_ocaml-compiler -j 7".

#=== ERROR while installing num.1.4 ===========================================#
# context     2.0.7 | linux/x86_64 | ocaml-base-compiler.4.09.0 | https://opam.ocaml.org#e1ac5eed
# path        ~/.opam/default/.opam-switch/build/num.1.4
# command     ~/.opam/opam-init/hooks/sandbox.sh install make install
# exit-code   2
# env-file    ~/.opam/log/num-132901-17d454.env
# output-file ~/.opam/log/num-132901-17d454.out
### output ###
# make -C src install
# make[1]: Entering directory '/home/clement/.opam/default/.opam-switch/build/num.1.4/src'
# install -d /home/clement/.opam/default/lib/ocaml
# sed -e 's/%%VERSION%%/1.4/g' META.in > META
# ocamlfind install num META
# ocamlfind: Package num is already installed
#  - (file /home/clement/.opam/default/lib/num/META already exists)
# make[1]: *** [Makefile:91: install] Error 2
# make[1]: Leaving directory '/home/clement/.opam/default/.opam-switch/build/num.1.4/src'
# make: *** [Makefile:14: install] Error 2


#=== ERROR while compiling js_of_ocaml-compiler.3.9.1 =========================#
# context     2.0.7 | linux/x86_64 | ocaml-base-compiler.4.09.0 | https://opam.ocaml.org#e1ac5eed
# path        ~/.opam/default/.opam-switch/build/js_of_ocaml-compiler.3.9.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p js_of_ocaml-compiler -j 7
# exit-code   1
# env-file    ~/.opam/log/js_of_ocaml-compiler-132901-ea5034.env
# output-file ~/.opam/log/js_of_ocaml-compiler-132901-ea5034.out
### output ###
#       ocamlc compiler/lib/.js_of_ocaml_compiler.objs/byte/js_of_ocaml_compiler__Js_parser.{cmi,cmti} (exit 2)
# (cd _build/default && /home/clement/.opam/default/bin/ocamlc.opt -w -40 -w -7-37 -safe-string -g -bin-annot -I compiler/lib/.js_of_ocaml_compiler.objs/byte -I /home/clement/.opam/default/lib/biniou -I /home/clement/.opam/default/lib/bytes -I /home/clement/.opam/default/lib/easy-format -I /home/clement/.opam/default/lib/menhirLib -I /home/clement/.opam/default/lib/ocaml/compiler-libs -I /home/[...]
# File "compiler/lib/js_parser.mli", line 1:
# Error: /home/clement/.opam/default/lib/menhirLib/menhirLib.cmi
#        is not a compiled interface for this version of OCaml.
# It seems to be for an older version of OCaml.



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build   js_of_ocaml-compiler 3.9.1
│ ∗ install num                  1.4
└─ 
╶─ No changes have been performed

My usual approach is to delete .opam completely. It happens every few months.

I do not see a documented way to "clean up" a switch to remove all these stale files, or even a way to completely rebuild a switch. I'm including the complete interaction below in the hope that it helps shed some light on this. As far as I can tell, the problem stems from OPAM not removing enough stuff during an upgrade: [WARNING] While removing ocaml-base-compiler.4.07.1: not removing files that changed since: (see below in --unlock-base section)

$ opam config report
# opam config report
# opam-version      2.0.7 
# self-upgrade      no
# system            arch=x86_64 os=linux os-distribution=linuxmint os-version=20.1
# solver            builtin-mccs+glpk
# install-criteria  -removed,-count[version-lag,request],-count[version-lag,changed],-changed
# upgrade-criteria  -removed,-count[version-lag,solution],-new
# jobs              7
# repositories      3 (http), 1 (version-controlled) (default repo at e1ac5eed)
# pinned            2 (git)
# current-switch    default

I cannot post a complete log of that session in an issue (the body of an issue is limited to 65536 characters), so here is a gist: https://gist.github.com/cpitclaudel/920d80f224515b22030d346b7e4b5d3a

[dra27]

Looks related to #4419 - that's due to be looked at as part of 2.2.0 so let's see if that also fixes this!