--assume-depexts makes no difference (opam 2.1.0~beta4)

[mseri]

$ opam config report
# opam config report
# opam-version         2.1.0~beta4 (fb248a98cd13ba50d5caf02eded887ad1b5b57a5) 
# self-upgrade         no
# system               arch=x86_64 os=macos os-distribution=macports os-version=11.3.1
# solver               builtin-mccs+glpk
# install-criteria     -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed
# upgrade-criteria     -removed,-count[avoid-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new
# jobs                 11
# repositories         2 (version-controlled)
# pinned               9 (git)
# current-switch       4.12.0
# ocaml:native         true
# ocaml:native-tools   true
# ocaml:native-dynlink true
# ocaml:stubsdir       /Users/mseri/.opam/4.12.0/lib/ocaml/stublibs:/Users/mseri/.opam/4.12.0/lib/ocaml
# ocaml:preinstalled   false
# ocaml:compiler       4.12.0

Issue:

$ opam install owl-ode-odepack
[ERROR] Package conflict!
  * Missing dependency:
    - owl-ode-odepack -> odepack -> conf-gfortran
    depends on the unavailable system package 'gcc'. Use `--assume-depexts'
      to attempt installation anyway.

No solution found, exiting

Problem: the suggested solution makes no difference

$ opam install --assume-depexts owl-ode-odepack
[ERROR] Package conflict!
  * Missing dependency:
    - owl-ode-odepack -> odepack -> conf-gfortran
    depends on the unavailable system package 'gcc'. Use `--assume-depexts'
      to attempt installation anyway.

No solution found, exiting

Expected: a different error or an attempt at the installation

[kit-ty-kate]

Indeed I can reproduce with master (with the small exception that the Use --assume-depexts to attempt installation anyway message was replaced by You can use --no-depexts to attempt installation anyway)

[dra27]

It's not worked since at least alpha2, as well 😱

[mseri]

With --no-depexts it works for me

[AltGr]

--assume-depext skips the installation of system packages, but not their detection. The manpage seems correct about this, but the message plain wrong and misleading then. I will check...

[mseri]

Even in that case, it should have not failed (since gcc was actually installed and working, and in fact --no-depext allowed the package to install just fine)

[mseri]

This is actually going to be getting very annoying in the future: macports' gcc package is virtual, right now it corresponds to gcc10, so opam keeps saying

[WARNING] Opam package conf-gfortran.0 depends on the following system package
          that can no longer be found: gcc

and trying to uninstall anything depending on conf-gfortran each time I install or update something (unless I run it with --no-depext)

[AltGr]

Hmm, that's absolutely not the indended behaviour :/
Once you successfully install with --no-depexts, --assume-depexts or through the confirmation prompts, opam should record the exception and not bother you about it. You can check and interact the exceptions through opam option depext-bypass.

We'll need to improve the support for macports as well if virtual packages aren't supported and that blocks.

[kit-ty-kate]

This is actually going to be getting very annoying in the future: macports' gcc package is virtual

Do you know how virtual packages are defined? I can't find anything in the documentation and I'm unable to install the gcc package through macports:

% sudo port install gcc
Error: Port gcc not found

I think it might be a bug in the conf-gfortran package on opam-repository's side.

[mseri]

Mmh, you are right, it must have changed at some point then...
Right now there are a billion gccs indeed:

gcc-devel @12-20210509 (lang)
    The GNU compiler collection, prerelease BETA snapshot.

gcc5 @5.5.0_6 (lang)
    The GNU compiler collection

gcc6 @6.5.0_6 (lang)
    The GNU compiler collection

gcc7 @7.5.0_2 (lang)
    The GNU compiler collection

gcc8 @8.4.0_3 (lang)
    The GNU compiler collection.

gcc9 @9.3.0_5 (lang)
    The GNU compiler collection

gcc10 @10.3.0 (lang)
    The GNU compiler collection

gcc11 @11.1.0_1 (lang)
    The GNU compiler collection

gcc43 @4.3.6_16 (lang)
    The GNU compiler collection

gcc44 @4.4.7_15 (lang)
    The GNU compiler collection

gcc45 @4.5.4_17 (lang)
    The GNU compiler collection

gcc46 @4.6.4_15 (lang)
    The GNU compiler collection

gcc47 @4.7.4_12 (lang)
    The GNU compiler collection

gcc48 @4.8.5_6 (lang)
    The GNU compiler collection

gcc49 @4.9.4_6 (lang)
    The GNU compiler collection

gcc_select @0.1_9 (sysutils)
    common files for selecting default gcc version

And each of them should be fine for what concerns conf-gfortran.
In this case probably we can check that port select --list gcc is not just none (gcc in this case indicates the "group" of packages). But it does not feel so robust, otherwise we can specify a way to list alternative depext names (which may be annoying to maintain, but would probably only affect gcc and maybe python)

You can see all groups available and their setup with port select --summary.

EDIT: let me add all outputs

$ port select --list gcc
Available versions for gcc:
	mp-gcc10
	none (active)

$ port select --summary
Name       Selected       Options
====       ========       =======
black      black38        none
cython     cython39       cython39 none
gcc        none           mp-gcc10 none
ipython    py39-ipython   py39-ipython none
ipython3   py39-ipython   py39-ipython none
mypy       none           mypy39 none
nosetests  none           nosetests39 none
pip        pip39          pip3-apple pip39 none
pip2       none           none
pip3       pip39          pip3-apple pip39 none
pygments   py39-pygments  py39-pygments none
pylint     pylint38       none
python     python39       python27-apple python38-apple python39 none
python3    python39       python38-apple python39 none

[mseri]

Maybe @pmetzger as macports maintainer can help us to figure this out

[AltGr]

Thanks for the details. One important feature we need is to be able to check many packages without so many requests, for performance reasons; your last command might be a good solution.

[mseri]

The last command is just for packages belonging to a group though, not all packages installed. Maybe there is some other and simpler way to get this with one call. I am too noob still for these details, but the cli seems rather powerful, I would be surprised if there is not such a thing.

[AltGr]

We don't have support for alternatives in depexts at the moment either, but that will probably need to be added at some point.

[pmetzger]

@mseri what is the exact code conf-gfortran is currently using to detect gfortran?

[kit-ty-kate]

@mseri what is the exact code conf-gfortran is currently using to detect gfortran?

not conf-gfortran (the binary detection in the package itself is only tangently related) but it's currently basically the rough equivalent of:

port search --line --regex "^(gcc)$"

[pmetzger]

That seems wrong; I would presume that for the most part you would want at least a particular version or minimal version of gcc?

[kit-ty-kate]

is there really no way of indicating "i want any version of gcc" ? (the later the better, but in the end it's really not a big deal if it's not)

[pmetzger]

Well, you could, but you will end up with differences in supported features etc. gcc 4 isn't going to support your Fortran 2008 feature. Still I suppose isn't the worst possible situation? That said, for example, I have gcc10 installed, and my gfortran is named gfortran-mp-10 given that I didn't explicitly select gcc10 as the default...

[pmetzger]

I suppose the real question is what do you need exactly? For me, when I'm installing a conf-* package I assume it is on me to make sure that the thing I'm supposed to supply is supplied in /opt/local under the expected name?

[dra27]

For me, when I'm installing a conf-* package I assume it is on me to make sure that the thing I'm supposed to supply is supplied in /opt/local under the expected name?

By convention, the conf-* packages check for the presence of an external command/library - the precise mechanism for getting it is irrelevant. You could, for example, compile gcc by hand and just ensure gfortran is in PATH. In opam 2.0, you’d just omit to run opam depext and in opam 2.1 you’d add --no-depexts.

[dra27]

At the moment, depext basically doesn’t do version checking. I think all we’re after checking here is what the MacPorts equivalent of apt-get install gcc as being “install a sensible version of gcc” is, and whether there’s a way of detecting that that package is available to be installed before the command is run.

[pmetzger]

opam invoked by a non-privileged user doesn't have privileges to install the package, though of course some flavor of sudo port install gcc11 (or another version) followed by an appropriate sudo port select to set it as the default gcc would work.

The problem with the select is that the user might expect a different version of gcc to be the default. One could of course invoke the non-default gfortran binary (for example, the gcc11 package will install it as /opt/local/bin/gfortran-mp-11 if you haven't made it the default gcc.)

port select --list gcc will show the available versions of gcc that could be made the default. port installed 'gcc[0-9]*' will show installed versions of gcc. However, if I really wanted to know if gfortran in particular was available, I would check if /opt/local/bin/gfortran* existed.

[dra27]

It's the answer to a different question 🙂 Assume I am on my Macbook armed with administrative rights, forced to install Macports for the first time and I want to install gcc with it. Is there any command I can run to install gcc without caring or specifying a version?

[pmetzger]

No. We don't have a generic "install the latest gcc" command. Perhaps (arguably) there should be some such mechanism, but there isn't currently. What we do have is a way for a user to say "this gcc package is what I want to be considered the default gcc".

[pmetzger]

(By the way, by the same token, we don't have a generic "install some version of python" mechanism or what have you; one needs to say "port install python39", but one can later select python39 as the default python.)

[dra27]

OK, thanks! It sounds as though what we may need to look at soon is separating the detection of a package (i.e. is gcc available) from how it gets installed (i.e. install the gcc11 port, etc.).

I don't think there's anything left on this to worry about for 2.1.0, therefore.

[pmetzger]

It sounds as though what we may need to look at soon is separating the detection of a package (i.e. is gcc available) from how it gets installed (i.e. install the gcc11 port, etc.).

I think that's the correct approach.

[dra27]

We believe the 2.1 issues are solved - I've opened a new issue to track future improvements to depext for dealing with the selection of the correct package.