local switches: shortcut for eval in older project

[avsm]

Right now, I shell alias this for whenever I switch back to building a project with a local switch:

eval $(opam env --switch=`pwd`)

It would be nice to have a shortcut for this, or some other recommendation for how to use it more seamlessly. opam exec bash works fine as well, but obviously requires a subshell.

[AltGr]

Just eval $(opam env) is a shortcut for it, since opam will choose the local switch by default. Unless, of course, OPAMSWITCH is already set, because that overrides the automatic local switch selection.

Also, --switch . works as well :). Or even --sw . if you want it as short as possible!

[avsm]

Oh! I was going by the guidelines that opam gives you when creating a new local switch.

I suspect what's going on is that I have multiple local switches, and I am changing directory between them. At this point, OPAMSWITCH is set from the first one, and so creating the second switch gives me a big command line.

Perhaps $(opam env) could be a special case to ignore OPAMSWITCH? It's a case where we want to explicitly ignore the old switch setting in order to change the current local switch to the current directory.

[AltGr]

I think the latter could be even more confusing in other cases. But I guess the source of your confusion here boils down to this:

[~/ocamlpro/ocaml-top]% opam env
OPAM_SWITCH_PREFIX='/home/lg/ocamlpro/ocaml-top/_opam'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/home/lg/ocamlpro/ocaml-top/_opam/lib/stublibs:'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/home/lg/ocamlpro/ocaml-top/_opam/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
MANPATH=':/home/lg/ocamlpro/ocaml-top/_opam/man'; export MANPATH;
PATH='/home/lg/ocamlpro/ocaml-top/_opam/bin:/home/lg/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games'; export PATH;

Here the current switch is inferred from CWD, hence opam doesn't set the OPAMSWITCH variable (it's only set when needed, i.e. the switch needs to be explicit for opam to find it again). But:

[~/ocamlpro/ocaml-top]% opam env --sw .
OPAMSWITCH='/home/lg/ocamlpro/ocaml-top'; export OPAMSWITCH;
OPAM_SWITCH_PREFIX='/home/lg/ocamlpro/ocaml-top/_opam'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/home/lg/ocamlpro/ocaml-top/_opam/lib/stublibs:'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/home/lg/ocamlpro/ocaml-top/_opam/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
MANPATH=':/home/lg/ocamlpro/ocaml-top/_opam/man'; export MANPATH;
PATH='/home/lg/ocamlpro/ocaml-top/_opam/bin:/home/lg/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games'; export PATH;

Here the switch is set explicitely, and opam doesn't actually check that it could have been inferred. Consequently, it sets the OPAMSWITCH variable.

So the two, in their current implementation, are slightly different. You would have been safe from the current problem using the first one, but basically:

• opam env updates the shell env, but has no effect on opam, and after chdir you'll get an out of sync env
• opam env --switch . updates the shell env and sets opam to the current switch on the current shell, even across chdirs.

It's clearly a bit too subtle. A first thing would be to have opam switch create . advise the first, which seems better to me. That wouldn't apply to opam switch create /foo/bar though.
What I can think of is always checking if the switch can be inferred from CWD before setting OPAMSWITCH. This would make the two commands above equivalent, with the first behaviour. But in some sense it makes the situation even more subtle since it now depends on CWD instead. E.g. eval $(opam env /foo/bar); cd /baz could make opam select a different switch for the following commands depending on whether CWD was originally below /foo/bar or not.

Any better CLI ideas ? Maybe you're right and we can't have opam env both read and write OPAMSWITCH without such quirks. Yet breaking export OPAMSWITCH=foo; eval $(opam env) is likely to cause trouble, even if it's easily replaced by a --switch flag.
As a last resort, adding stderr messages about having set OPAMSWITCH or not could help the user a tiny bit.

[AltGr]

@dbuenzli, you generally are of good advice regarding such interface concerns. Do you have an opinion on this ?

[dbuenzli]

So after an hour and half of head scratching I think I no longer understand the issue: why is opam env setting OPAMSWITCH at all ?

If we look at the problem from a standard cli ui perspective, environment variables are simply a mean for the end-user to override a value for a cli option value.

In our case this is the default value for the option --switch, whose default, without an environment variable set is looked up from cwd up to ~/ (aka as opam switch set). This is all classical cli ui and good.

What is odd is that by doing:

export OPAMSWITCH=bla 
eval $(opam env --switch bli)

I get the environment for bli but it also change my user default value of bla to bli.

I would argue that this is not what opam env is supposed to do, it is supposed to give me access to the install base of bli not set the default switch to operate on. Two operations are being conflated -- but only under some subtle condition -- in a single one which is why I think it becomes confusing.

There are enough other ways of specifying the default switch to operate on, intra (OPAM_SWITCH) or inter (opam switch set) shell and I don't think the evaluation of the opam env command "under some condition" should be one of theses.

If you stop doing so the semantics becomes clear, in any environment we get:

1. eval $(opam env) give me access to the install base of the default switch as determined by
   the classic default cli ui: env var or config,.
2. eval $(opam env --switch SW) give me access to the install base of the switch SW.

and that's all.

[AltGr]

@dbuenzli: thanks a lot for helping with this :). Indeed, we try to do two things at the same time, which intersect, but are quite different. The fact that there is some magic involved in deciding whether we do one of the two doesn't help.

One side-note: we speak about OPAMSWITCH/--switch, but OPAMROOT/--root is concerned in exactly the same way.

The original, well-intentioned purpose was to provide "shell specific" selection of a switch¹ (so both for opam and the environment). But we got tempted by the unusual fact that we could actually change the environment in this case. And local switches made the problem more apparent.

Now, while I want to fix this (maybe providing an option --fix-switch or similar to still set the variable?), this is extremely likely to cause all kinds of trouble in many scripts. @avsm, @samoht, do you have an idea of the impact that would have in your specific cases ? In any case, the change would be best before 2.0.0 if we can roll it.

¹ This raises another question: should opam exec set OPAMSWITCH, or stay similar to opam env ? Nobody would do opam exec --switch foo -- opam bla anyway would they ?

[AltGr]

note: that won't help much in script, but running opam env --switch foo could print a warning about setting OPAMSWITCH ?

[dbuenzli]

¹ This raises another question: should opam exec set OPAMSWITCH, or stay similar to opam env ? Nobody would do opam exec --switch foo -- opam bla anyway would they ?

Maybe not directly but via a script that invokes opam... I think the answer for opam exec should be simple: it should simply exec the tool in the environment as updated by eval $(opam env) that is you want this equation to hold:

opam exec [opt]... -- tool ... = eval $(opam env [opt]...); tool ...

(maybe providing an option --fix-switch or similar to still set the variable?),

I think --set-switch or even --set-opamswitch would be clearer.

Another option to ponder, also w.r.t. our discussion in #3181 (comment) would be a --restrict option that unsets all environment variables and setups only those defined by opam.

[dbuenzli]

(Oh and since I'm there for opam env, sometimes I'd like to be able to access the environment as it will be whenever opam executes command from the various opam file section that allow to specify commands test:, install:, build:, etc.)

[samoht]

I think it is a good idea to remove the heuristic used in OPAMSWITCH / OPAMROOT. The only script that I remember using this is https://github.com/moby/vpnkit/blob/master/repo/update-upstream.sh which is setting an environment to make opam install resolves dependencies in a different setting. I guess we don't need this anymore with opam2 so that should be fine ...

[Khady]

On 24 Jan 2018, at 00:48, Louis Gesbert ***@***.***> wrote: ¹ This raises another question: should opam exec set OPAMSWITCH, or stay similar to opam env ? Nobody would do opam exec --switch foo -- opam bla anyway would they ?

I do `opam exec --sw=foo -- opam upgrade` to upgrade in background while working in another switch bar. But I guess it's not a very good behavior.

[dbuenzli]

@Khady Simply doing opam upgrade --sw=foo doesn't work ?

[Khady]

@Khady Simply doing opam upgrade --sw=foo doesn't work ?

I was not aware of this option. Thanks!