Support discovering multiple conda envs with same interpreter path

[rchiodo]

Testing #18348

1. Install conda
2. Create two conda environments like so:
   conda create -n test1
conda create -n test2
3. Activate each and get python path. They should be the same
4. Try using the API to find them.

They don't show up at all.

[karthiknadig]

@rchiodo Which API did you use? if the extension was already loaded and you created these then you may have to call refreshInterpreters with clear cache.

/cc @karrtikr This seems like a underlying discovery issue.

[karthiknadig]

@rchiodo conda create -n test1 does not create a conda environment with python in it.. you have to do conda create -n test1 python

as you can see in my case there is no global python, so when i ask for python path after activation i get nothing:

but if you create conda environment with python then you get python path:

[rchiodo]

I guess I have a global python set. I didn't get that error

[rchiodo]

Here's what I used to test:
https://github.com/rchiodo/test-python-api

[karthiknadig]

So if the environment itself does not have python, then we don't detect it or report anything about it. does this happen if you create it like this:

conda create -n test1 python=3.9
conda create -n test2 python=3.9

This is 3.9 because 3.10 in conda is broken.

[rchiodo]

No that works fine. That's usually how I have to create them in order to get them to work.

[rchiodo]

I mean this is a problem in the old API too.

[karthiknadig]

Then this is working as expected. if the environment was not created to include python we don't detect those.

[rchiodo]

I don't think a customer is going to expect this. They have multiple conda environments but they don't see any of them.

[karthiknadig]

You can have pure R environments in conda. Do we expect python extension to detect those? The python in environment created using conda create -n env1 is not the python from the environment. So, if we say install things in the python, then it would be the global python. Essentially, all installation will happen to that. I think that this might lead to greater problems with the python on the machine itself.

/cc @brettcannon, @luabud what do you suggest, should we list every conda environment or only those envs that have been setup with python. Currently we only detect those that have python in the env scripts/bin folder.

[rchiodo]

So, if we say install things in the python, then it would be the global python.

This isn't true. If I do something like so:

conda activate test1
conda install matplotlib

It installs it here (even though python is global):

C:\Users\aku91\miniconda3\envs\test1\Lib\site-packages\matplotlib>

If I run python from that active environment, its sys path has been modified to use the conda location, and not the global:

(test1) C:\Users\aku91\miniconda3\envs\test1\Lib\site-packages\matplotlib>python
Python 3.10.2 | packaged by conda-forge | (main, Jan 14 2022, 07:57:10) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\Users\\aku91\\miniconda3\\envs\\test1\\python310.zip', 'C:\\Users\\aku91\\miniconda3\\envs\\test1\\DLLs', 'C:\\Users\\aku91\\miniconda3\\envs\\test1\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\test1', 'C:\\Users\\aku91\\miniconda3\\envs\\test1\\lib\\site-packages']
>>>

Conda supports this option just fine.

[karthiknadig]

In the extension, we support both pip install and conda install for conda environments. The problems happens when people do pip install. In my scenario where I don't have global python, nothing works even from regular terminal, running a python script after env activation also does not work (all outside of vscode). So, we need to do something about this. May be add text that says it uses global python (if we decide to go the route of detecting these).

[rchiodo]

If we change pip install to instead use the activated environment and run python -m pip install, it works for me.

Running this:

python -m pip install pandas

Put pandas in the site-packages for the environment.

In fact pip seems to work by itself? I can't get it to put it in a different spot (as long as I run it from the environment).

So maybe the only caveat here is that you need a global python so that those commands can run.

[karthiknadig]

@karrtikr The switch to conda run during detection should handle this scenario, right? if not then this is a bug.

[karthiknadig]

In my case this happens:

There is no python that it can pick, so it eventually reverts to the pyenv shim.

In your case did it install to the base env or did it install to some global python. What does this get you python -c "import sys;print(sys.executable)"

[karrtikr]

The scenario where 2 environments share the same interpreter path was never supported, so this is not a bug at all. So far we haven't received users asking for this so I'm inclined to close this, unless I'm missing something?

[rchiodo]

We have (in jupyter). That could be because people doing notebooks use conda more often.

[greazer]

I just ran into this myself by accident.

conda create ocaml-kernel
conda activate ocaml-kernel

Everything seems fine, except I cannot find the environment in VS Code no matter what I do. Of course if I examine my environments with conda, I see it too. This is all highly confusing.

conda env list
# conda environments:
#
                         C:\JupyterLab\resources\jlab_server
base                  *  C:\Users\jimg\anaconda3
covid-analysis           C:\Users\jimg\anaconda3\envs\covid-analysis
donsrepro                C:\Users\jimg\anaconda3\envs\donsrepro
jupyter-lab-test-env     C:\Users\jimg\anaconda3\envs\jupyter-lab-test-env
my-jupyter-lab           C:\Users\jimg\anaconda3\envs\my-jupyter-lab
myenv                    C:\Users\jimg\anaconda3\envs\myenv
ocaml-kernel             C:\Users\jimg\anaconda3\envs\ocaml-kernel
try-new-ipython          C:\Users\jimg\anaconda3\envs\try-new-ipython
xeus-python              C:\Users\jimg\anaconda3\envs\xeus-python

I'm not sure I believe that this was never supported since most of the environments I've ever created I have used anaconda and I haven't specified a python version instance to install to it. In fact, the only reason I ever thought to do that was in case I wanted to use a different version of python than is available on the system. Perhaps I'm incorrect in that thinking, but it seems to make sense.

Also note if I do an internet search for "How do I create a new conda environment" the first link I get is this:
https://towardsdatascience.com/manage-data-science-and-machine-learning-environments-using-conda-42b9b2f0689a#:~:text=1%20Creating%20a%20New%20Conda%20Environment.%20You%20will,specify%20the%20version%20at%20the%20time%20of%20

It basically says what I just said above. :)

[brettcannon]

@greazer it's never been supported by us; see https://code.visualstudio.com/docs/python/environments#_where-the-extension-looks-for-environments and point 7:

Conda environments that contain a Python interpreter. VS Code does not show conda environments that don't contain an interpreter.

conda itself may be manifesting the interpreter lazily upon conda install or something.