Allow a machine to configure the default Python interpreter version

[lostintangent]

Codespaces recently updated its base image to pre-install Jupyter along with a handful of ubiquitous ML libraries (e.g. pandas, torch, etc.). These libraries are installed into the context of the latest version of Python that's installed (currently 3.10.4), which is also set as the default Python version in the user's $PATH. Because of this, it would be ideal if when a user opened an *.ipynb file in a Codespace, that we could have it automatically select the kernel for the latest Python version. That way, there's a seamless continuity between the terminal and notebooks, without the user needing to select a kernel manually, and potentially get confused.

For example, the following is a screenshot of what the kernel picker looks like in a Codespace, and it would be awesome if we could somehow configure the environment to pre-select Python 3.10.4 64-bit.

[lostintangent]

Interestingly enough, if I run the Create: New Jupyter Notebook command, the new untitled notebook seems to automatically select the desired kernel version. But as soon as you save the untitled notebook, it reloads, and the kernel picker switches back to Select kernel.

So if would be cool if there was a way to automatically select a preferred Python version for *.ipynb files, in the same way that appears to occur for untitled files.

[IanMatthewHuff]

Yeah, this suggestions ties in well with a lot of work that we are currently doing around selecting a preferred kernel.

microsoft/vscode-jupyter#9615

Currently when we open up a new blank notebook file without any existing metadata if the language is Python then we actually delegate to using the current active Python interpreter from the Python extension for our preferred kernel. I would actually expect that in this case (without tweaking any python extension settings) that the $PATH setting would be honored by the python extension and the correct kernel would be selected. I think that is what you are seeing the Create: New Jupyter Notebook case. If you are getting something different when opening a .ipynb I believe that there might be something in that file's metadata that's causing the Jupyter extension to prefer a different kernel from the ones available.

Just to mention, going forward one of our considerations might be to allow configuring a default kernel on the user or workspace level and have that win out over any heuristic guesses or python extensions settings. Might be an option for better supporting Codespaces.

[IanMatthewHuff]

Is this change live in Codespaces? I'm not a regular code spaces user myself, but I'd love to take a peek to see what the default behavior looks like here.

One other side note, I'm not sure how the lifetime of codespaces are handled, but in general installing packages and using the main python interpreter directly as a kernel is something that we are looking at moving away from. We have lots of issues with global python installs getting polluted with module installs and getting folks into bad situations. What might be a better solutions here is to instead install a separate jupyter kernel with DS packages installed independent of the main python interpreter.

[lostintangent]

Yep this makes sense, and the changes I mentioned above are live in Codespaces.

Since Codespaces are ephemeral and meant to be used as project/task-specific environments, we chose to install Jupyter and ML libs in the global installation, since that allowed users to have a "zero config" onboarding experience, without needing to deal with virtual environments. We might evolve that over time, but this was also the recommendation by @brettcannon.

[IanMatthewHuff]

Yeah, I'd defer to Brett on that. Given the statement about ephemerality of codespaces working with the global interpreter seems fine.

This bit here does sound like at least a bit of a bug:

But as soon as you save the untitled notebook, it reloads, and the kernel picker switches back to Select kernel.

There is a known issue with the core notebook that when you save it, it loses the current controller selection. However when it is saved it should then run the same algorithm as when first created. If it correctly selects the right kernel when created it should also reselect that kernel when saved. I'll take a peek at that.

[DonJayamanne]

Transferring to Python extension, as this should ideally be fixed in the Python extension.
I.e. if Python extension were to have the Active Interpreter of the workspace point to the item thats been suggested, then the Jupter extension would pick that up

[brettcannon]

I'm not quite following from this discussion what's not working or expected here from the Python extension side. Is it due to the fact that there are two Python 3.10 installs and we select the wrong one by default?

[karrtikr]

These libraries are installed into the context of the latest version of Python that's installed (currently 3.10.4), which is also set as the default Python version in the user's $PATH

As I understand the ask is to select the default Python version on $PATH automatically, in that case this is a dup of #19818. Let me know if that's not the case and we can reopen the issue.