Python Interactive Windows errors with vscode-remote

[vnijs]

Environment data

• VS Code version: insider-version 1.34.0
• Extension version (available under the Extensions sidebar): tried current and insider-version but makes no difference
• OS and version: Docker with ubuntu 18.04 on macOS Mojave
• Python version (& distribution if applicable, e.g. Anaconda): 3.6.7
• Type of virtual environment used: N/A

Expected behaviour

Python code running in the Python Interactive Window

Actual behaviour

Steps to reproduce:

1. Create a project directory with a sub directory with a .devcontainer/devcontainer.json file as follows:

{
	"name": "rsm-msba",
	"image": "vnijs/rsm-msba",
	"workspaceFolder": "/",
	"extensions": [
		"ms-python.python"
	]
}

2. Try to run the python code below in the Python interactive window. This works in the standard terminal but I get the strange output as shown in the screenshot above in the interactive window

cc-ing @chrmarti in case he has any comments/suggestions

#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------

#%%
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

# Data for plotting
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2 * np.pi * t)

fig, ax = plt.subplots()
ax.plot(t, s)

ax.set(xlabel='time (s)', ylabel='voltage (mV)',
       title='About as simple as it gets, folks')
ax.grid()

fig.savefig("plot.png")
plt.show()

print('Open test-project/plot.png to see the result!')


# %%

import os
print(os.getcwd())

[chrmarti]

It tries to run the script in bash. Not sure if this is an issue with the devcontainer or the Python extension. Hoping for insights from the Python extension team.

[rchiodo]

This wouldn't be us. This is jupyter running that command. We send it code to execute and it's not using python for some reason.

[rchiodo]

Can you attach your developer console output? (Help | Toggle Developer Tools, click on console, right click, save as)

[vnijs]

See attached with some errors:

remote-interactive-window.log

I also tried restarting the "rebuild container" option and that also produced a few errors.

remote-interactive-windows-restart.log

And one more when I (think I) just switched to the the devcontainer.json file.

remote-interactive-windows-change-file.log

[vnijs]

Just opening hello.py I just saw the follow flash by.

[rchiodo]

Yeah that looks like the extension isn't finding the python in use. Or at least one of them. It does start jupyter though.

What happens if you open a terminal and type 'python -m jupyter notebook --version'

[rchiodo]

Actually it looks like it's using 'python3', so type 'python3 -m jupyter notebook --version'

[vnijs]

Will check tonight. Weird that it can start jupyter without finding python though right?

[vnijs]

I think I found the issue. This container has multiple kernels, including R and Bash. When I deactivated the Bash kernel the python code does run but I still get the error shown below.

So there seems to be an issue with the extension using the correct kernel (based on the open file type). This seems directly related to the following issue:

https://github.com/microsoft/vscode-python/issues/3763

It would be great if you could specify which kernel(s) you would want to use in devcontainer.json. Even better if the user could select from a set of available kernels (https://github.com/microsoft/vscode-python/issues/3763#issuecomment-470265627)

FYI Requested output shown below:

python3 -m jupyter notebook --version
5.7.8

[chrmarti]

(The devcontainer.json can currently not include extension specific configuration.)

[DonJayamanne]

What's the value of python.pythonPath in the settings.json file?
What's displayed on the bottom left of the status bar of vscode? What version of python?
Please send a screenshot of vscode

[vnijs]

The only thing in the settings file is:

Screenshot vscode insider shown below. The selected python is python3

[vnijs]

I just noticed that there is a .vscode/settings.json file inside the directory I'm mounting into the container. "Run file in terminal" works fine but "Python Interactive" still seems to be trying to run the python code in the "Bash kernel"

{
    "python.pythonPath": "/usr/bin/python3"
}

[rchiodo]

We use the default kernel unless the user has ipykernel installed and we can change it. Can you check what 'python3 -m ipykernel --version' returns from the terminal window?

[vnijs]

From a terminal inside the container I get.

python3 -m ipykernel --version 7.2.0

[rchiodo]

Hmm. I wonder why it's not writing it then.

What does this list:

python3 -m jupyter kernelspec list

We use that to enumerate the installed ones and try to match the best one if ipykernel fails.

[vnijs]

I get the following:

python3 -m jupyter kernelspec list
Available kernels:
  bash       /usr/local/share/jupyter/kernels/bash
  ir         /usr/local/share/jupyter/kernels/ir
  python3    /usr/local/share/jupyter/kernels/python3

[rchiodo]

We must not be matching for some reason. I was thinking the python one might not show up or not say python in the name, but it does, so it should match.

In order to repro this you installed the bash and ir kernels into jupyter, right?
Or did you just start with a specific container that I might be able to also try?

[vnijs]

I created a Docker file with Jupyter and R and Bash kernels and the image is available on Docker hub as vnijs/rsm-msba. My current devcontainer.json is shown below, although I'm not able to access the specified ports after activating the container through vscode-insider. All kernels work fine and I can connect through the (local) browser just fine when not using vscode. More information and install instructions I use with my students linked below. Let me know if you need any more information. Thx!

https://github.com/radiant-rstats/docker/tree/master/install

{
	"name": "rady-test",
	"image": "vnijs/rsm-msba",
	"workspaceFolder": "rady",
	"extensions": [
		"ms-python.python",
		"christian-kohler.path-intellisense",
		"Ikuyadeu.r",
		"REditorSupport.r-lsp",
		"VisualStudioExptTeam.vscodeintellicode"
	],
	"appPort": [
		"127.0.0.1:8080:8080",
		"127.0.0.1:8787:8787",
		"127.0.0.1:8989:8989",
		"127.0.0.1:8765:8765"
	]
}

[rchiodo]

Thanks. That helps.

I'm able to repro now.

[vnijs]

Great. Thanks for looking into this @rchiodo. Please do let me know if I can test anything. A fix for automatic selection would be great but even better would be to have an option to set which kernel to start (or running kernel to connect to) like hydrogen has would be a fantastic feature to have (https://github.com/microsoft/vscode-python/issues/3763#issuecomment-470265627) 💯 :)

[rchiodo]

We have the selection for python on the bottom, it's supposed to use that one as your 'kernel' for the Python Interactive window.

Would you want something else (besides picking the python to use)? Other language kernels?

[vnijs]

Ah. I see. The python selection would be fine as is ... if it works :) but I would indeed really like to use R (and perhaps some other kernels) if possible in an interactive window where plots can be shown.

[rchiodo]

Using other languages is a long term goal, but unfortunately it's going to take a while :(

[vnijs]

Would be interested to hear what the challenges are to (1) connect to running kernels and (2) connect to non-python kernels.

Any idea if the kernel issue that started this issue is addressable?

[rchiodo]

1. Connecting to running kernels should be relatively easy. We just specify a different kernel when start our session.
2. Using other language kernels is much tougher. All of our internal code we run now is python based. We send a bunch of stuff to get plots and directories correct. Our code for requesting variables is all python based. So we'd have to rework these per language or figure out a language agnostic way to do the same thing. That's the part that's going to take awhile.

This bug should be addressable though. It's likely just picking the first kernel instead of the python one for some reason.

[vnijs]

I see. Thanks for the clarification @rchiodo. It seems that the approach in hydrogen is all JS based (https://github.com/nteract/hydrogen/tree/master/lib) so presumably easier to extend to other kernels. Any chance you could point me to where the relevant python code is in the repo? For some reason searches like "jupyter" don't give any code results. Just curious.

[rchiodo]

The variable explorer bits are probably the biggest amount of work. Not even sure they're possible in other languages. Other stuff is just doing things like setting the current directory and modifying matplotlib (not sure if there's an equivalent in say R)

We use the python files here to query variables from jupyter:
https://github.com/microsoft/vscode-python/tree/master/pythonFiles/datascience

[vnijs]

Thanks @rchiodo! A variable inspector for R (and other languages) would be nice but not critical. Just running (and plotting) would already be a huge plus.

[vnijs]

FYI I upgraded jupyter and other packages in the vnijs/rsm-msba container (see link below) and am now getting the following error when trying to run a cell in vscode 1.34.20-insider

image

jupyter-1.0.0.log

https://github.com/radiant-rstats/docker/blob/master/rsm-msba/requirements.txt

Everything works fine when I launch without vscode and in vscode with the mounted container I see the following information so jupyter is definitely available and installed in the container. This happens with both the release and insider version of the ms-python extension.

python3 -m ipykernel --version
7.5.0

Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jupyter_core
>>> import notebook
>>> jupyter_core.version_info
(4, 4, 0)
>>> notebook.version_info
(5, 7, 8)

[rchiodo]

Starting jupyter is giving this:

workbench.main.js:238 [Extension Host] Python Extension: Error: /usr/local/lib/python3.6/dist-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.2) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)

Sounds like urllib3 or chardet are out of date.

[rchiodo]

Note to self (or whoever gets this bug), here's the directions on debugging a remote extension:

https://code.visualstudio.com/api/advanced-topics/remote-extensions

[vnijs]

It seems this latest issue update I posted was a urllib3 version issue. I downgraded to urllib3==1.24 in the container (vnijs/rsm-msba:1.5.1) and now we are back to the "bash kernel being used instead of python3 kernel" issue we started with. Sorry for the confusion. I wrongly assumed this updated issue was related to changes in vscode-python.

[rchiodo]

@vnijs, Good news I just submitted a fix for this. It was a bug in our kernel spec matching algorithm. It was picking the bash kernel because it was using the same version of python under the covers.

Our next insider's build will have this fix in it.

[vnijs]

It is working @rchiodo. Thanks!