Jupyter Extension in VSCode does not recognize updated environment variables from host's $HOME/.bashrc

[alien2327]

Applies To

• Notebooks (.ipynb files)
• Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

Environment

• Host OS: Ubuntu 22.04 LTS
• Client OS: Windows or Linux
• VSCode Details:
  • Version: 1.83.1
  • Commit: f1b07bd25dfad64b0167beb15359ae573aecd2cc
  • Date: 2023-10-10T23:45:31.402Z
  • Electron: 25.8.4
  • ElectronBuildId: 24154031
  • Chromium: 114.0.5735.289
  • Node.js: 18.15.0
  • V8: 11.4.183.29-electron.0
  • OS: Linux x64 6.2.0-35-generic snap
• Jupyter Extension Version: v2023.9.1102792234
• Python Kernel Version: 3.11.6
• juypter related version from pip list
  • jupyter 1.0.0
  • jupyter_client 8.5.0
  • jupyter-console 6.6.3
  • jupyter_core 5.4.0
  • jupyter-events 0.8.0
  • jupyter-lsp 2.2.0
  • jupyter_server 2.9.1
  • jupyter_server_terminals 0.4.4
  • jupyterlab 4.0.7
  • jupyterlab-pygments 0.2.2
  • jupyterlab_server 2.25.0
  • jupyterlab-widgets 3.0.9

Description

When using the Jupyter notebook kernel in VSCode via SSH, the extension does not recognize the latest environment variables written in the host OS's $HOME/.bashrc. Interestingly, it's not defaulting to a completely default set of environment variables. Instead, it seems to retain environment variables from a previous version of the .bashrc. Attempts to update variables like LD_LIBRARY_PATH and LIBRARY_PATH using %set_env do not resolve Python import errors related to these variables.

Steps to Reproduce:

1. Update environment variables in $HOME/.bashrc on the host OS.
2. Connect to the host through VSCode via SSH.
3. Start a Jupyter notebook in VSCode.
4. Attempt to import Python packages that depend on the newly set environment variables.
5. Observe the import error.

What I've tried:

• On the client side:
  • Reinstalled the Jupyter extension.
  • Restarted the PC.
• On the host side:
  • Deleted all files inside $HOME/.vscode-server.
  • Force-reinstalled Python 3.11.6 Jupyter package using --force-reinstall.
  • Checked and attempted to write environment variables in .bashrc, .bash_profile, and .profile.

Expected Behavior:

The Jupyter notebook kernel in VSCode should recognize and utilize the latest environment variables from the host's .bashrc.

Actual Behavior:

The kernel does not recognize the most recent environment variables, and instead seems to retain an older version of them.

Additional Context

What I was trying to accomplish

I was trying to utilize the GPAW, a Python-based DFT (Density Functional Theory) calculation tool. For the purpose of building GPAW:

• I installed AMD's aocc compiler and aocl numerical computation library in the /opt directory.
• Initially, only the path to AMD's aocc compiler was listed in the .bashrc. Currently, this is the only part that Jupyter in VSCode seems to recognize.
• Subsequently, I updated the LD_LIBRARY_PATH and LIBRARY_PATH in .bashrc to facilitate the building of GPAW using AMD's blis, flame and other related libraries.

What's working

• Using GPAW's CLI tools directly from the terminal works fine.
• Running Python scripts that import gpaw from the terminal also works without any issues.
• Initiating an IPython session from the terminal and using gpaw within that session is successful.
• Launching a Jupyter notebook server from the terminal and then using gpaw within those sessions also correctly recognizes the environment variables. This is consistent whether accessed directly or over SSH.

What's not working

When trying to use .ipynb notebooks directly in VSCode, the issue arises and the environment variables aren't recognized correctly, leading to failures in using gpaw.

VS Code Version

1.83.1

Jupyter Extension Version

v2023.9.1102792234

Jupyter logs

09:27:55.434 [info] Dispose Kernel process 3792078.
09:27:55.550 [warn] Failed to get activated env vars for /usr/bin/python in 60ms
09:27:55.551 [error] Unable to determine site packages path for python /usr/bin/python (Unknown)
09:27:55.593 [info] Process Execution: /usr/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
09:27:55.636 [info] Process Execution: /usr/bin/python -m ipykernel_launcher --ip=127.0.0.1 --stdin=9023 --control=9021 --hb=9020 --Session.signature_scheme="hmac-sha256" --Session.key=b"f3552177-c888-44c2-893d-40f46afb5ea7" --shell=9022 --transport="tcp" --iopub=9024 --f=~/.local/share/jupyter/runtime/kernel-v2-37917503NUyjZArDS4r.json
    > cwd: ~/Documents
09:27:55.777 [info] ipykernel version & path 6.26.0, ~/.local/lib/python3.11/site-packages/ipykernel/__init__.py for /usr/bin/python
09:27:56.003 [warn] StdErr from Kernel Process 0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
09:27:56.257 [info] Restarted 52972a18-2afb-40a4-bcaf-6a5ce0c2d4c9
09:28:02.348 [info] Handle Execution of Cells 0 for ~/Documents/test.ipynb
09:28:02.360 [info] Kernel acknowledged execution of cell 0 @ 1698280082359
09:28:03.181 [info] End cell 0 execution @ 1698280083181, started @ 1698280082359, elapsed time = 0.822s
09:28:05.676 [info] Handle Execution of Cells 1 for ~/Documents/test.ipynb
09:28:05.690 [info] Kernel acknowledged execution of cell 1 @ 1698280085689
09:28:05.692 [info] End cell 1 execution @ 1698280085692, started @ 1698280085689, elapsed time = 0.003s
09:28:07.090 [info] Handle Execution of Cells 2 for ~/Documents/test.ipynb
09:28:07.097 [info] Kernel acknowledged execution of cell 2 @ 1698280087097
09:28:07.294 [info] End cell 2 execution @ 1698280087293, started @ 1698280087097, elapsed time = 0.196s
09:28:07.294 [warn] Cancel all remaining cells due to cancellation or failure in execution

Coding Language and Runtime Version

Python v3.11.6

Language Extension Version (if applicable)

No response

Anaconda Version (if applicable)

No response

Running Jupyter locally or remotely?

Remote

[DonJayamanne]

Duplicate of #10707
microsoft/vscode-remote-release#1671