Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jupyter notebook doesn't load env variables #8270

Closed
ListIndexOutOfRange opened this issue Nov 12, 2021 · 14 comments
Closed

Jupyter notebook doesn't load env variables #8270

ListIndexOutOfRange opened this issue Nov 12, 2021 · 14 comments
Assignees
Labels
info-needed Issue requires more information from poster

Comments

@ListIndexOutOfRange
Copy link

Does this issue occur when all extensions are disabled?: Can't check because i only have a remote ssh access to the concerned computer.

  • VS Code Version:
    Version: 1.62.1
    Commit: f4af3cbf5a99787542e2a30fe1fd37cd644cc31f
    Date: 2021-11-05T10:56:50.217Z
    Electron: 13.5.2
    Chrome: 91.0.4472.164
    Node.js: 14.16.0
    V8: 9.1.269.39-electron.0
    OS: Linux x64 5.4.0-89-generic snap (Ubuntu 18.04.5 LTS)

I'm not sure how to reproduce this issue since it mysteriously stopped working overnight. Anyway, here what happens:

  • I'm running vs-code remotely via SSH. I'm running python inside a venv. I have a variable inside my .bashrc file alike:
export SOME_VARIABLE=/some/path/

so that in a python file, os.getenv('SOME_VARIABLE') works just fine.

  • However inside a jupyter notebook (with the good kernel), this variable is set to None. If it can help, the PYTHONPATH is:
    /path/to/my/home/.vscode-server/extensions/ms-toolsai.jupyter-2021.10.1101450599/pythonFiles:/path/to/my/home/.vscode-server/extensions/ms-toolsai.jupyter-2021.10.1101450599/pythonFiles/lib/python

  • I tried setting:

    "terminal.integrated.inheritEnv": true,
    "terminal.integrated.defaultProfile.linux": "bash",

in the json settings file but it didn't change anything.

I guess this is a jupyter bug since it works in a python file.

@rebornix rebornix transferred this issue from microsoft/vscode Nov 16, 2021
@rchiodo
Copy link
Contributor

rchiodo commented Nov 29, 2021

Thanks for the bug.

Can you include the output of the 'jupyter' output tab? We should be starting the venv the same way as normal python but something might not be working.

@rchiodo rchiodo added info-needed Issue requires more information from poster and removed needs-triage labels Nov 29, 2021
@rchiodo rchiodo self-assigned this Nov 29, 2021
@Bai-YunHan
Copy link

Bai-YunHan commented Dec 1, 2021

Having the same issue.
Both python/ipython interactive console can read the environment variable.
And the environment variable is also accessible through echo $SOME_VARIABLE in terminal
But Jupiter fails to get the environment variable set in .bashrc.

@heatxg
Copy link

heatxg commented Feb 4, 2022

Having the same issue.
Has anyone come across a solution?

@rchiodo
Copy link
Contributor

rchiodo commented Feb 4, 2022

We don't run your bash rc script before starting a kernel. You'd need to launch VS code from a bash shell and then it should inherit it.

@heatxg
Copy link

heatxg commented Feb 4, 2022

@rchiodo Even if I export the environment variables through the terminal within VSCode before or while the jupyter notebook is active, I am unable to access them.

@rchiodo
Copy link
Contributor

rchiodo commented Feb 4, 2022

Even if I export the environment variables through the terminal within VSCode before or while the jupyter notebook is active, I am unable to access them.

Sorry I'm not following what you're doing here.

VS code itself has to be started from a terminal with the environment variables set. Like so:

rich@CORTANA:~/test$ export FOO=BAR
rich@CORTANA:~/test$ code . <-- This is launching VS code from this folder. It will inherit FOO=BAR

image

@rchiodo rchiodo closed this as completed Feb 4, 2022
@heatxg
Copy link

heatxg commented Feb 4, 2022

@rchiodo I am ssh into a remote machine with VSCode, not launching from the machine. Then I open the notebook and attempt to access the environment variables. I have tried:

  • setting them in .bashrc
  • exporting them through a terminal in VSCode before opening the jupyter notebook
  • exporting them through a terminal in VSCode after opening the jupyter notebook

I am having no success with any of those approaches.

@rchiodo
Copy link
Contributor

rchiodo commented Feb 4, 2022

Sorry but that's not supported. Your .bashrc won't be run before we start a kernel.

You could workaround this by setting this value to set some environment variables:

"jupyter.runStartupCommands": []

Or we could change this issue to an enhancement and add an option to run a kernel from a shell script instead of launching it directly.

@rchiodo
Copy link
Contributor

rchiodo commented Feb 4, 2022

This issue is likely relevant too. It seems weird to me to run a kernel from a shell script, but an .env should work.

@heatxg
Copy link

heatxg commented Feb 4, 2022

I agree, that feels weird. Thank you for the redirect. I will take a look.

@onnoeberhard
Copy link

onnoeberhard commented Jun 24, 2022

I'm not sure if this is the same problem, but for me in a notebook in VS Code import os; print(os.environ['PATH']) outputs /Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/Users/onno/mpi/.venv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
(a ridiculous output)

In the integrated terminal, with the same virtual environment activated, python -c "import os; print(os.environ['PATH'])" outputs /Users/onno/mpi/.venv/bin:/Users/onno/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/homebrew/bin:/opt/homebrew/sbin.

(Critically, the TeX directory is missing in the notebook. This is not set in .zshrc, but in /etc/profile). When I open a jupyter notebook in the browser from the terminal, I get the correct path, the same as Python in the terminal gives me. I even started VS Code by typing code .

I'm not sure what to do.. I tried restarting everything including my computer, but nothing changed.

Update: Apparently this is related to a different issue. This comment helped me solve the problem: #9774 (comment)

@AminnimAMoh
Copy link

@rchiodo That is the Perfect answer. The content of .bashrc used to be pre-loaded to the Jupyter before, but it stopped working for some reason. Lunching VSCode from a bash terminal fixed the issue for me. Many thanks.

@nateGeorge
Copy link

nateGeorge commented May 21, 2023

I find it amazing that something as basic as using ~/.bashrc to load env variables in a remote VS code server with Azure cloud ML is made so difficult it's basically impossible. Instead you can use python-dotenv:

from dotenv import load_dotenv

load_dotenv()

You need a .env file in the directory you're running this from (or specify the path I think in load_dotenv). The file should have pairs like API_KEY="12345".

@nickovchinnikov
Copy link

The same issue was reproduced again, and the solution is in this comment
#9774 (comment)

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
info-needed Issue requires more information from poster
Projects
None yet
Development

No branches or pull requests

9 participants