Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Creating a new Conda Env creates a ton of calls to onDidChangeEnvironments #19900

Closed
IanMatthewHuff opened this issue Sep 27, 2022 · 5 comments · Fixed by #19922
Closed

Creating a new Conda Env creates a ton of calls to onDidChangeEnvironments #19900

IanMatthewHuff opened this issue Sep 27, 2022 · 5 comments · Fixed by #19922
Assignees
@karrtikr
Labels
area-environments Features relating to handling interpreter environments feature-request Request for new features or functionality needs PR Ready to be worked on verification-needed Verification of issue is requested verified Verification succeeded
Milestone
October 2022

Comments

@IanMatthewHuff
Copy link
Member

IanMatthewHuff commented Sep 27, 2022 •
edited by karrtikr
Loading

Testing #19886

The API around onDidChangeEnvironments didn't seem to do what I was expecting when I created a new conda env. I got a ton of calls to this function, an add and an update for the new env were some where in the list, but I was just expecting an add and maybe an update for the new env. Instead it seems like I got an update for basically everything in my python list.

Repro:

  1. Created extension for pythonAPI usage
  2. Hooked up to onDidChangeEnvironments
  3. Logged e.env.name and e.env.path and e.type on onDidChangeEnvironments
  4. In a terminal, did conda create --name endgameEnv4 python=3.9
  5. I expect an add / update for the new env, instead I got the below

Debug Logging

Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.10.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.13/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/bin/python3 update
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python3 update
Extension is using deprecated python APIs which will be removed soon
Changed Env: base /Users/ianhuff/opt/miniconda3/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: base /Users/ianhuff/opt/miniconda3/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.10.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.13/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/bin/python3 update
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python3 update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: base /Users/ianhuff/opt/miniconda3/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.10.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.13/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/bin/python3 update
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python3 update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: base /Users/ianhuff/opt/miniconda3/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.10.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.13/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/bin/python3 update
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python3 update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: base /Users/ianhuff/opt/miniconda3/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Changed Env: .venv /Users/ianhuff/Documents/DataScience/WWBD/.venv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.10.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.0/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /Users/ianhuff/.pyenv/versions/3.9.13/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon
Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/bin/python3 update
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python update
Extension is using deprecated python APIs which will be removed soon
Changed Env: undefined /usr/local/bin/python3 update
Extension is using deprecated python APIs which will be removed soon
Extension is using deprecated python APIs which will be removed soon

Maybe this is expected? But It was not what I was expecting out of this event.
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Sep 27, 2022
@karrtikr karrtikr mentioned this issue Sep 27, 2022
Merged
@karrtikr
Copy link

This seems expected as currently we trigger a full refresh if a new environment is detected, I think that's why so many changed events were emitted. But I don't see endgameEnv4 emitted in the logs, was the environment visible in environment.all property?

@karrtikr karrtikr added the info-needed Issue requires more information from poster label Sep 27, 2022
karrtikr pushed a commit that referenced this issue Sep 27, 2022
Only log deprecation warning once for each extension (#19901)
a500e16 
For #19900
@IanMatthewHuff
Copy link
Member Author

Sorry the formatting was all wonky on that log. I just tried it again with a new env and I do see the new env (endgameEnv5) getting an add and an update.

Changed Env: endgameEnv5 /Users/ianhuff/opt/miniconda3/envs/endgameEnv5/bin/python add
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:94
Changed Env: jupyterEnv /Users/ianhuff/opt/miniconda3/envs/jupyterEnv/bin/python update
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:94
Changed Env: endgameEnv5 /Users/ianhuff/opt/miniconda3/envs/endgameEnv5/bin/python update

Note: This is in with all the others.

Still seems a bit wonky to me that I get all the other updates, just from an API expectation it's not what I was thinking, but I guess the consumers can manage things themselves if they are storing the results and just comparing on what came before.

There also seem to be some entries like this that get added and removed right after so they don't ever show up in .all. Not sure what these are.

Changed Env: pipenvFolder-qBnzia5Z /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python add
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:94
Changed Env: undefined /usr/bin/python3 update
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:94
Changed Env: undefined /Users/ianhuff/.local/share/virtualenvs/pipenvFolder-qBnzia5Z/bin/python remove

@karrtikr karrtikr added area-environments Features relating to handling interpreter environments and removed info-needed Issue requires more information from poster labels Sep 27, 2022
@karrtikr
Copy link

karrtikr commented Sep 28, 2022 •
edited
Loading

There also seem to be some entries like this that get added and removed right after so they don't ever show up in .all. Not sure what these are.

When running a python binary fails, they're removed from the list. Python debug logs should contain the error in detail as to why they were invalid.

@karrtikr
Copy link

karrtikr commented Sep 28, 2022 •
edited
Loading

About the API itself, I'll see if there's some way that we can only fire an update for the changed environment. I see two approaches:

  • Trigger a scoped refresh instead of a full refresh.
    For eg. in this case this would mean we only trigger a refresh from the conda locator as that is responsible for emitting the event. This would still lead in events from all conda environments.

  • Change how we do a full refresh.
    Do not update envs during a refresh, and wait for refresh to complete before deciding whether to update an env. This would mean all those extra events won't be emitted as envs haven't really changed.

@karrtikr karrtikr added needs proposal Need to make some design decisions feature-request Request for new features or functionality and removed triage-needed Needs assignment to the proper sub-team labels Sep 28, 2022
@karrtikr karrtikr modified the milestones: October 2022, September 2022 Sep 28, 2022
@karrtikr karrtikr mentioned this issue Sep 28, 2022
Merged
@karrtikr karrtikr modified the milestones: September 2022, October 2022 Sep 28, 2022
@karrtikr karrtikr mentioned this issue Sep 29, 2022
Merged
eleanorjboyd pushed a commit to eleanorjboyd/vscode-python that referenced this issue Oct 4, 2022
@eleanorjboyd
Only log deprecation warning once for each extension (microsoft#19901)
e20334d 
For microsoft#19900
karrtikr pushed a commit that referenced this issue Oct 6, 2022
Wait for refresh to complete before deciding whether an environment h…
ce62901 
…as updated (#19902)

For #19900

Once environment details have loaded, ensure we do not generate change
events for environments if it hasn't really changed.
@karrtikr karrtikr added needs PR Ready to be worked on and removed needs proposal Need to make some design decisions labels Oct 6, 2022
karrtikr pushed a commit that referenced this issue Oct 12, 2022
Make lookup for new environments detected faster (#19922)
76f7763 
Closes #19900

Trigger a scoped refresh instead of a full refresh when an environment
gets created, preferably only looking for the environment which gets
created.
@karrtikr karrtikr added the verification-needed Verification of issue is requested label Oct 25, 2022
@IanMatthewHuff IanMatthewHuff added the verified Verification succeeded label Oct 25, 2022
@IanMatthewHuff
Copy link
Member Author

Looks good. I just see a single call now when adding a new env:
image

eleanorjboyd pushed a commit to eleanorjboyd/vscode-python that referenced this issue Nov 7, 2022
@eleanorjboyd
Only log deprecation warning once for each extension (microsoft#19901)
75bfc97 
For microsoft#19900
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 25, 2022
wesm pushed a commit to posit-dev/positron that referenced this issue Mar 28, 2024
@wesm
Make lookup for new environments detected faster (microsoft/vscode-py…
454ad5c 
…thon#19922)

Closes microsoft/vscode-python#19900

Trigger a scoped refresh instead of a full refresh when an environment
gets created, preferably only looking for the environment which gets
created.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@karrtikr karrtikr

Labels
area-environments Features relating to handling interpreter environments feature-request Request for new features or functionality needs PR Ready to be worked on verification-needed Verification of issue is requested verified Verification succeeded
Projects
None yet
Milestone
October 2022
Development

Successfully merging a pull request may close this issue.

Make lookup for new environments detected faster karrtikr/vscode-python
2 participants
@IanMatthewHuff @karrtikr