python.defaultInterpreterPath fails to resolve when using both ${workspaceFolder:name} and relative paths

[nrs-mparker]

Type: Bug

Behaviour

Expected vs. Actual

I'm trying to make a multi-root workspace for an Unreal Engine project that has Python in many different folders, and I've specified the paths to all of them as relative paths from the .code-workspace file. Unreal ships with its own copy of the interpreter, so all roots should use that same one. However, if I use workspaceFolder:name and a relative path from there, defaultInterpreterPath fails to resolve it and repeatedly prompts me to select a new interpreter. Here's a relevant snippet of the file:

"folders": [
	{
		"name": "MyGame",
		"path": "../.."
	},
	{
		"name": "Sequencer Scripting",
		"path": "../../../../../Engine/Plugins/MovieScene/SequencerScripting/Content/Python"
	}
],
"settings": {
	"python.defaultInterpreterPath": "${workspaceFolder:MyGame}/../../../Engine/Binaries/ThirdParty/Python3/Win64/python.exe"
},

Everything about the workspace functions well on my and another person's machine except the interpreter path. When I try to pick that one as the interpreter, I get:

2023-11-22 10:12:51.364 [warning] Identifier for virt-virtualenv failed to identify ..\..\..\Engine\Binaries\ThirdParty\Python3\Win64\python.exe [Error: ENOENT: no such file or directory, scandir 'C:\Engine\Binaries\ThirdParty\Python3\Win64'] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'scandir',
  path: 'C:\\Engine\\Binaries\\ThirdParty\\Python3\\Win64'
}

The workspace and all code files are not even on the C: drive, so it's clear that it's starting from something like where VS Code or the Python extension are installed instead. I've tried changing the workspaceFolders themselves to be absolute and got the same result, so expanding that doesn't seem to be the problem.

Expected: The interpreter path is resolved correctly and recognized as a valid interpreter.
Actual: The interpreter path is not resolved correctly, so no interpreter is found at that location.

Seems to be related to/part of #22452

Steps to reproduce:

1. Create a multi-root workspace with named folders (optional: paths to those folders are relative paths from the .code-workspace location).
2. Set python.defaultInterpreterPath in the .code-workspace to a path relative to one of the workspace folders. For example: "python.defaultInterpreterPath": "${workspaceFolder:named-folder}/../../python.exe"
3. Use the Select Interpreter command, and pick "Select at workspace level", then "Use Python from 'python.defaultInterpreterPath' setting".

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda):
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Unknown
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-11-29 13:43:35.769 [warning] Identifier for virt-virtualenv failed to identify ..\..\..\Engine\Binaries\ThirdParty\Python3\Win64\python.exe [Error: ENOENT: no such file or directory, scandir 'C:\Engine\Binaries\ThirdParty\Python3\Win64'] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'scandir',
  path: 'C:\\Engine\\Binaries\\ThirdParty\\Python3\\Win64'
}

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

Extension version: 2023.21.13331008
VS Code version: Code 1.84.2 (1a5daa3a0231a0fbba4f14db7ec463cf99d7768e, 2023-11-09T10:51:52.184Z)
OS version: Windows_NT x64 10.0.19045
Modes:

System Info

Item	Value
CPUs	AMD Ryzen Threadripper PRO 3975WX 32-Cores (64 x 3493)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	127.85GB (108.25GB free)
Process Argv	--crash-reporter-id 8a51896e-2a97-4832-af93-5c09d9200735
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30410667
py29gd2263:30899288
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
282f8724:30602487
f6dab269:30613381
showlangstatbar:30737416
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
welcomedialogc:30887144
pythonnosmt12:30797651
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
dsvsc013:30795093
dsvsc014:30804076
dsvsc015:30845448
pythontestfixtcf:30871695
pythonregdiag2:30871582
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
pythontbext0:30879054
accentitlementsc:30887149
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
aa_t_chat:30882232

[nrs-mparker]

After searching through a lot of other tickets, I eventually tried a "Clear Workspace Interpreter Setting" > "Clear all" (not "Clear at workspace level", and then redid the "Select Interpreter" > "Select at workspace level" > "Use Python from 'python.defaultInterpreterPath' setting". That seemed to miraculously fix everything, and opening files in multiple workspace folders no longer prompts me to select an interpreter. So I guess now this is a report that maybe that internal storage can get in a bad state that selecting the interpreter that way does not fix.

[karrtikr]

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority. Thanks for understanding and the patience!

[walsha2]

After searching through a lot of other tickets, I eventually tried a "Clear Workspace Interpreter Setting" > "Clear all"

@nrs-mparker This was it for me as well! After scouring all the issues and documentation, this was the only thing that finally got my defaultInterpreterPath to be honored.

It made debugging the VSCode workspace settings super difficult because you don't know if the setting is actually wrong, or of this clear operation needs to be triggered first.

@karrtikr Any idea where this "Clear Workspace Interpreter Setting" is caching that setting? An more importantly, why is it caching it at all and not just honoring whatever is defined in python.defaultInterpreterPath?

[karrtikr]

@walsha2 If you hover over the setting, we have a description stating how it works:

The primary purpose of "python.defaultInterpreterPath" is primarily to share settings among teams, and internal storage is meant to capture the actual active interpreter.

[karrtikr]

@nrs-mparker Appreciate your diagnosis, let us know if you run into such a bad state again and we'll be happy to reopen the issue and take a look:

2023-11-22 10:12:51.364 [warning] Identifier for virt-virtualenv failed to identify ..\..\..\Engine\Binaries\ThirdParty\Python3\Win64\python.exe [Error: ENOENT: no such file or directory, scandir 'C:\Engine\Binaries\ThirdParty\Python3\Win64'] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'scandir',
  path: 'C:\\Engine\\Binaries\\ThirdParty\\Python3\\Win64'
}

But even so, this error being a warning should not ideally affect the behavior. If you still face an issue, please provide the debug logs which can help diagnose the issue:

• Run Developer: Set log level command
• Select Trace
• Run Developer: Reload Window command
• Provide the output:

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

[nrs-mparker]

@karrtikr I have not encountered the issue again since my post, but I will be sure to do that if it reoccurs. But I do want to say that even though the output is only a warning, the real loss of functionality is in not having an interpreter selected and not being able to select the intended one. A user basically can't use the program in that state, though readers of this thread will see the workaround. I just provided that output in case it indicated to you or anyone else what the cause might be. Perhaps the next person to encounter this can set their logs to be verbose and reload before clearing out the interpreter settings.