Scoped ${workspaceFolder:<folder>} doesn't resolve named folders in multi-root workspace

[OmeGak]

Behaviour

Potentially related to #18650.

Expected vs. Actual

I have a multi-root workspace and I name the folders for clarity in the file explorer and search results. The ${workspaceFolder} variable, however, doesn't seem to be resolved correctly when scoped by folder name in Python settings like python.defaultInterpreterPath. Given the following folders definition in my .code-workspace file:

{
    "folders": [
        {
            "name": "named-folder",
            "path": "org/projects/project-folder"
        }
    ]
}

I expect to be able to use ${workspaceFolder:named-folder} in python.defaultInterpreterPath:

{
    "settings": {
        "python.defaultInterpreterPath": "${workspaceFolder:named-folder}/.venv/bin/python",
    }
}

Instead, ${workspaceFolder:named-folder} is not resolved at all and I'm forced to use the implicit folder name as derived from its path:

{
    "settings": {
        "python.defaultInterpreterPath": "${workspaceFolder:project-folder}/.venv/bin/python",
    }
}

This is inconsistent with how ${workspaceFolder:<folder>} is resolved in other places. For instance, ${workspaceFolder:project-folder} will not resolve in a task when the folder is named like in my example above:

{
    "tasks": {
        "version": "2.0.0",
        "tasks": [
            {
                "label": "echo",
                "type": "shell",
                "command": "echo ${workspaceFolder:project-folder}",
            }
        ]
    },
}

Variable ${workspaceFolder:project-folder} can not be resolved. No such folder 'project-folder'.

Steps to reproduce:

1. Create a multi-root workspace with at least one explicitly named folder, like in my example above.
2. Set the defaultInterpreterPath to ${workspaceFolder:<explicit-folder-name>}/<relative-path-to-python-interpreter>
3. Observe in the Python output that the ${workspaceFolder} was not resolved and, thus, no interpreter is found.

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.16
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Unknown
• Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-11-09 12:16:51.050 [warning] Identifier for virt-virtualenv failed to identify /Users/ome/Code/org/projects/project-folder/${workspaceFolder:named-folder}/.venv/bin/python [Error: ENOENT: no such file or directory, scandir '/Users/ome/Code/org/projects/project-folder/${workspaceFolder:named-folder}/.venv/bin'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'scandir',
  path: '/Users/ome/Code/org/projects/project-folder/${workspaceFolder:named-folder}/.venv/bin'
}

[OmeGak]

This is not a feature-request but reporting from the built-in issue reporter in VSCode was not working.

[karrtikr]

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority. Thanks for understanding and the patience!

[OmeGak]

I'll be happy to try and fix it myself. If I understand correctly, the bug is coming from this line of code unconditionally setting the workspace folder name to the path basename.

vscode-python/src/client/common/variables/systemVariables.ts

Lines 110 to 111 in a1fac81

	this._workspaceFolder = workspaceFolder ? workspaceFolder.uri.fsPath : rootFolder || __dirname;
	this._workspaceFolderName = Path.basename(this._workspaceFolder);

[karrtikr]

I see, so what're you expecting it to be set to?

[OmeGak]

If a folder is explicitly named, I would be expecting that one to be set as this._workspaceFolderName, rather than the basename of the path. From what I can see, WorkspaceFolder has a name attribute and I'm guessing that that is the one that stores the explicitly set name. If that is the case, my naive solution would be this:

this._workspaceFolderName = workspaceFolder.name || Path.basename(this._workspaceFolder);

[OmeGak]

On a different note, I'm surprised that there is no canonical way to resolve workspace folder names in VSCode. This leaves how to resolve the scoped ${workspaceFolder} variable up to extension authors, clearly leading to inconsistent behavior.

Also, the official documentation remains ambiguous as to how scoped ${workspaceFolder} variables should be resolved. Here, the documentation says nothing about the name attribute in workspace folders. Here, the documentation states that the name attribute in workspace folders is for display purpose in the Explorer. However, [here],(https://code.visualstudio.com/docs/editor/variables-reference#_how-can-i-know-a-variables-actual-value) the documentation implies that the canonical way to resolve variables is via echo in a task.

Then, as I mentioned in the description, if a workspace folder is named, this task will fail:

"echo ${workspaceFolder:<workspace-folder-path-basename>}"

While this one will work:

"echo ${workspaceFolder:<workspace-folder-name>}"

It may be worth eventually resolving this directly on VSCode to avoid confusion. I would imagine:

• A resolveName() or resolveScope() function in the WorkspaceFolder class that can be used by VSCode and extensions to resolve the scope canonically.
• Clarifying here that scoped ${workspaceFolder} variables are resolved with either their explicitly set name or with their path basename.
• Clarifying here that the name attribute in workspace folder is not used only for display in Explorer but also to scope ${workspaceFolder} variables.

What would you reckon is the best way to bring attention of VSCode maintainers to this?

[karrtikr]

Feel free to create an issue on VS Code's end: https://github.com/Microsoft/vscode/issues. There's a hesitancy of supporting such variables due to similar reasons mentioned in #18650 (comment). But if we mention that a similar functionality already exists: https://code.visualstudio.com/docs/editor/variables-reference#_variables-scoped-per-workspace-folder, with the support ideas you mentioned, it's more likely for them to add support.

[karrtikr]

Regarding the issue in hand, name is supposed to be same as basename according to VS Code docs:

Could you elaborate on what is the value of each in your scenario?

[karrtikr]

Added support to also resolve using using WorkspaceFolder.name: #22471

Workspace folder name can be relative whereas basename is always absolute, hence they can be different.

[karrtikr]

Fix should be out in the pre-release version of the extension in a day, use the following to try it out:

[OmeGak]

Thanks for the quick reaction, @karrtikr! I tried the pre-release version and it's working as expected now.

[nrs-mparker]

I was running into this issue yesterday and came across this thread while looking for fixes. I've tried the pre-release version, but it did not fix it for me. I'm new to VS Code, so it's possible that I'm attempting something that's not supported. Basically I'm trying to make a multi-root workspace for an Unreal Engine project that has Python in many different folders, and I've specified the paths to all of them as relative paths from the .code-workspace file. Unreal ships with its own copy of the interpreter, so all roots should use that same one. But it seems some combination of all this relative pathing breaks the folder resolution. Here's a relevant snippet of the file:

"folders": [
	{
		"name": "MyGame",
		"path": "../.."
	},
	{
		"name": "Sequencer Scripting",
		"path": "../../../../../Engine/Plugins/MovieScene/SequencerScripting/Content/Python"
	}
],
"settings": {
	"python.defaultInterpreterPath": "${workspaceFolder:MyGame}/../../../Engine/Binaries/ThirdParty/Python3/Win64/python.exe",
	"python.analysis.extraPaths": [
		"../../Intermediate/PythonStub"
	],
	"python.autoComplete.extraPaths": [
		"../../Intermediate/PythonStub"
	]
},

Everything about the workspace functions well on my and another person's machine except the interpreter path. When I try to pick that one as the interpreter, I get:

2023-11-22 10:12:51.364 [warning] Identifier for virt-virtualenv failed to identify ..\..\..\Engine\Binaries\ThirdParty\Python3\Win64\python.exe [Error: ENOENT: no such file or directory, scandir 'C:\Engine\Binaries\ThirdParty\Python3\Win64'] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'scandir',
  path: 'C:\\Engine\\Binaries\\ThirdParty\\Python3\\Win64'
}

The workspace and all code files are not even on the C: drive, so it's clear that it's starting from where VS Code or the Python extension are installed instead. Please let me know if there's something unsupported I've tried here. The extension version is v2023.21.13261010.

[nrs-mparker]

@karrtikr Sorry to comment again, but I wasn't sure if you got a notification for comments since the issue is Closed. We also had the long holiday of course, so I don't expect anything to be addressed already, but neither do I want it to fall through the cracks.

As an interesting side note, when I go to select my interpreter to get the output above, I'm picking "Select at workspace level", then "Use Python from 'python.defaultInterpreterPath' setting". I noticed that there is a darkened path/tooltip next to that text showing what that path becomes, and the path is correct there (other than having mixed path separators because I'm on Windows but used forward slashes). The "Recommended" interpreter in that menu with the star icon has the same path next to it (and produces the same issue).

I tried changing my path separators to backslashes in my workspace folder and defaultInterpreterPath just in case that was the problem. When I tried after that, I get a notification saying "An Invalid Python interpreter is selected for workspace ..." that I did not get before. That could also be related to when I save the .code-workspace file though, as it also appears when I change it back and save again.

Another interesting thing is that darkened path/tooltip had been evaluated from relative to absolute to show me exactly where it was pointing, except the part that came after ${workspaceFolder:MyGame}. Not sure if that's any indication of failing to fully convert to an absolute path.

[karrtikr]

@nrs-mparker Can you open a new issue with these details please? We'll be happy to have a look. Generally we don't track comments in already fixed issues so you'll be better off creating a new one.