Internal FileNotFoundError error when using autoReload with a drive directly in the watch list

[john-aws]

Environment data

• VS Code version: 1.61.2
• Extension version (available under the Extensions sidebar): v2021.10.1365161279
• OS and version: Windows 10
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.9.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A
• Relevant/affected Python-related VS Code extensions and their versions: N/A
• Value of the python.languageServer setting: Default

[NOTE: If you suspect that your issue is related to the Microsoft Python Language Server (python.languageServer: 'Microsoft'), please download our new language server Pylance from the VS Code marketplace to see if that fixes your issue]

Expected behaviour

Clicking Debug Python File in Terminal would run my Python script cleanly.

Actual behaviour

Debugging works fine, but the Python Debug Console shows an internal FileNotFoundError error in the extension/pydev, per below. This specifically happens when I am working with Python source code on a subst'd drive (S: which is a subst of a local OneDrive folder). If, instead of opening folder S:, I open the real, underlying folder that is subst'd to S: (and that real folder is beneath C:\Users\xxxx\OneDrive\Documents\xyz) then everything works cleanly. The problem is only apparent when on the subst'd drive.

S:\> cmd /C "C:\Users\xxxx\AppData\Local\Microsoft\WindowsApps\python3.9.exe c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.10.1365161279\pythonFiles\lib\python\debugpy\launcher 58255 -- s:\69723661.py "
pydev debugger: critical: unable to get real case for file. Details:
filename: s:\
drive: S:\
parts: ['']
(please create a ticket in the tracker to address this).
Traceback (most recent call last):
  File "c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.10.1365161279\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 221, in _get_path_with_real_case
    return _resolve_listing(drive, iter(parts))
  File "c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.10.1365161279\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 192, in _resolve_listing
    raise FileNotFoundError('Unable to find: %s in %s' % (
FileNotFoundError: Unable to find:  in S:\

During handling of the above exception, another exception occurred:

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. Create a new folder on your C: drive e.g. c:\users\me\testing
2. Add a sample Python script to that folder
3. Open the folder in VS Code, then open the sample Python script
4. Click Debug Python File in Terminal, result is (hopefully) a clean run and the Python Debug Console output appears normal
5. Use the Windows subst utility to map S: to c:\users\me\testing
6. Open the S:\ folder in VS Code, then open the (same) sample Python script
7. Click Debug Python File in Terminal, this time the resulting Python Debug Console output should include an exception similar to that shown above

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

User belongs to experiment group 'pythonaa'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'pythonSurveyNotificationcf'
User belongs to experiment group 'PythonPyTorchProfiler'
User belongs to experiment group 'pythonDeprecatePythonPath'
User belongs to experiment group 'pythonSortEnvs'
User belongs to experiment group 'pythonRunFailedTestsButtonDisplayed'
User belongs to experiment group 'pythonRefreshTestsButtonDisplayed'
User belongs to experiment group 'pythonRememberDebugConfig'
> conda info --json
> ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe ~\.vscode\extensions\ms-python.python-2021.10.1365161279\pythonFiles\interpreterInfo.py
> ~\.anaconda\Scripts\conda.exe info --json
> ~\.conda\Scripts\conda.exe info --json
> ~\.condarc\Scripts\conda.exe info --json
Python interpreter path: ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe
Starting Microsoft Python language server.
> conda --version
> ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe -m black --diff --quiet s:\69723661.py.297d2e6504dc26c5cfee95a8a0b60256.tmp
cwd: s:\
> ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe -m black --diff --quiet s:\69723661.py.297d2e6504dc26c5cfee95a8a0b60256.tmp
cwd: s:\
> ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe -m black --diff --quiet s:\69723661.py.297d2e6504dc26c5cfee95a8a0b60256.tmp
cwd: s:\
> ~\AppData\Local\Microsoft\WindowsApps\python3.9.exe -m black --diff --quiet s:\69723661.py.297d2e6504dc26c5cfee95a8a0b60256.tmp
cwd: s:\

[int19h]

We had some issues with subst before, but I don't think any of them are quite like this one.

[john-aws]

Hugely underutilized feature of Windows imo. Makes it much simpler to reproduce an environment on multiple computers, for example. Also great for cloud drive mapping (for example D: for DropBox, G: for Google Drive, O: for OneDrive etc.)

[fabioz]

I've tried to reproduce this locally but I wasn't able to do it with the given instructions.

Can someone that can reproduce it provide the logs? (@john-aws some crucial element may be missing from the instructions to reproduce and I hope that the logs can provide more info on that)

i.e.:

• Open VS Code
• Select the command Extensions: Open Extensions Folder
• Locate the Python extension directory, typically of the form ms-python.python-2021.*.****
• In that directory ensure you do not have any debug*.log files, if you do, please delete them
• Go back into VS Code and modify your launch.json to add the setting "logToFile": true, see below:

{
    "name": "Python: Current File (Integrated Terminal)",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "console": "integratedTerminal",
    "logToFile": true
}

• Start debugging
• When done, go back into the extension directory and upload the debug*.log files into this GitHub issue.

i.e.: working for me...

[john-aws]

Thanks for the follow-up. It looks like you are invoking the Python debugger in a different way to me. Specifically, you are choosing it from the launch dropdown at the top of the left-hand navigation pane. If I debug the code from there, it works fine.

However, I'm not using that. I'm using a Run Python File/Debug Python File dropdown which appears top-right of the VS Code window, between the last open tab and the Actions icons, thus:

I presume this menu is a feature of the VS Code debugpy debugger, but apologies if it's actually unrelated.

The difference between what these two debug launch options do is as follows.

Working:

S:\> s: && cd s:\ && cmd /C "set "DEBUGPY_LOG_DIR=c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.12.1559732655" && C:\Users\xxxx\AppData\Local\Microsoft\WindowsApps\python3.9.exe c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\launcher 50218 -- s:\69723661.py "

Failing:

S:\> cmd /C "C:\Users\xxxx\AppData\Local\Microsoft\WindowsApps\python3.9.exe c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\launcher 50183 -- s:\69723661.py "
pydev debugger: critical: unable to get real case for file. Details:
filename: s:\
drive: S:\
parts: ['']
(please create a ticket in the tracker to address this).
Traceback (most recent call last):
  File "c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 221, in _get_path_with_real_case
    return _resolve_listing(drive, iter(parts))
  File "c:\Users\xxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 192, in _resolve_listing
    raise FileNotFoundError('Unable to find: %s in %s' % (
FileNotFoundError: Unable to find:  in S:\

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...

In the former, working case, a debug log is written to the extension sub-folder. In the latter, failing case (the subject of this issue), no debug log is written (and hence I have not attached one). I'm hoping the information provided above will suffice.

[fabioz]

Thanks for the additional info.

The problem isn't really related to subst, rather the file translation isn't able to translate a filename with just the drive directly.

This can be reproduced with a launch with autoReload specified (which is the case when you use that shortcut).

            "autoReload": {
                "enable": true,
                "watchDirectories": ["S:\\"]
            },

As a note, everything should still work as usual in the debugger (even though it prints that error, that error is handled).

[fabioz]

I've just provided a fix for this.

@john-aws if you can apply the fix locally and check if it works for you that'd be nice...

To do that, you can run in debug a file which does:

import pydevd_file_utils
print(pydevd_file_utils.__file__)

Then open that file and do the change from https://github.com/microsoft/debugpy/pull/804/files at the pydevd_file_utils.py file in line 221 .

[john-aws]

Confirmed that this is working correctly with the patch, thanks.