Spike - Use 'conda run' when debugging user code

[kimadeline]

Use conda run when debugging user code, sequel to the conda run work in #7696.
Modify the in-memory config (in the debug configuration resolver) to add 'conda run' if necessary, @karthiknadig is it only for launch or for both launch and attach?

[karthiknadig]

These notes are for the python debug adapter only.

• For attach there is no python involved from the extension side so there is nothing to do there. Also the communication happens over socket so it is not impacted by conda run weirdness with streaming stdin/stdout.
• For launch:
  • The adapter itself should be started using the python binary. This is because the python adapter communicates with VS Code via stdin/stdout. conda run does not stream hence this prevents any communication between extension and adapter. So ensure DebugAdapterExecutable has the right path to the python executable.
  • The debug configuration that will be passed to the python debug adapter (via launch request) should use conda run as the pythonPath. This can be done in LaunchConfigurationResolver . Should look some thing like this for current file config for example:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "pythonPath": "conda run" // <--- This is what you will have to add
        }

I need to confirm this from the debugger side that we will use pythonPath if it is provided.
/cc @int19h @DonJayamanne does this make sense?

[DonJayamanne]

"pythonPath": "conda run"

I wouldn't use pythonPath for this. conda run isn't an executable it's a command. We might have to escape environment names or do other stuff...
Why not be specific rather than using it for some other meaning.

Today it's meant to point to the executable.
I.e use another setting launchCommand pythonLaunchCmd

[int19h]

We deprecated "pythonPath" in 5.x - it's just "python" now, and it can be an array to allow for arguments to the interpreter.

In this case, I think it's reasonable to say that "conda" is the interpreter (and it doesn't have to be fully spelled out - indeed, we default to "python" if nothing is specified, it's just that VSC always specifies it in practice), and "run" is an argument to that interpreter. So, this should do the right thing, and take care of proper escaping etc:

"python": ["conda", "run"]

("conda run" as a single value won't work, because it will be quoted as a single command-line argument when the launcher performs run-in-terminal or spawn.)

[int19h]

Slight correction: if I'm understanding the docs right, you still need to tell Conda that you're running Python. So:

"python": ["conda", "run", ..., "--", "python"]

Where "..." is where environment name etc goes, as needed.

[DonJayamanne]

Note, there's no --

[karthiknadig]

Adding this for ease of reference:

conda run --help
usage: conda-script.py run [-h] [-n ENVIRONMENT | -p PATH] [-v] [--dev]
                           [--debug-wrapper-scripts] [--cwd CWD]
                           ...

Run an executable in a conda environment. [Experimental]

Use '--' (double dash) to separate CLI flags for 'conda run' from CLI flags sent to
the process being launched.

Example usage:

    $ conda create -y -n my-python-2-env python=2
    $ conda run -n my-python-2-env python --version

Options:

positional arguments:
  executable_call       Executable name, with additional arguments to be
                        passed to the executable on invocation.

optional arguments:
  -h, --help            Show this help message and exit.
  -v, --verbose         Use once for info, twice for debug, three times for
                        trace.
  --dev                 Sets `CONDA_EXE` to `python -m conda`, assuming the
                        CWD containsthe root of conda development sources.
                        This is mainly for useduring tests where we test new
                        conda source against old Pythonversions.
  --debug-wrapper-scripts
                        When this is set, where implemented, the shell wrapper
                        scriptswill echo to stderr a lot of debugging
                        information.
  --cwd CWD             Current working directory for command to run in.
                        Defaults to cwd

Target Environment Specification:
  -n ENVIRONMENT, --name ENVIRONMENT
                        Name of environment.
  -p PATH, --prefix PATH
                        Full path to environment location (i.e. prefix).

[int19h]

Per the docs, -- is not required, but I think it's safer to always use it. It'll also make the command line clearer in the terminal, especially once we add all the ptvsd switches in there.

[karrtikr]

I was trying out this launch config,

{
    "name": "Python: Current File",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "console": "integratedTerminal",
    "python": ["conda", "run", "-n","base", "--", "python"]
}

but run into this error.

Seems that I am not able to add "python" in the launch config.

[karrtikr]

Blocked on microsoft/ptvsd#1944

[int19h]

The error message above is because the extension itself injects "pythonPath" into the config, and ptvsd doesn't know which one it is supposed to use when it sees both.

[karrtikr]

Yes, I corrected that. It's now blocked on the ptvsd issue.

[karthiknadig]

Looking at the logs on that debugger issue, the debuggee is being launched using conda run.

Command line: [
                 "conda",
                 "run",
                 "-n",
                 "base",
                 "python",
                 "c:\\Users\\karraj\\Desktop\\vscode-python\\pythonFiles\\lib\\python\\new_ptvsd\\wheels\\ptvsd",
                 "--client",
                 "--host",
                 "127.0.0.1",
                 "--port",
                 "54984",
                 "c:\\Users\\karraj\\Desktop\\yo\\tests\\test_one.py"
             ]

That is all the we want the extension to do, i.e., send the right launch configuration. The issue has been fixed and is available in master of ptvsd.

[karrtikr]

debugger.vscode_b7868e94-7f63-49ed-a39b-85c511d76e0e.log
ptvsd.adapter-756.log
ptvsd.launcher-24048.log
ptvsd.server-22160.log

I get the same error on master of ptvsd as well. Here the logs again.

[karthiknadig]

@karrtikr It looks like it is still using 5.0.0a8 not master. You may have to delete the new_ptvsd directory. then run the following commands, from the extension project root.

pip install -t .\pythonFiles\lib\python\new_ptvsd\no_wheels git+https://github.com/microsoft/ptvsd.git
pip install -t .\pythonFiles\lib\python\new_ptvsd\wheels git+https://github.com/microsoft/ptvsd.git

The version looks like this when you install from master:

>py -3.7
Python 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ptvsd
>>> ptvsd.__version__
'5.0.0a8+1.gad36606'
>>>

[karrtikr]

Not sure what was happening. But I deleted the directory multiple times and it's now using master.

The prior error is gone now. The extension is sending the right launch configuration, but I was expecting to see conda run... when I launch the debugger. Is this expected?

Logs

ptvsd.adapter-12800.log
ptvsd.launcher-10280.log
ptvsd.server-16972.log
debugger.vscode_775fa546-2dd6-4cea-a4db-35599cf22df4.log

[karthiknadig]

Yep that is as expected. User code is actually running in using conda run if you see launcher.logs

             Command line: [
                 "conda",
                 "run",
                 "-n",
                 "base",
                 "python",
                 "c:\\Users\\karraj\\Desktop\\vscode-python\\pythonFiles\\lib\\python\\new_ptvsd\\wheels\\ptvsd",
                 "--client",
                 "--host",
                 "127.0.0.1",
                 "--port",
                 "51637",
                 "c:\\Users\\karraj\\Desktop\\wheels\\mytest.py"
             ]

[karthiknadig]

The launcher itself gets launched using the same python executable as the adapter. This is so that we don't have mismatch issues since both adapter and launcher are tightly coupled. User code will be started using the conda.

[karrtikr]

Listed the findings of the spike in #8422 (comment). Closing the spike now.