[Investigate] Remote attach without launching adapter subprocess

[RockShea]

Environment data

• debugpy version: 1.2.1
• OS and version: Windows 10 Pro
• Python version (& distribution if applicable, e.g. Anaconda): Python27 embedded into 3DsMax 2018
• Using VS Code or Visual Studio: VSCode

Actual behavior

Remote attach times out from 3DsMax 2018 - which runs embedded Python27.
Here is the stacktrace, from 3DsMax 2018:

Traceback (most recent call last):
  File "C:\p4\Tools\Art\Python\!Shared\Development\R_Shared\R_Debugger.py", line 27, in start
    debugpy.listen(inAddress)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\__init__.py", line 113, in listen
    return api.listen(address)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 143, in debug
    log.reraise_exception("{0}() failed:", func.__name__, level="info")
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 141, in debug
    return func(address, settrace_kwargs, **kwargs)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 232, in listen
    raise RuntimeError("timed out waiting for adapter to connect")
RuntimeError: timed out waiting for adapter to connect

And the error message in VSCode:

Expected behavior

Remote attach to correctly connect, as it does with 3DsMax 2021 - which runs embedded Python 37.

Steps to reproduce:

VSCode config settings which work in 3DsMax 2021 - Python37, but not 3DsMax 2018 - Python27:

"configurations": [
    {
        "name": "Python: Remote Attach",
        "type": "python",
        "request": "attach",
        "port": 3000,
        "host": "0.0.0.0",
        "pathMappings": [
            {
                "localRoot": "C:/p4/Tools/Art/Python",
                "remoteRoot": "C:/p4/Tools/Art/Python"
            }
        ]
    },
}

Code to run within 3DsMax, which works in 2021 - Python37 and not 2018 - Python27:

import debugpy
import os
import traceback


def start(inAddress, waitForAttach=False):

    try:

        debugpy.configure(python=<externalPythonPath>)
        debugpy.listen(inAddress)
        if waitForAttach:

            print("Waiting for debugger to attach...")
            debugpy.wait_for_client()

    except Exception:

        traceback.print_exc()


def stop():

    return


if __name__ in ("__main__", "__builtin__"):

    start(("0.0.0.0", 3000), waitForAttach=False)

[int19h]

0.0.0.0 means "listen on all known interfaces on the specified port". But when connecting, you need to specify the specific address to connect to - I'm not even sure what 0.0.0.0 is supposed to mean when it's used for a client socket, but it can't be right.

If you're connecting to 3DS Max instance that's running on the same machine as VSCode, you should be using 127.0.0.1 on both sides. Also note that listening on 0.0.0.0 means that anybody who can connect to your machine over the network to the port it's listening on, can debug it - unless you're on a private secure network, you really don't want to do that, because the debugger allows execution of arbitrary code (via e.g. Watch).

[RockShea]

Thanks for the reply @int19h

VSCode and the instance of Max are on the same machine - thankfully we are on a private secure network. Thanks for the information though, that is good to know.
I found the suggestion to use 0.0.0.0 on other threads with connection issues.
Unfortunately, if I swap the address to localHost or 127.0.0.1 the problem persists.
Would the use of 0.0.0.0 not illustrate that there is a problem on Py27 as it still not connecting even if it is listening on all ports?

Stacktrace from within Max2018:

Traceback (most recent call last):
  File "C:\p4\Tools\Art\Python\!Shared\Development\R_Shared\R_Debugger.py", line 28, in start
    debugpy.listen(inAddress)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\__init__.py", line 113, in listen
    return api.listen(address)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 143, in debug
    log.reraise_exception("{0}() failed:", func.__name__, level="info")
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 141, in debug
    return func(address, settrace_kwargs, **kwargs)
  File "C:\p4\Tools\Python\environment\2.7\Lib\site-packages\debugpy\server\api.py", line 232, in listen
    raise RuntimeError("timed out waiting for adapter to connect")
RuntimeError: timed out waiting for adapter to connect

Error message in VSCode:

Supporting Py27 is a pain, unfortunately we are stuck with it due to the software we use having it as it's embedded version.

[int19h]

Listening on 0.0.0.0 is fine (security aside), it's connecting that has unclear semantics. It should be okay to listen on 0.0.0.0, but connect to 127.0.0.1. In your case, since it's all local, 127.0.0.1 is the right choice for both ends. It is also the default if you specify the port, but omit the host.

[int19h]

Can you double-check that the version of Python interpreter that you're passing to debugpy.configure() is valid? Note that it doesn't have to match the version of Python used by the app - any version supported by debugpy should do.

If that's the case, then we'll need logs to figure out what's going on here. You can get those by doing:

debugpy.log_to(r'C:\Temp')  # must be a directory

The resulting log files will be named debugpy*.log.

[RockShea]

The standalone interpreter is a studio distribution of Python37, which has debugpy installed on it.
I use the same version for Max2018 and 2021.
I will setup the log file and see what it reports.

Thanks again for the help.

[RockShea]

Here is the log file generated from within 3DsMax 2018:
debugpy.server-10580.log
Am I able to generate any log from within VSCode?

[int19h]

The most interesting log in this case is the one that comes from the adapter, but it looks like it didn't even get to the point where it started logging. But it might be possible to diagnose this manually by replicating what the server is trying to do; specifically:

I+00000.006: debugpy.listen() spawning adapter: [
                 "C:\\p4\\Tools\\Python\\environment\\3.7\\Scripts\\python.exe", 
                 "C:\\p4\\Tools\\Python\\environment\\2.7\\Lib\\site-packages\\debugpy\\adapter", 
                 "--for-server", 
                 "61270", 
                 "--host", 
                 "localHost", 
                 "--port", 
                 "3000", 
                 "--server-access-token", 
                 "4d0918d8f29ed47123d016bc38618bea1a86cb0dc6b335a23630c2eca6511780", 
                 "--log-dir", 
                 "C:\\p4\\Tools\\Art\\Data\\Debugger"
             ]

Can you try running this same thing manually in the terminal, and see what errors (if any) it produces?

[RockShea]

So if I run debugpy in our Python27 venv it seems to work correctly:

C:\P4\Tools\Python\environment\2.7\Scripts\activate
python -m debugpy --listen localhost:3000 --log-to -debug folder path- -site setup .py file-

It seems to work without issue and generates two log files:
debugpy.adapter-4188.log
debugpy.server-17940.log

[int19h]

Interesting. Sounds like Python version mismatch is a problem in this scenario. This is unexpected, and I'll take a look at what breaks there, but for now, I guess this is the workaround. Thank you for your help in diagnosing this!

[RockShea]

Ah - that is good that you have spotted an issue!
Thanks for the quick support :)
Just to confirm, the workaround you suggest is using the command line to debug?

[djay]

I'm having similar problems with Kodi on macos. Why does debugpy have to spawn? it would be very nice if it supported a non-spawn mode to make life simpler. I just want to set a set_trace().

Kodi on macos doesn't have a command line interpreter as far as I can tell. It's embedded.

$ /Applications/Kodi.app/Contents/MacOS/Kodi 
2021-02-03 02:54:06.175 XBMCHelper[8457:4490273] ParseOptions - force VerboseMode on
2021-02-03 02:54:06.175 XBMCHelper[8457:4490273] ParseOptions - ReadExternal on
2021-02-03 02:54:06.176 XBMCHelper[8457:4490273] ParseOptions - force VerboseMode on
2021-02-03 02:54:06.176 XBMCHelper[8457:4490273] ParseOptions - server port 9777
2021-02-03 02:54:06.176 XBMCHelper[8457:4490273] ParseOptions - AppPath /Applications/Kodi.app/Contents/MacOS/Kodi
2021-02-03 02:54:06.176 XBMCHelper[8457:4490273] ParseOptions - AppHome /Applications/Kodi.app/Contents/Resources/Kodi
2021-02-03 02:54:06.176 XBMCHelper[8457:4490273] XBMCHelper 0.8 starting up...
2021-02-03 02:54:06.234 XBMCHelper[8457:4490273] [ERROR] Driver has not found any remotes it could use. Will use remotes as they become available.
Traceback (most recent call last):
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/runpy.py", line 174, in _run_module_as_main
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/runpy.py", line 72, in _run_code
  File "/Users/dylanjay/Projects/kodi/debugpy/src/debugpy/adapter/__main__.py", line 7, in <module>
    import argparse
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/argparse.py", line 85, in <module>
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/collections.py", line 20, in <module>
ImportError: No module named _collections

Note /Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/ doesn't exist on my machine.

Config is

        import collections # Just to show it is there
        import debugpy
        debugpy.configure(
            subProcess=True,
            python="/usr/local/bin/python",
        )
        debugpy.listen(5678)

$ cat ~/logs/debugpy.server-8450.log 
I+00000.023: Darwin-19.6.0-x86_64-i386-64bit x86_64
             CPython 2.7.15 (64-bit)
             debugpy 1.2.1+9.gcbcfe22

I+00005.960: Initial environment:
             
             System paths:
                 sys.prefix: /Applications/Kodi.app/Contents/Libraries
                 sys.base_prefix: <missing>
                 sys.real_prefix: <missing>
                 site.getsitepackages(): /Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages
                                         /Applications/Kodi.app/Contents/Libraries/lib/site-python
                 site.getusersitepackages(): /Users/dylanjay/.local/lib/python2.7/site-packages
                 sys.path (site-packages): /Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages
                                           /Users/dylanjay/.local/lib/python2.7/site-packages
                 sysconfig.get_path('stdlib'): /Applications/Kodi.app/Contents/Libraries/lib/python2.7
                 sysconfig.get_path('platstdlib'): /Applications/Kodi.app/Contents/Libraries/lib/python2.7
                 sysconfig.get_path('purelib'): /Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages
                 sysconfig.get_path('platlib'): /Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages
                 sysconfig.get_path('include'): /Applications/Kodi.app/Contents/Libraries/include/python2.7
                 sysconfig.get_path('scripts'): /Applications/Kodi.app/Contents/Libraries/bin
                 sysconfig.get_path('data'): /Applications/Kodi.app/Contents/Libraries
                 os.__file__: /Applications/Kodi.app/Contents/Libraries/lib/python2.7/os.pyo
                 threading.__file__: /Applications/Kodi.app/Contents/Libraries/lib/python2.7/threading.pyo

D+00005.974: configure(None, {'python': '/usr/local/bin/python', 'subProcess': True})

D+00006.003: listen((u'127.0.0.1', 5678), **{})

I+00006.005: Initial debug configuration: {
                 "python": "/usr/local/bin/python", 
                 "qt": "none", 
                 "subProcess": true
             }

I+00006.074: Waiting for adapter endpoints on 127.0.0.1:64508...

I+00006.076: debugpy.listen() spawning adapter: [
                 "/usr/local/bin/python", 
                 "/Users/dylanjay/Library/Application Support/Kodi/addons/script.module.ptvsd/lib/debugpy/adapter", 
                 "--for-server", 
                 "64508", 
                 "--host", 
                 "127.0.0.1", 
                 "--port", 
                 "5678", 
                 "--server-access-token", 
                 "99af9c3f27dfdd19553fa3e242a12400ff985bc45f10813c3591c59ab2d866ee", 
                 "--log-dir", 
                 "/Users/dylanjay/logs"
             ]

I+00013.148: Timed out waiting for adapter to connect:
...

$ cat ~/logs/debugpy.pydevd.8450.log 
os.path.exists('/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/posixpath.py') returned False.
It seems the debugger cannot resolve /Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/posixpath.py
This may make the debugger miss breakpoints in the standard library.
Related bug: https://bugs.python.org/issue1180193
Using GEVENT_SUPPORT: False
pydevd __file__: /Users/dylanjay/Library/Application Support/Kodi/addons/script.module.ptvsd/lib/debugpy/_vendored/pydevd/pydevd.pyo

And no adapter log generated

If I run the adapter command line manually it seems to work fine

$ /usr/local/bin/python /Users/dylanjay/Library/Application\ Support/Kodi/addons/script.module.ptvsd/lib/debugpy/adapter
Content-Length: 148

{"seq": 1, "type": "event", "event": "output", "body": {"category": "telemetry", "output": "ptvsd", "data": {"packageVersion": "1.2.1+9.gcbcfe22"}}}Content-Length: 150

{"seq": 2, "type": "event", "event": "output", "body": {"category": "telemetry", "output": "debugpy", "data": {"packageVersion": "1.2.1+9.gcbcfe22"}}}

[RockShea]

I was able to get it to work by having the external Python interpreter match the version of the embedded version:

def start(inAddress, waitForAttach=False, log=False):
    """Start the debugger in the current python interpreter.
    This allows connection from a remote debugger, such as Sublime or VSCode.

    Args:
        inAddress (int): Port or address: port to connect to.
            Either ("0.0.0.0", 3000) or 3000
        waitForAttach (bool, optional): If True, will block program execution
            until the debugger is attached.
            Defaults to False.

    Returns:
        bool: True on success, False on failure.
    """

    try:

        if log:

            debugpy.log_to(os.path.join(RockToolsPathData.toolsData, "Debugger"))

        pythonExecutable = (
            os.path.join(RockToolsPathData.toolsPython, "environment\\3.7\\Scripts\\python.exe")
            if __py3__
            else os.path.join(RockToolsPathData.toolsPython, "environment\\2.7\\Scripts\\python.exe")
        )
        debugpy.configure(python=pythonExecutable)
        debugpy.listen(inAddress)
        if waitForAttach:

            print("Waiting for debugger to attach...")
            debugpy.wait_for_client()

    except Exception:

        traceback.print_exc()

[int19h]

@djay It has to spawn because some things that debugger needs to do, cannot be done reasonably well from within the same process that is being debugged. The older versions (pre-ptvsd 5.x) didn't have a multiproc architecture, and there was a slew of bugs and limitations stemming from that.

And, unfortunately, it's a fairly fundamental architectural decision, so we can't easily support a "no-spawn mode" or something like that for debugpy. Although it might be possible to use pydevd as a standalone DAP server at this point - it might be fully compatible with VSCode by now even without debugpy in the picture. @fabioz, can you advise?

Also note that you don't need the interpreter offered by the app doing the embedding. You just need some Python interpreter, somewhere. As @RockShea noted above, at the moment, it seems to only work correctly if the language version matches (although that wasn't intended). So, you just need to install a matching version of Python, and then use debugpy.configure() to point at that.

[djay]

@int19h I'm not sure the language version is the problem in this case. how close do they have to be?
The system python I am pointing to is '2.7.17 (default, Oct 24 2019, 12:57:47) \n[GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.33.8)]'.
The logs above seem to indicate the embedded kodi interpreter is Darwin-19.6.0-x86_64-i386-64bit x86_64 CPython 2.7.15 (64-bit).
I can't see that changing my version to 2.7.15 instead of 2.7.15 is likely to solve the problem of ImportError: No module named _collections is it?
It seems more likely that something about how it is setting up the python paths to start the process is more likely to blame?
As you can see from me starting the server using the system python (without trying to adjust python paths etc) it seems to start fine.

[int19h]

@djay When the adapter is spawned, it does adjust PYTHONPATH so that it can import other modules from itself. However, this should not be affecting anything from stdlib:

debugpy/src/debugpy/adapter/__main__.py

Lines 191 to 217 in cbcfe22

	if __name__ == "__main__":
	# debugpy can also be invoked directly rather than via -m. In this case, the first
	# entry on sys.path is the one added automatically by Python for the directory
	# containing this file. This means that import debugpy will not work, since we need
	# the parent directory of debugpy/ to be in sys.path, rather than debugpy/adapter/.
	#
	# The other issue is that many other absolute imports will break, because they
	# will be resolved relative to debugpy/adapter/ - e.g. `import state` will then try
	# to import debugpy/adapter/state.py.
	#
	# To fix both, we need to replace the automatically added entry such that it points
	# at parent directory of debugpy/ instead of debugpy/adapter, import debugpy with that
	# in sys.path, and then remove the first entry entry altogether, so that it doesn't
	# affect any further imports we might do. For example, suppose the user did:
	#
	# python /foo/bar/debugpy/adapter ...
	#
	# At the beginning of this script, sys.path will contain "/foo/bar/debugpy/adapter"
	# as the first entry. What we want is to replace it with "/foo/bar', then import
	# debugpy with that in effect, and then remove the replaced entry before any more
	# code runs. The imported debugpy module will remain in sys.modules, and thus all
	# future imports of it or its submodules will resolve accordingly.
	if "debugpy" not in sys.modules:
	# Do not use dirname() to walk up - this can be a relative path, e.g. ".".
	sys.path[0] = sys.path[0] + "/../../"
	__import__("debugpy")
	del sys.path[0]

What's particularly interesting about your error is that it complains about _collections rather than collections. The latter is an actual Python file in the standard library, but the former is a built-in Python module - i.e. it doesn't have any corresponding file, not even .so; it's an integral part of the Python interpreter. So it should always be importable, regardless of what's going on with PYTHONPATH, because there's nothing to locate. That it fails to import indicates some very fundamental issue with this Python install.

The other thing is, the error seems to be happening very early in the adapter process - before it gets a chance to do anything, really; this is literally the first executable line of its __main__.py. So whatever goes wrong there, has gone wrong before.

One thing that might help diagnose it is manually logging some information early on in that __main__.py, before it imports anything other than sys. Specifically, sys.executable, sys.path, and sys.argv - can you give that a try?

[djay]

/usr/local/Cellar/python@2/2.7.17/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ['/Users/dylanjay/Projects/kodi/repo-scripts/script.module.ptvsd/lib/debugpy/adapter', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages/setuptools-18.4-py2.7.egg', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages/Pillow-3.0.0-py2.7-macosx-10.14-x86_64.egg', '/Applications/Kodi.app/Contents/Libraries', '/Applications/Kodi.app/Contents/Libraries/lib/python27.zip', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/plat-darwin', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/plat-mac', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/plat-mac/lib-scriptpackages', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/lib-tk', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/lib-old', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/lib-dynload', '/Users/dylanjay/.local/lib/python2.7/site-packages', '/Applications/Kodi.app/Contents/Libraries/lib/python2.7/site-packages'] ['/Users/dylanjay/Projects/kodi/repo-scripts/script.module.ptvsd/lib/debugpy/adapter', '--for-server', '61775', '--host', '127.0.0.1', '--port', '5678', '--server-access-token', '9357150e03d28595d6ad142cc01a236131092db6a8d28223bca3d2ca514dd5d4', '--log-dir', '/Users/dylanjay/logs']
Traceback (most recent call last):
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/runpy.py", line 174, in _run_module_as_main
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/runpy.py", line 72, in _run_code
  File "/Users/dylanjay/Projects/kodi/debugpy/src/debugpy/adapter/__main__.py", line 10, in <module>
    import argparse
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/argparse.py", line 85, in <module>
  File "/Users/Shared/jenkins/workspace/OSX-64_Leia/tools/depends/xbmc-depends/macosx10.13_x86_64-target-release/lib/python2.7/collections.py", line 20, in <module>
ImportError: No module named _collections

It's using the homebrew python I specified but with the paths of the embedded python. But I guess because kodis python is embedded it can't get hold of those internal modules like _collections. Shouldn't the paths also include paths from the python I specify?

[djay]

If I manipulate the path to put in some of the paths from my system python then it works. The path it needed was '/usr/local/Cellar/python@2/2.7.17/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload'
Except it can't match up the file locations but thats probably because I'm using a symbolic link to put the code into my kodi addon directory.
And also the exception paths aren't being recognised correctly because they have a space in the dir name.

[fabioz]

And, unfortunately, it's a fairly fundamental architectural decision, so we can't easily support a "no-spawn mode" or something like that for debugpy. Although it might be possible to use pydevd as a standalone DAP server at this point - it might be fully compatible with VSCode by now even without debugpy in the picture. @fabioz, can you advise?

@int19h (it seems I missed that message, sorry for the delay)... the auto-attach to subprocess wouldn't work, but apart from that it should be possible to get that working (although I haven't really tested connecting directly to VSCode, it's something that I think would be reasonably straightforward, especially when connecting to applications that have Python builtin as spawning a Python subprocess isn't always easy in such cases as mismatches aren't that uncommon due to misconfigurations -- still, it needs some testing on my side to make sure it actually works as I expect it to).

[fabioz]

I've done some experiments here and with a few changes it is possible to debug using the process itself without spawning a separate debug adapter process (so the connection is managed directly by pydevd).

For the API I was thinking about:

def listen(__endpoint: Endpoint | int, *, in_process_debug_adapter: bool=False) -> Endpoint:

Other ideas on a better API/naming are welcome.

I think it should be even possible to support launching new subprocesses by sending the debugpyAttach message to the client (I'm investigating on that -- it needs some additional work to pass the needed config to subprocesses).

Thoughts @rchiodo @int19h @karthiknadig?

[int19h]

I wonder if this would be better exposed as a public pydevd API? debugpy isn't really doing anything useful in this scenario.

For the web, we'll need some custom wrapper anyway to set up the environment and handle DAP "launch", so it would also transparently call this API then.

[fabioz]

I wonder if this would be better exposed as a public pydevd API? debugpy isn't really doing anything useful in this scenario.

I think it boils down to whether we'd like to keep debugpy as a public API here.

i.e.: it may not make much difference in the web case, but on cases where the user has issues on getting a python executable just for the debug adapter process it may be a better API since users are expecting to interact with debugpy.

-- also, a plain import pydevd would fail to find the vendored pydevd unless the user did something as import debugpy._vendored.force_pydevd first as just a plain import debugpy doesn't really import pydevd at that point.

[fabioz]

The basic listen mode without an additional adapter is working (in #1050) and I was checking how to implement the auto attach to subprocesses in this case.

After playing with it a bit, I must say that the only I can see it working on all cases would be spawning a secondary process to handle it (otherwise cases such as Linux where the user uses some API which replaces the current process don't work because we need to ask for the connection to be done in the original process, but the child process replaces the original process prior to that) -- but then, this has no advantage over just using debugpy as the adapter, so, it's probably not worth implementing.

Still, this is only a problem because we're doing a listen in the debugger side. If the connection was done the other way around (i.e.: if VSCode opens a server socket and starts debugging any incoming connection) then this use-case would work out of the box (so, we'd add the in_process_debug_adapter flag to connect to and not just listen)...

So, my question is how to proceed given that. The options I can see are:

1. Start the server socket in the VSCode side and have it automatically debug any new connection done to it (so, we wouldn't do a listen in the server, we'd do a connect to the port that is waiting for connections in the VSCode side) -- as a note, this is how the debugger works in Eclipse to and in general I like that approach better (because the user can just start it once and then just use debugpy.connect without having to redo the attach manually each time).
2. Support the use case partially (so, we'd support subprocesses but not the use case where the child process replaces the current process -- the approach would be creating a new thread which creates 2 server sockets, one for VSCode to attach to and one for the subprocess and then forward messages back and forth, close to what debugpy does but in the existing process).
3. Don't support auto-attach of subprocesses in this use-case.

What do you think @rchiodo @int19h @karthiknadig?

[int19h]

For this scenario as well as the web one, subprocesses are not really in the picture, so I think we should proceed with the last approach.

[fabioz]

Seems good to me... I just need to do a couple of fixes in the PR then (don't monkey-patch subprocess in this scenario and update some docs there -- I'll probably merge on Thursday).

[fabioz]

This is fixed (in 30a96bf).