Make debugger gevent compatible

[DonJayamanne]

From @underyx on June 5, 2017 10:27

Environment data

VS Code version: 1.12.2
Python Extension version: 0.6.4
Python Version: 2.7.13
OS and version: macOS Sierra

Actual behavior

Running apps that use gevent/eventlet/greenlet/green threads/etc. with the debugger active causes the app to just freeze early on. This is the same behavior that I see when I run the same app with the PyCharm debugger with its 'Gevent compatible' switch turned off.

Expected behavior

They should work just like synchronous apps. In PyCharm, there's an option for this called 'Gevent compatible' on the debugger settings page, that sounds like a reasonable thing to have here as well.

Steps to reproduce:

1. Take any Python application.
2. pip install gevent.
3. Add from gevent import monkey; monkey.patch_all() in the app's __init__.py.
4. Try to run it with the debugger.

Settings

Your launch.json (if dealing with debugger issues):

        {
            "name": "my api",
            "type": "python",
            "request": "launch",
            "stopOnEntry": false,
            "pythonPath": "${config:python.pythonPath}",
            "program": "env/bin/gunicorn",
            "args": [
                "--config=.misc/gunicorn_config.py",
                "--reload",
                "myapi"
            ],
            "cwd": "${workspaceRoot}",
            "env": {},
            "envFile": "${workspaceRoot}/.env",
            "debugOptions": [
                "WaitOnAbnormalExit",
                "WaitOnNormalExit",
                "RedirectOutput"
            ]
        },
``

Your settings.json:   
```json
{
    "python.pythonPath": "env/bin/python"
}

Logs

Output from Python output panel

(empty)

Output from Console window (Help->Developer Tools menu)

(empty)

Edit: oh wow, 1000! 🎊

Copied from original issue: DonJayamanne/pythonVSCode#1000

[DonJayamanne]

From @m0ngr31 on June 12, 2017 15:56

+1. I thinking that is part of the problem with the issue I opened up.

[DonJayamanne]

From @underyx on June 12, 2017 16:36

Said issue: https://github.com/DonJayamanne/pythonVSCode/issues/985

[underyx]

So at the PyCon sprint, just to add some context, I've learned the following:

• This issue was created for a legacy debugger which is now replaced with pydevd on master
• pydevd already should support gevent debugging (it is the debugger used by PyCharm, as well)
• What should probably be done here is just adding a test for gevent debugging to verify that the integration works as expected

I'll try to do this now.

[underyx]

Here's a quick report on what I've been up to:

• As it turns out, reproducing gevent failures is way more difficult than I expected. I can reproduce a freeze in a complex real-life app, see the traceback of that below, however just using patched threading or time does not induce any issues. I'll try manipulating sockets next as per the traceback below.
• What's good news is, gevent support does work! It's configured with the GEVENT_SUPPORT environment variable. Adding "env": {"GEVENT_SUPPORT": "False"} to my launch configuration for the above mentioned complex real-life app got it running 🎉

gevent freeze traceback

Python 2.7.15 with gevent==1.2.2

[…]
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/ddtrace/contrib/requests/patch.py", line 74, in _traced_request_func
    resp = func(*args, **kwargs)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/sessions.py", line 499, in request
    prep.url, proxies, stream, verify, cert
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/sessions.py", line 672, in merge_environment_settings
    env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/utils.py", line 692, in get_environ_proxies
    if should_bypass_proxies(url, no_proxy=no_proxy):
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/requests/utils.py", line 676, in should_bypass_proxies
    bypass = proxy_bypass(netloc)
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1519, in proxy_bypass
    return proxy_bypass_macosx_sysconf(host)
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1479, in proxy_bypass_macosx_sysconf
    hostIP = socket.gethostbyname(hostonly)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/_socketcommon.py", line 243, in gethostbyname
    return get_hub().resolver.gethostbyname(hostname)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/resolver_thread.py", line 59, in gethostbyname
    return self.pool.apply(_socket.gethostbyname, args)
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/pool.py", line 325, in apply
    return self.spawn(func, *args, **kwds).get()
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/event.py", line 382, in get
    self._wait_core(timeout, ())
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/event.py", line 106, in _wait_core
    result = self.hub.switch()
  File "/Users/underyx/kiwi/booking/env/lib/python2.7/site-packages/gevent/hub.py", line 630, in switch
    return RawGreenlet.switch(self)
KeyboardInterrupt

[underyx]

To-do

• Add a "debugGevent": true, option to pythonExperimental launch configurations
• Add a Python Experimental: Gevent launch configuration template
• Add a unit test to ptvsd perhaps? — with the way we just inject an environment variable now, this seems unfeasible
• Create issue to document this flag (blocked now as there's no docs for the experimental debugger) — Document how to debug gevent apps #1700

A vscode-python test is not needed as it would be flaky anyway.