Add Python remote debugger for Visual Studio Code #76

thinkh · 2019-08-20T16:17:38Z

This PR adds the Python Tools for the Visual Studio Debugger (short: ptvsd) to phovea_server and let's you debug Python code within Visual Studio Code.

Related PR: phovea/generator-phovea#166

How to start the debugger

(Pull the recent changes for phovea_server)
yo phovea:update to update the requirements_dev.txt and docker-compose-debug.yml
docker-compose build api

Activate the debugger by adding a flag to the <workspace>/config.json

{
  "phovea_server": {
     "ptvsd_debugger": true
  }
}

Add the Python remote debugger to Visual Studio Code using the cogs icon or copy the .vscode/launch.json from to <workspace>/.vscode/launch.json (see commit c4450ca)
Start the docker container in debug mode
```
./docker-compose-debug up
```
Heads up: The usual command docker-compose up will not work here!
The api container will wait for a debugger to attach
Set a breakpoint in a python file by clicking left to the line number
Start debugging in Visual Studio Code

Stop debugging

Disconnect the debugger in Visual Studio Code.
Stop the container in your terminal with Ctrl + C.

Links

Useful links when implementing this feature:

https://vinta.ws/code/remotely-debug-a-python-app-inside-a-docker-container-in-visual-studio-code.html

Adds the Python debugger for Visual Studio Code. Requires a phovea_server version with merged PR phovea/phovea_server#76.

If this line is active logs from the launch.py are not shown after loading this file. Disabling this line solves the problem and all logs are shown.

thinkh · 2019-08-22T16:57:16Z

There is still a problem when using the debugger and the reloader at the same time.

Usually the server is started twice, due to the werkzeug reloader:

Starting ordino_develop_api_1 ... done                                                                                                                                                                                                                                                    Attaching to ordino_develop_api_1

api_1          | configuration file: /phovea/config.json

api_1          | 16:45:19 INFO phovea_server.launch: enabling development mode      
api_1          | 16:45:19 INFO phovea_server.launch: Tip: You can enable the remote debugger `ptvsd` for Visual Studio Code by adding configuration `"ptvsd_debugger": true` to `phovea_server` section in the config.json of your workspace. 
api_1          | 16:45:19 INFO phovea_server._plugin_parser: looking for development neighbors 
api_1          | 16:45:19 INFO phovea_server._plugin_parser: discovered 17 plugins: [u'ordino', u'ordino_public', u'phovea_clue', u'phovea_core', u'phovea_data_mongo', u'phovea_data_redis', u'phovea_security_flask', u'phovea_security_store_generated', u'phovea_server', u'phovea_ui', u'phovea_workspace', u'phovea_workspace', u'tdp_core', u'tdp_gene', u'tdp_matomo', u'tdp_publicdb', u'tdp_uploaded_data']

api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_clue     
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_data_mongo          
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_data_redis          
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_security_flask      
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_security_store_generated       
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: phovea_server   
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: tdp_core        
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: tdp_matomo

api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: tdp_publicdb    
api_1          | 16:45:19 INFO phovea_server._plugin_parser: importing module: tdp_uploaded_data          
api_1          | 16:45:19 INFO phovea_server.launch: add command: api

api_1          | 16:45:19 INFO phovea_server.launch: set default command: api       
api_1          | 16:45:19 INFO phovea_server.launch: loading and initializing the command: api 
api_1          | 16:45:19 INFO phovea_server.plugin: importing phovea_server.server 
api_1          | 16:45:20 INFO phovea_server.server: init application: <Flask 'phovea_server.mainapp'>

api_1          | 16:45:20 INFO phovea_security_flask.flask_login_impl: init flask login        
api_1          | 16:45:20 INFO phovea_security_flask.flask_login_impl: init flask app

api_1          | 16:45:20 INFO phovea_security_flask.flask_login_impl: add flask routes

api_1          | 16:45:20 INFO phovea_server.server: add application: caleydo-clue-screenshot at namespace: /api/clue
api_1          | 16:45:20 INFO phovea_server.server: add application: caleydo-dataset at namespace: /api/dataset     
api_1          | 16:45:20 INFO phovea_server.server: add application: caleydo-idtype at namespace: /api/idtype       
api_1          | 16:45:20 INFO phovea_server.server: add application: tdp_core at namespace: /api/tdp/proxy          
api_1          | 16:45:20 INFO phovea_server.server: add application: db_connector at namespace: /api/tdp/db         
api_1          | 16:45:20 INFO phovea_server.server: add application: tdp_storage at namespace: /api/tdp/storage     
api_1          | 16:45:20 INFO phovea_server.server: add application: tdp_swagger at namespace: /api/tdp/ui          
api_1          | 16:45:20 INFO phovea_server.server: add application: tdp_config at namespace: /api/tdp/config       
api_1          | 16:45:20 INFO phovea_server.server: add application: tdp_xlsx2json at namespace: /api/tdp/xlsx      
api_1          | 16:45:20 INFO phovea_server.server: prepare server that will listen on 0.0.0.0:80    
  
api_1          | 16:45:20 INFO werkzeug:  * Restarting with stat reloader



api_1          | configuration file: /phovea/config.json

api_1          | 16:45:20 INFO phovea_server.launch: enabling development mode      
api_1          | 16:45:20 INFO phovea_server.launch: Tip: You can enable the remote debugger `ptvsd` for Visual Studio Code by adding configuration `"ptvsd_debugger": true` to `phovea_server` section in the config.json of your workspace. 
api_1          | 16:45:20 INFO phovea_server._plugin_parser: looking for development neighbors 
api_1          | 16:45:20 INFO phovea_server._plugin_parser: discovered 17 plugins: [u'ordino', u'ordino_public', u'phovea_clue', u'phovea_core', u'phovea_data_mongo', u'phovea_data_redis', u'phovea_security_flask', u'phovea_security_store_generated', u'phovea_server', u'phovea_ui', u'phovea_workspace', u'phovea_workspace', u'tdp_core', u'tdp_gene', u'tdp_matomo', u'tdp_publicdb', u'tdp_uploaded_data']

api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_clue     
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_data_mongo          
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_data_redis          
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_security_flask      
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_security_store_generated

api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: phovea_server   
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: tdp_core

api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: tdp_matomo      
api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: tdp_publicdb

api_1          | 16:45:20 INFO phovea_server._plugin_parser: importing module: tdp_uploaded_data

api_1          | 16:45:20 INFO phovea_server.launch: add command: api

api_1          | 16:45:20 INFO phovea_server.launch: set default command: api

api_1          | 16:45:20 INFO phovea_server.launch: loading and initializing the command: api 
api_1          | 16:45:20 INFO phovea_server.plugin: importing phovea_server.server 
api_1          | 16:45:21 INFO phovea_server.server: init application: <Flask 'phovea_server.mainapp'>

api_1          | 16:45:21 INFO phovea_security_flask.flask_login_impl: init flask login

api_1          | 16:45:21 INFO phovea_security_flask.flask_login_impl: init flask app

api_1          | 16:45:21 INFO phovea_security_flask.flask_login_impl: add flask routes        
api_1          | 16:45:21 INFO phovea_server.server: add application: caleydo-clue-screenshot at namespace: /api/clue
api_1          | 16:45:21 INFO phovea_server.server: add application: caleydo-dataset at namespace: /api/dataset     
api_1          | 16:45:21 INFO phovea_server.server: add application: caleydo-idtype at namespace: /api/idtype       
api_1          | 16:45:21 INFO phovea_server.server: add application: tdp_core at namespace: /api/tdp/proxy          
api_1          | 16:45:21 INFO phovea_server.server: add application: db_connector at namespace: /api/tdp/db         
api_1          | 16:45:21 INFO phovea_server.server: add application: tdp_storage at namespace: /api/tdp/storage     
api_1          | 16:45:21 INFO phovea_server.server: add application: tdp_swagger at namespace: /api/tdp/ui          
api_1          | 16:45:21 INFO phovea_server.server: add application: tdp_config at namespace: /api/tdp/config       
api_1          | 16:45:21 INFO phovea_server.server: add application: tdp_xlsx2json at namespace: /api/tdp/xlsx      
api_1          | 16:45:21 INFO phovea_server.server: prepare server that will listen on 0.0.0.0:80

When activating the debugger the server exists when starting the second time:


api_1          | configuration file: /phovea/config.json

api_1          | 16:51:14 INFO phovea_server.launch: enabling development mode      
api_1          | 16:51:14 INFO phovea_server.launch: Debugger is started

api_1          | 16:51:14 INFO phovea_server.launch: You can now start the debugger in Visual Studio Code 
api_1          | 16:51:14 INFO phovea_server.launch: Waiting for a debugger to attach ...      
api_1          | 16:51:16 INFO phovea_server.launch: Debugger successfully attached

api_1          | 16:51:16 INFO phovea_server._plugin_parser: looking for development neighbors

api_1          | 16:51:16 INFO phovea_server._plugin_parser: discovered 17 plugins: [u'ordino', u'ordino_public', u'phovea_clue', u'phovea_core', u'phovea_data_mongo', u'phovea_data_redis', u'phovea_security_flask', u'phovea_security_store_generated', u'phovea_server', u'phovea_ui', u'phovea_workspace', u'phovea_workspace', u'tdp_core', u'tdp_gene', u'tdp_matomo', u'tdp_publicdb', u'tdp_uploaded_data']

api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_clue     
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_data_mongo          
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_data_redis          
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_security_flask      
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_security_store_generated       
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: phovea_server   
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: tdp_core        
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: tdp_matomo      
api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: tdp_publicdb

api_1          | 16:51:16 INFO phovea_server._plugin_parser: importing module: tdp_uploaded_data

api_1          | 16:51:16 INFO phovea_server.launch: add command: api

api_1          | 16:51:16 INFO phovea_server.launch: set default command: api       
api_1          | 16:51:16 INFO phovea_server.launch: loading and initializing the command: api 
api_1          | 16:51:16 INFO phovea_server.plugin: importing phovea_server.server 
api_1          | 16:51:16 INFO phovea_server.launch: add command instance to parser

api_1          | 16:51:20 INFO phovea_server.server: init application: <Flask 'phovea_server.mainapp'>

api_1          | 16:51:20 INFO phovea_server.plugin: read currently executed command from config: api     
api_1          | 16:51:20 INFO phovea_server.plugin: creating singleton mappingmanager phovea_server.mapper          
api_1          | 16:51:20 INFO phovea_server.plugin: creating singleton registry server        
api_1          | 16:51:20 INFO phovea_server.plugin: creating singleton idmanager phovea_data_redis.assigner         
api_1          | 16:51:20 INFO phovea_server.plugin: creating singleton security_manager phovea_security_flask.flask_login_impl 
api_1          | 16:51:20 INFO phovea_server.plugin: creating singleton cachemanager phovea_data_redis.cache         
api_1          | 16:51:20 INFO phovea_server.plugin: importing phovea_security_flask.flask_login_impl     
api_1          | 16:51:20 INFO phovea_security_flask.flask_login_impl: init flask login

api_1          | 16:51:20 INFO phovea_server.plugin: importing phovea_security_store_generated.store      
api_1          | 16:51:20 INFO phovea_security_flask.flask_login_impl: init flask app

api_1          | 16:51:20 INFO phovea_security_flask.flask_login_impl: add flask routes        
api_1          | 16:51:20 INFO phovea_server.server: add application: caleydo-clue-screenshot at namespace: /api/clue
api_1          | 16:51:20 INFO phovea_server.server: add application: caleydo-dataset at namespace: /api/dataset     
api_1          | 16:51:20 INFO phovea_server.server: add application: caleydo-idtype at namespace: /api/idtype       
api_1          | 16:51:20 INFO phovea_server.server: add application: tdp_core at namespace: /api/tdp/proxy          
api_1          | 16:51:20 INFO phovea_server.server: add application: db_connector at namespace: /api/tdp/db         
api_1          | 16:51:20 INFO phovea_server.server: add application: tdp_storage at namespace: /api/tdp/storage     
api_1          | 16:51:20 INFO phovea_server.server: add application: tdp_swagger at namespace: /api/tdp/ui          
api_1          | 16:51:20 INFO phovea_server.server: add application: tdp_config at namespace: /api/tdp/config       
api_1          | 16:51:20 INFO phovea_server.server: add application: tdp_xlsx2json at namespace: /api/tdp/xlsx

api_1          | 16:51:20 INFO phovea_server.server: prepare server that will listen on 0.0.0.0:80        
api_1          | 16:51:20 INFO phovea_server.launch: start application using reloader...       
api_1          | 16:51:20 INFO werkzeug:  * Restarting with stat reloader

api_1          | Traceback (most recent call last):
api_1          |  
api_1          | Exception gevent.hub.LoopExit: LoopExit('This operation would block forever', <Hub at 0x7f46cec90eb0 epoll default pending=0 ref=0 fileno=9>) in <module 'threading' from '/usr/local/lib/python2.7/threading.pyc'> ignored
ordino_develop_api_1 exited with code 1

Error:

api_1          | 16:51:20 INFO werkzeug:  * Restarting with stat reloader

api_1          | Traceback (most recent call last):
api_1          |  
api_1          | Exception gevent.hub.LoopExit: LoopExit('This operation would block forever', <Hub at 0x7f46cec90eb0 epoll default pending=0 ref=0 fileno=9>) in <module 'threading' from '/usr/local/lib/python2.7/threading.pyc'> ignored
ordino_develop_api_1 exited with code 1

Following this error it must be the combination of ptvsd debugger, gevent, and werkzeug reloader. So far I could not find any solution for this problem. The current workaround is to disable the reloader and start the main() directly:

  if args.use_reloader:
    _log.info('start application using reloader...')
    # from werkzeug._reloader import run_with_reloader
    # run_with_reloader(main, extra_files=_config_files())
    main()
  else:
    _log.info('start application...')
    main()

Note that the server will not restart once a file has been changed.

Removed because the debugger does not seem to work with our server startup process. For further information see #76.

thinkh added 3 commits August 20, 2019 17:47

Add ptvsd library to requirements_dev.txt

bddcd0e

Add attach debugger function to launch.py

72a6d07

Fix logging in launch.py

b41c5f9

thinkh requested a review from a user August 20, 2019 16:17

thinkh added the Type: Enhancement label Aug 20, 2019

thinkh assigned ghost Aug 20, 2019

Remove .travis.yml

9a1c911

ghost requested a review from puehringer August 21, 2019 05:40

thinkh added 2 commits August 22, 2019 11:28

Add ptvsd port to docker-compose-debug.partial.yml

b88bbc7

Add example of the .vscode/launch.json

c4450ca

thinkh mentioned this pull request Aug 22, 2019

Add Python debugger for Visual Studio Code to workspace phovea/generator-phovea#166

Closed

thinkh added a commit to phovea/generator-phovea that referenced this pull request Aug 22, 2019

Create .vscode/launch.json

94990cf

Adds the Python debugger for Visual Studio Code. Requires a phovea_server version with merged PR phovea/phovea_server#76.

thinkh added 2 commits August 22, 2019 18:46

Add documentation to launch process

6fb0326

Disable setting the logging config in server.py

4e3eb22

If this line is active logs from the launch.py are not shown after loading this file. Disabling this line solves the problem and all logs are shown.

Fix flake8

0948e9e

thinkh removed request for a user and puehringer October 2, 2019 12:00

thinkh unassigned ghost Oct 2, 2019

rumersdorfer added type: feature New feature or request and removed Type: Enhancement labels Nov 19, 2019

thinkh added a commit that referenced this pull request Dec 12, 2019

Remove ptvsd debugger again

c22dbc9

Removed because the debugger does not seem to work with our server startup process. For further information see #76.

keckelt mentioned this pull request Jan 24, 2020

Keckelt/python remote debugger #99

Closed

thinkh marked this pull request as draft April 10, 2020 07:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Python remote debugger for Visual Studio Code #76

Add Python remote debugger for Visual Studio Code #76

thinkh commented Aug 20, 2019 •

edited

Loading

thinkh commented Aug 22, 2019 •

edited

Loading

Add Python remote debugger for Visual Studio Code #76

Are you sure you want to change the base?

Add Python remote debugger for Visual Studio Code #76

Conversation

thinkh commented Aug 20, 2019 • edited Loading

How to start the debugger

Links

thinkh commented Aug 22, 2019 • edited Loading

thinkh commented Aug 20, 2019 •

edited

Loading

thinkh commented Aug 22, 2019 •

edited

Loading