Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest test discovery fails when pytest writes directly to process' stderr #8187

Closed
gqmelo opened this issue Oct 24, 2019 · 1 comment
Closed

Comments

@gqmelo
Copy link

gqmelo commented Oct 24, 2019

Issue Type: Bug

Since the python discovery was revamped (I don't recall when), my test discovery stopped working with pytest, giving the following message:

python <HOME_PATH>/.vscode/extensions/ms-python.python-2019.10.41019/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear -vvv -ra --color=no --no-cov test
Test Discovery failed: 
Error: 2019-10-23 11:24:14 <SOME MESSAGE FROM IN-HOUSE CODE>

The problem is caused by an internal pytest plugin that runs subprocess.check_output like this:

    ...
    stdout = subprocess.check_output(arguments)
    # Parse the output to discover some paths
    ...

This subprocess write some log messages to stderr and those messages go directly to the current process' stderr, not sys.stderr.

So when vscode runs the run_adapter.py script, the following code can't capture those log messages:

    with (util.hide_stdio() if hidestdio else util.noop_cm()) as stdio:
        ec = _pytest_main(pytestargs, [_plugin])

When running the run_adapter.py script manually I get:

2019-10-23 11:06:33 <SOME MESSAGE FROM IN-HOUSE CODE>
[<JSON_CONTENT>]

But if I run the script re-directing stderr (i.e. 2> /dev/null), I get only the json content.

So while we can fix this on our side by doing subprocess.check_output(arguments, stderr=sys.stderr), I believe it would be better if vscode tries to parse only run_adapter.py's stdout. Otherwise anyone using subprocess or doing something else that writes to the process' stderr can break vscode test discovery.

Extension version: 2019.10.41019
VS Code version: Code 1.39.2 (6ab598523be7a800d7f3eb4d92d7ab9a66069390, 2019-10-15T15:33:40.634Z)
OS version: Linux x64 4.15.0-65-generic

System Info
Item Value
CPUs Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz (4 x 3441)
GPU Status 2d_canvas: enabled
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
native_gpu_memory_buffers: disabled_software
oop_rasterization: disabled_off
protected_video_decode: unavailable_off
rasterization: disabled_software
skia_deferred_display_list: disabled_off
skia_renderer: disabled_off
surface_synchronization: enabled_on
video_decode: unavailable_off
viz_display_compositor: disabled_off
webgl: enabled
webgl2: enabled
Load (avg) 3, 3, 2
Memory (System) 15.55GB (0.15GB free)
Process Argv --unity-launch
Screen Reader no
VM 0%
@ghost ghost added the triage-needed Needs assignment to the proper sub-team label Oct 24, 2019
@brettcannon
Copy link
Member

Duplicate of #6594

@brettcannon brettcannon marked this as a duplicate of #6594 Oct 24, 2019
@ghost ghost removed the triage-needed Needs assignment to the proper sub-team label Oct 24, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants