pytestArgs: ["-n", "4"] not working for running tests in parallel with pytest-xdist

[luabud]

Easy fix is to change docs to have "-n 4" instead of two args, but I don't understand why "-n", "4" is not working.

Environment data

• VS Code version: 1.35.1
• Extension version (available under the Extensions sidebar): 2019.7.21668-dev
• OS and version: Windows 10
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.1
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
• Relevant/affected Python packages and their versions:
  

Expected behaviour

"python.testing.pytestArgs": ["-n 4"] and "python.testing.pytestArgs": ["-n", "4"] should have the same behaviour, shouldn't it?

Actual behaviour

Having "python.testing.pytestArgs": ["-n", "4"] in settings.json throws an error:

pytest.py: error: argument -n/--numprocesses: invalid parse_numprocesses value: '.\\testing\\python\\approx.py'

and approx.py is the file from which I tried to execute the tests.

Steps to reproduce:

1. Clone pytest repo
2. Install dependencies if needed. Make sure pytest-xdist is installed
3. Configure tests in the extension to use pytest framework and add
   "python.testing.pytestArgs": ["-n", "4"] to settings.json file
4. Open the test explorer
5. Discover tests works
6. Run all tests works
7. Run an individual test file and watch the "There was an error in running the tests" prompt appear

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

=========================== short test summary info ===========================
FAILED testing/logging/test_formatter.py::test_colored_short_level - Assertio...
===================== 1 failed, 2 passed in 0.98 seconds ======================
Error: Error: cannot open file:///c%3A/Users/luabud/pytest/c. Detail: Unable to read file (Error: File not found (c:\Users\luabud\pytest\c))
Test Discovery failed: 
TypeError: i.startsWith is not a function
Test Discovery failed: 
TypeError: i.startsWith is not a function
python C:\Users\luabud\.vscode\extensions\ms-python.python-2019.7.21668-dev\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear -n 4
ERROR: usage: pytest.py [options] [file_or_dir] [file_or_dir] [...]
pytest.py: error: argument -n/--numprocesses: invalid parse_numprocesses value: '.\\testing\\python\\approx.py'

Error: TypeError: Cannot read property 'testsuite' of null
Test Discovery failed: 
TypeError: i.startsWith is not a function
Test Discovery failed: 
TypeError: i.startsWith is not a function
Test Discovery failed: 
TypeError: i.startsWith is not a function
python C:\Users\luabud\.vscode\extensions\ms-python.python-2019.7.21668-dev\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear -n 4
ERROR: usage: pytest.py [options] [file_or_dir] [file_or_dir] [...]
pytest.py: error: argument -n/--numprocesses: invalid parse_numprocesses value: '.\\testing\\python\\integration.py'

Error: TypeError: Cannot read property 'testsuite' of null

[DonJayamanne]

Looks like a problem with pytest. Could be because if the way pytests parsed arguments and pass them to the plugins

[DonJayamanne]

Works for me.

[DonJayamanne]

Cause:

We're appending the file name to the list of args.
The arguments are ["-n", "4"], and when we append file name here, it turns into ["-n", "4", "<file name>"]
Instead it should be ["<file name>", "-n", "4"]

[DonJayamanne]

Prescribed Solution

• Modify https://github.com/microsoft/vscode-python/blob/566d759340d527a0f5b628bcca95d1eecaf0c698/src/client/testing/pytest/runner.ts

Line: 52
           testArgs.push(...testPaths);

[ericsnowcurrently]

I'm not sure we understand the problem yet. Passing the file last should work fine. If there's a pytest bug then we a workaround is fine as long as we file an upstream bug and document the workaround.

[luabud]

Validated