Tests not found after upgrade

[david-waterworth]

The latest release of vscode seems to have broken my unit tests. Although there seems to be a built in assumption that there's only 1 test folder per project which isn't the case for this project, I got the following to at least work with the previous version (I have multiple packages, each package has it's own test folder) although I had to edit it manually to switch from one test folder to another

"python.unitTest.cwd": "${workspaceFolder}/packages/package_1",

Now though, after upgrading to 1.33.0 I get a prompt asking me to configure the test framework, this now also seems to assume not only 1 test folder per project, it also seems to expect this to be off the root? Ideally vscode wouldn't make these sort of assumptions. I edited settings.json but in any case I now get tests not found

"python.unitTest.unittestArgs": [
    "-v",
    "-s",
    "./packages/package_1/tests",
    "-p",
    "test_*.py"
],

It's not clear to me how this interacts with python.unitTest.cwd - does test discovery first cwd, it doesn't appear to? I've tried other combinations, i.e.

"python.unitTest.cwd": "${workspaceFolder}/packages/package_1",
"python.unitTest.unittestArgs": [
    "-v",
    "-s",
    "./tests",
    "-p",
    "test_*.py"
],

There's a number of errors in the output which appear pyenv related which I've copied at the bottom.

• VS Code version: 1.33.0
• Extension version (available under the Extensions sidebar): python 2019.3.6558
• OS and version: ubuntu 18.04
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.1
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pyenv
• Relevant/affected Python packages and their versions: unittest

Output from Console under the Developer Tools:

console.ts:134 [Extension Host] (node:10170) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

console.ts:134 [Extension Host] Python Extension: Detection of Python Interpreter for Command python3.7 and args  failed as file  does not exist

console.ts:134 [Extension Host] Python Extension: Failed to parse interpreter information for '/home/david/.pyenv/shims/python3.7' with JSON pyenv: python3.7: command not found

The `python3.7' command exists in these Python versions:
  3.7.1
  3.7.1/envs/machine-learning-3.7.1
  3.7.1/envs/test-upgrade-3.7.1
  machine-learning-3.7.1
  test-upgrade-3.7.1 SyntaxError: Unexpected token p in JSON at position 0
	at JSON.parse (<anonymous>)
	at f.<anonymous> (/home/david/.vscode/extensions/ms-python.python-2019.3.6558/out/client/extension.js:83:304721)
	at Generator.next (<anonymous>)
	at s (/home/david/.vscode/extensions/ms-python.python-2019.3.6558/out/client/extension.js:83:304048)

[david-waterworth]

Actually this might be another case of something I noticed in the past.

One of my unit tests had an import error in it (I'd miss-spelt the name of the test class), this caused the test to fail. Similarly, when I changed the name of a class in my package but failed to update the packages __init__.py such that the package cannot be imported.

In each case this causes vscode to report tests not found instead of either a more descriptive error (i.e. the actual import error) or better still not treating an import error in the test fixture as a failure to find the tests

[brettcannon]

We are going through old issues and we noticed that no one from the team had replied to this issue. I want to apologize for the oversight and to let you know that the issue was reviewed by the team and triaged (as shown by the labels applied to this issue).