Running all tests in a test class raises an error

[hampgoodwin]

Environment data

• VS Code version: 1.23.1
• Extension version (available under the Extensions sidebar): 2018.6.0
• OS and version: Win10
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.5
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pipenv venv
• Relevant/affected Python packages and their versions: XXX

Actual behavior

gif of issue being produced
Issue occurs at about 40 seconds in.
When attempting to run all tests within a specific test class, no tests are ran and an error is thrown; after the error is thrown, the green check tests passed bar seems to aggregate all past succesful runs of any tests instead of the current run.
In this you'll see that the test methods within the erroring test class will run, yet this is not always the functionality when this occurs, more often than not the same error notification will appear, yet within the console the test will indicate it has ran.

Expected behavior

Successful testing of all methods contained within a test class. Correct test count. No inexplicit error popup.

Steps to reproduce:

I've recreated the folder structure and files with minimal code and had the app_test.py behave similarly.

Logs

test_repr (tests.unit.blog_test.BlogTest) ... ok
test_repr_multiple_posts (tests.unit.blog_test.BlogTest) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK
test_create_post_in_blog (tests.integration.blog_test.BlogTest) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
test_print_blogs (tests.system.app_test.AppTest) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
test_create_blog (tests.unit.blog_test.BlogTest) ... ok
test_repr (tests.unit.blog_test.BlogTest) ... ok
test_repr_multiple_posts (tests.unit.blog_test.BlogTest) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

dom.ts:179 [Violation] Added non-passive event listener to a scroll-blocking 'mousewheel' event. Consider marking event handler as 'passive' to make the page more responsive.
console.ts:136 [Extension Host] Python Extension: Error: read ECONNRESET 
t.log @ console.ts:136
t._logExtensionHostMessage @ extensionHost.ts:393
(anonymous) @ extensionHost.ts:210
emitTwo @ events.js:106
emit @ events.js:194
process.nextTick @ internal/child_process.js:766
_combinedTickCallback @ internal/process/next_tick.js:73
_tickCallback @ internal/process/next_tick.js:104
log.ts:171   ERR Unexpected end of JSON input: SyntaxError: Unexpected end of JSON input
	at JSON.parse (<anonymous>)
	at Socket.socket.on (C:\Users\abelg\.vscode\extensions\ms-python.python-2018.6.0\out\client\unittests\unittest\socketServer.js:77:38)
	at emitOne (events.js:96:13)
	at Socket.emit (events.js:191:7)
	at readableAddChunk (_stream_readable.js:178:18)
	at Socket.Readable.push (_stream_readable.js:136:10)
	at TCP.onread (net.js:560:20)
12console.ts:136 [Extension Host] Python Extension: Error: read ECONNRESET 
t.log @ console.ts:136
t._logExtensionHostMessage @ extensionHost.ts:393
(anonymous) @ extensionHost.ts:210
emitTwo @ events.js:106
emit @ events.js:194
process.nextTick @ internal/child_process.js:766
_combinedTickCallback @ internal/process/next_tick.js:73
_tickCallback @ internal/process/next_tick.js:104
dom.ts:179 [Violation] Added non-passive event listener to a scroll-blocking 'mousewheel' event. Consider marking event handler as 'passive' to make the page more responsive.
browser.ts:166 [Violation] Added non-passive event listener to a scroll-blocking 'touchstart' event. Consider marking event handler as 'passive' to make the page more responsive.

[DonJayamanne]

Would it be possible to provide a sample repo that we could use to replicate this issue.

[hampgoodwin]

Yes:

https://github.com/abelgoodwin1988/automated-software-testing-with-python

Falling asleep for now, ill provide any additional needed info in the morning.

[DonJayamanne]

Please provide the contents of your settings.json.
Works for me, here's my settings.json:

{
    "python.pythonPath": "/Users/donjayamanne/.local/share/virtualenvs/automated-software-testing-with-python-l2Qoc3kW/bin/python",
    "python.unitTest.unittestArgs": [
        "-v",
        "-s",
        "./blog",
        "-p",
        "*_test.py"
    ],
    "python.unitTest.pyTestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.unittestEnabled": true
}

[hampgoodwin]

{
    "python.pythonPath": "C:\\Users\\abelg\\.virtualenvs\\automated-software-testing-with-python-f48EVg63",
    "python.unitTest.unittestArgs": [
        "-v",
        "-s",
        "./blog",
        "-p",
        "*_test.py"
    ],
    "python.unitTest.pyTestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.unittestEnabled": true,
    "files.exclude": {
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/__pycache__": true
    }
}

[hampgoodwin]

from the developers console there is this piece of logging which appears when I get the error message:

ERR Unexpected end of JSON input: SyntaxError: Unexpected end of JSON input
	at JSON.parse (<anonymous>)
	at Socket.socket.on (C:\Users\abelg\.vscode\extensions\ms-python.python-2018.6.0\out\client\unittests\unittest\socketServer.js:77:38)
	at emitOne (events.js:96:13)
	at Socket.emit (events.js:191:7)
	at readableAddChunk (_stream_readable.js:178:18)
	at Socket.Readable.push (_stream_readable.js:136:10)
	at TCP.onread (net.js:560:20)

[d3r3kk]

Thanks for the extra information @abelgoodwin1988, we'll attempt a repro on our end.

[hampgoodwin]

Let me know if there's any additional info I can provide or anything I can do to help.

[DonJayamanne]

@abelgoodwin1988 Is this issue occurring in all workspaces?
I.e. if you were to create a new workspace and use a different version of Python, do you still experience this issue?

[hampgoodwin]

Ran in a python 3.5.4 environment with same workspace settings and the issue still exists.

Edit: Went through some iterations and it's behaving the same as before. This evening I'm going to try and reinstall VScode itsself and see if that sorts anything out.

[hampgoodwin]

No joy, reinstalled after clearing cached vscode settings and issue persists.

[hampgoodwin]

I've just reproduced this on two other windows 10 machines.

[d3r3kk]

@DonJayamanne, I can reproduce this on Linux (Ubuntu 18.04) with the latest insiders build + latest production version of vscode-python extension.

I can look into it further and do some debugging...

[d3r3kk]

@abelgoodwin1988 I believe I found the root cause and corrected it with my change from yesterday.
Thanks so much for the report & detailed reproduction data!

[hampgoodwin]

Great News, thanks for your work on this!