Show unittest.subTest failures in the Test Explorer

[DolajoCZ]

Hello, can you please add support of unittest subTest. Thank you

[jakebailey]

Can you describe what it is about this function that isn't working correctly? At first glance, I don't know what might be broken (and we get unittest stubs from the typeshed project). A code example would be helpful.

[DolajoCZ]

If I use code from picture below, status of test is unknown (left top corner) although it should fail as it is shown on next image - result from command line.

If I use code without subTest everything is ok - test fail.

[jakebailey]

Oh, this isn't a Pylance thing, this is the text explorer in the Python extension. I'll transfer it.

[kimadeline]

Hi @DolajoCZ,

Thank you for reaching out! I can confirm that this issue hasn't been introduced recently, I could reproduce it using Python 3.9.0 and the following versions of the extension:

• 2021.1.502429796
• 2020.12.424452561
• 2020.11.371526539

We'll have to investigate it.

---

Notes:

1. When running tests using the subtest feature, a "No tests were ran" notification appears:

2. the output is slightly different when having tests that don't use subtests, especially the Content part:

successful tests:

test (test_file.test)
Content ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

failing tests:

test (test_file.test)
Content ... FAIL
NoneType: None

======================================================================
FAIL: test (test_file.test)
Content
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kimiguel/Documents/Sandbox/unittest-subtest/test_file.py", line 13, in test
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (failures=1)

failing tests with subtests:

test (test_file.test)
Content ... 
======================================================================
FAIL: test (test_file.test) (i=1)
Content
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kimiguel/Documents/Sandbox/unittest-subtest/test_file.py", line 13, in test
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test (test_file.test) (i=3)
Content
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kimiguel/Documents/Sandbox/unittest-subtest/test_file.py", line 13, in test
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test (test_file.test) (i=5)
Content
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kimiguel/Documents/Sandbox/unittest-subtest/test_file.py", line 13, in test
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=3)

[DolajoCZ]

Any update?

[kimadeline]

We made some testing-related changes in the latest version of the extension, is it still not working? #16769

[DolajoCZ]

Unfortunatelly not. I used example bellow but without succes. My version of Python extension is 2021.8.1105858891 and Python in version 3.9.2. Can you please share with me working code and potential additonal setting if necessary? Thank you @kimadeline

"""Test module."""
import unittest


class TestSbTest(unittest.TestCase):
    """Unittest"""

    def test_fail_case(self) -> None:
        """Test fail case."""
        for i in range(6):
            with self.subTest(i=i):
                self.assertEqual(i % 2, 0)

            # self.assertEqual(i % 2, 0)

[kimadeline]

We still haven't addressed it yet then.

[DolajoCZ]

Sorry, I misinterpreted your comment. I thought that this fix was already released.

[kimadeline]

Sorry for the confusion! This will be addressed as part of #16936.

[karrtikr]

Closing in favor of #16936 where we'll keep this issue in mind while fixing.