Coverage sometimes report missing branches for excluded lines (more cases)

[skirpichev]

Just in case you miss this comment and below...

Example 1:

$ cat -n test_bug1.py 
     1  def foo(x):
     2      if x == 1:
     3          return x
     4      elif x == 2:
     5          raise NotImplementedError
     6  
     7  
     8  def test_foo():
     9      foo(1)
$ pytest -q --cov test_bug1 test_bug1.py
.                                                                                            [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug1.py       6      1      2      1    75%   4
----------------------------------------------------------
TOTAL              6      1      2      1    75%

1 passed in 0.07s

IMHO, elif on the line 4 should be ignored together with an exception and the line 2 should be marked as partially covered: no 2->exit. Currently it reports (in the html), that there is not jump 2->4.

Example 2:

$ cat -n test_bug2.py 
     1  def foo(x):
     2      try:
     3          return x + 1
     4      except TypeError:
     5          return NotImplemented
     6  
     7  
     8  def test_foo():
     9      foo(1)
$ pytest -q --cov test_bug2 test_bug2.py
.                                                                                                  [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug2.py       6      1      0      0    83%   4
----------------------------------------------------------
TOTAL              6      1      0      0    83%

1 passed in 0.07s

Example 3:

$ cat -n test_bug3.py 
     1  def foo(x):
     2      if False and not all(_ for _ in f):
     3          raise NotImplementedError
     4  
     5  
     6  def test_foo():
     7      foo([1])
$ pytest -q --cov test_bug3 test_bug3.py --cov-branch
.                                                                                         [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug3.py       4      0      2      1    83%   2->exit
----------------------------------------------------------
TOTAL              4      0      2      1    83%

1 passed in 0.07s

All on the git version, after 9209c55 with config:

$ cat pyproject.toml 
[tool.coverage.run]
branch = true
[tool.coverage.report]
exclude_lines = ['pragma: no cover',
                 'except NotImplementedError:',
                 'raise NotImplementedError',
                 'return NotImplemented']
show_missing = true

[nedbat]

Thanks for these examples. I don't think these are bugs, but they are subtle. Here's my explanations:

test_bug1

In your code as written, the elif will not be excluded, because it could still be executed, for example if x == 3. If you change the elif to an else, as I did at line 12, then the else is exclude, and if x == 1 is not considered a partial line.

Similarly, here line 23 could still execute, if an exception other than TypeError is raised. An exception like that was not raised, so that line has never executed.

Here line 31 is partial because the comprehension in all(...) never completed (or started). The if statement has two exits to start with, and the comprehension adds more, which makes it hard to interpret the coverage results.

I hope this clears things up.

[skirpichev]

Thank you, seems fair. I have not found similar reports, but maybe it could be documented (e.g. in "Branch coverage measurement")?