[internal] Generator-expression exit arcs aren't always filtered out of executed_branch_arcs

[zackw]

Consider this code snippet:

import sys

def foo(args):
    if any(arg.startswith("-") for arg in args):
        print("option found")

foo(sys.argv)

(Any use of a generator expression in the controlling condition of an if should provoke the problem, albeit I have not tried it with a multi-line controlling condition.)

If I run this snippet as coverage run --branch test.py or coverage run --branch test.py nonoption it produces this arc table:

sqlite3 -table .coverage 'select fromno, tono from arc order by fromno, tono'

fromno	tono
-4	4
-3	4
-1	1
1	3
3	7
4	-4
4	-3
4	4
7	-1

If I run it as coverage run -branch test.py -flag I get this arc table instead:

fromno	tono
-4	4
-3	4
-1	1
1	3
3	7
4	4
4	5
5	-3
7	-1

The problem is with the arc (4, -4), which is (as far as I can tell) the exit arc for the generator expression. The analysis code is supposed to be filtering this arc out as uninteresting, but either it only does this for missing_branch_arcs() and not for executed_branch_arcs(), or there's a bug in its handling of this construct. When the input is the first of the above two arc tables, executed_branch_arcs()[4] is {-4, -3, 4}.

The HTML report generator doesn't care about this because it only looks at missing arcs, but the LCOV report generator looks at both missing and executed arcs and trips over the (4, -4) edge. With the code in #1851, the difference in coverage lcov output for the above two arc tables is

--- coverage.1.lcov	2024-09-12 14:19:43.422893468 -0400
+++ coverage.2.lcov	2024-09-12 14:20:11.648031969 -0400
@@ -2,17 +2,16 @@
 DA:1,1
 DA:3,1
 DA:4,1
-DA:5,0
+DA:5,1
 DA:7,1
 LF:5
-LH:4
+LH:5
 FN:3,5,foo
 FNDA:1,foo
 FNF:1
 FNH:1
-BRDA:4,0,to line 5,0
-BRDA:4,0,to exit 3,1
-BRDA:4,0,to exit 4,1
+BRDA:4,0,to line 5,1
+BRDA:4,0,to exit,0
 BRF:2
 BRH:1
 end_of_record

The BRDA: lines we should be generating are

-BRDA:4,0,to line 5,0
-BRDA:4,0,to exit,1
+BRDA:4,0,to line 5,1
+BRDA:4,0,to exit,0

but I do not see any way to make that happen from inside lcovreport.py.

[nedbat]

Are you sure you are running the code from the tip of master when measuring coverage? There should no longer be a (4, -4) arc in the data, as of commit 2afe04d:

2afe04d1 2024-08-26 refactor: don't test arcs, test branches

[zackw]

Yes, I'm sure. I added three tests to the testsuite specifically for this; all of the test failures on the most recent push for #1851 are because in two out of three cases the equivalent of the (4, -4) arc is visible to the lcov reporter.

[zackw]

Fine-tuned the tests a little more in commit 402bee8. Given this function

def foo(a):
    if any(x > 0 for x in a):
        print(f"{a!r} has positives")

the genexpr exit arc is pruned by the analysis engine if and only if the arc exiting the whole function is never taken. For example, if foo is never called at all, or if the only call is foo([1]), then the genexpr exit arc will be pruned, but if the only call is foo([0]) or if it's called twice as foo([0]) and foo([1]) then the genexpr exit arc will not be pruned.

CI test results for 402bee8

[nedbat]

Is this issue done now also?

[zackw]

Thanks for your patience; have only just now had a chance to test the new code against the full application that prompted this report. It generally looks good. There are no longer any cases where the LCOV report includes three or more branch destinations for a single line. There are also no longer any cases where negative arc destination numbers appear in the LCOV output.

I do see a few places where the generic "jump to the function exit" text appears in the LCOV output. I will file follow-up bugs for these later today. I think you can go ahead and mark this issue as resolved in 7.6.2.

[nedbat]

This is now fixed as part of coverage 7.6.2.