Incorrect coverage report with cover-branches under python 2.7

[nedbat]

Originally reported by Artem Dayneko (Bitbucket: jamert, GitHub: jamert)

---

coverage report shows non-zero coverage on completely untested files (tested under 2.7.11 and 3.5.1 on Mac OS X El Capitan - only Python 2 is affected)

Minimal example of code you can see in repository:
https://bitbucket.org/jamert/coverage-branch-report-issue

As preview: code like this

#!python

class Klass(object):
    def method(self):
        while True:
            return 1

while completely untested, shows non-zero coverage with the following configuration:

#!bash

$ cat .coveragerc
[run]
source = src
branch = True

(I would expect zero percent coverage on it).

Steps for reproducing:

#!bash

$ coverage run src/tested.py  # empty file
$ coverage report
Name            Stmts   Miss Branch BrPart  Cover
-------------------------------------------------
src/code.py         4      4      2      0    33%
src/tested.py       0      0      0      0   100%
-------------------------------------------------
TOTAL               4      4      2      0    33%

Under 3.5.1 is shows different numbers:

#!bash

$ coverage report
Name            Stmts   Miss Branch BrPart  Cover
-------------------------------------------------
src/code.py         4      4      0      0     0%
src/tested.py       0      0      0      0   100%
-------------------------------------------------
TOTAL               4      4      0      0     0%

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/502

[nedbat]

Hmm, odd. Thanks for the report. BTW, if you change "while True:" to "while x:", it correctly shows 0%, so there's something going on with the constant-branch detection.

[nedbat]

Original comment by Artem Dayneko (Bitbucket: jamert, GitHub: jamert)

---

Yes, I forgot to mention this. Also, if you make it regular function, and not method then it shows 0%, too.

[nedbat]

When I make it a function like this:

def function():
    while True:
        return 1

I get 40% as the coverage number.

[nedbat]

Original comment by Artem Dayneko (Bitbucket: jamert, GitHub: jamert)

---

So do I. It seems I tested something wrong when tried to reduce example)

[nedbat]

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)

---

The while True is statically marked as covered because it only has one branch (the loop will never end). When computing statistics, it is excluded from the list of missing arcs regardless of the fact that the code leading to this arc was run or not.

The problem does not exist in python3 because coverage.py find while True using AST instead of regexps on the sources and avoid creating arcs that cannot be reached.

#!bash

$ COVERAGE_ASTDUMP=1 COVERAGE_TRACK_ARCS=1 coverage report -m
Statements: set([1, 2, 3, 4])
Multiline map: {}
<Module
    body: [
        <ClassDef @ 1
            name: 'Klass'
            bases: [
                <Name @ 1 id: 'object'>
            ]
            body: [
                <FunctionDef @ 2
                    name: 'method'
                    args:
                        <arguments
                            args: [
                                <Name @ 2 id: 'self'>
                            ]
                            vararg: None
                            kwarg: None
                            defaults: []
                        >
                    body: [
                        <While @ 3
                            test:
                                <Name @ 3 id: 'True'>
                            body: [
                                <Return @ 4
                                    value:
                                        <Num @ 4 n: 1>
                                >
                            ]
                            orelse: []
                        >
                    ]
                    decorator_list: []
                >
            ]
            decorator_list: []
        >
    ]
>

Adding arc: (-1, 1): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
          _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @931
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (1, -1): None, "didn't exit the module"
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
          _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @933
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (-1, 1): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
        _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @950
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (1, 2): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
        _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @951
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (2, -1): None, "didn't exit the body of class 'Klass'"
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
        _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @955
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (-2, 3): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
     _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @942
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (3, 4): 'the condition on line {lineno} was never true', None
     _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @942
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @617
                      add_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @588
                _handle__While : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @904
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (4, -2): "the return on line {lineno} wasn't executed", "didn't return from function 'method'"
                _handle__While : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @904
                 add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @617
                      add_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @588
               _handle__Return : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @782
          process_return_exits : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @691
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (3, -2): 'the condition on line {lineno} was never false', "didn't return from function 'method'"
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
     _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @943
          process_return_exits : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @691
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
Statements: set([])
Multiline map: {}
<Module body: []>

Adding arc: (-1, 1): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
          _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @936
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521

Adding arc: (1, -1): None, None
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                          arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                  _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                       analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
          _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @937
                       add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
Name            Stmts   Miss Branch BrPart  Cover   Missing
-----------------------------------------------------------
src/code.py         4      4      2      0    33%   1-4
src/tested.py       0      0      0      0   100%
-----------------------------------------------------------
TOTAL               4      4      2      0    33%

[nedbat]

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)

---

For the record https://bitbucket.org/ned/coveragepy/pull-requests/103/also-use-ast-for-while-constants-in-python/diff should fix the issue for python 2.7+

[nedbat]

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)

---

For the record an alternate fix was proposed earlier this year

[nedbat]

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)

---

For the record the updated pull request is at https://bitbucket.org/ned/coveragepy/pull-requests/117/also-use-ast-for-while-constants-in-python/diff

[nedbat]

also use AST for while constants in python-2.7 #502

The node.id is set to False, True or None is python-2.7: there is no
reason to only check for it with python-3. It is more reliable than
using the DEFAULT_PARTIAL_ALWAYS regexps on source lines.

close #502

→ <<cset d9cc167007ef (bb)>>

[nedbat]

Fixed in the commits culminating in deaddb34f45e (bb)

[nedbat]

This fix was released as part of Coverage.py 4.3.