Running doctest prevents complete coverage collection

[nedbat]

sample.py:

#!python

import doctest

TEST = """
>>> x = 1
>>> x
1
"""

doctest.run_docstring_examples(TEST, {})

q = 11
r = 12

Running coverage leaves the last two lines unmeasured:

$ coverage run sample.py && coverage report -m
Name        Stmts   Miss  Cover   Missing
-----------------------------------------
sample.py       5      2    60%   11-12

Switching from the C tracer to the Python tracer fixes it:

$ coverage run --timid sample.py && coverage report -m
Name        Stmts   Miss  Cover   Missing
-----------------------------------------
sample.py       5      0   100%

The problem only appears in Python 3. It's due to the sys.settrace in the finally clause in this commit: python/cpython@31f5929, which was to fix this bug: http://bugs.python.org/issue10990

There seems to be something wrong with getting and restoring the C trace function. (Odd though, since there are tests that check this...)

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/575

[nedbat]

Thanks to @chriswithers for the original report.

[nedbat]

A simpler reproducer, with no doctest:

sample.py:

from swap import swap_it

print(3)
swap_it()
print(5)

swap.py:

import sys

def swap_it():
    no_really()

def no_really():
    save_trace = sys.gettrace()
    sys.settrace(save_trace)

Three runs:

$ coverage run sample.py && coverage report -m
3
5
Name        Stmts   Miss  Cover   Missing
-----------------------------------------
sample.py       4      0   100%
swap.py         6      0   100%
-----------------------------------------
TOTAL          10      0   100%
$ coverage run --source=sample sample.py && coverage report -m
3
5
Name        Stmts   Miss  Cover   Missing
-----------------------------------------
sample.py       4      1    75%   5
$ coverage run --source=sample --timid sample.py && coverage report -m
3
5
Name        Stmts   Miss  Cover   Missing
-----------------------------------------
sample.py       4      0   100%

I'm not sure why --source changes the behavior. Also, if the settrace(gettrace) happens in sample.py, then the problem doesn't happen. All of this is the same on Python 2.7 and Python 3.6 (the py3 claim above is because of changes in doctest).

[nedbat]

More details as they dribble in: this also happens with coverage.py 3.7.1.

Seems that if the file that does the settrace(gettrace) isn't being traced itself, the problem happens. This is why it happens with doctest, and with my own file without --source.

[nedbat]

Fixed in 03b48c292790 (bb)

[nedbat]

Original comment by Chris Withers (Bitbucket: chrisw, GitHub: chrisw)

---

Fun times, thanks Ned! Glad you fight this craziness so the rest of us don't have to ;-)
When's 4.4 due out?

[nedbat]

I'm aiming for within a week.