coverage combine when combining results from different interpreter versions results in missing coverage

[graingert]

Describe the bug
coverage combine when combining results from different interpreter versions results in missing coverage

To Reproduce
How can we reproduce the problem? Please be specific. Don't just link to a failing CI job. Answer the questions below:

1. What version of Python are you using? 3.5, 3.6, 3.7, 3.8
2. What version of coverage.py are you using? The output of coverage debug sys is helpful. 5.3
3. What versions of what packages do you have installed? The output of pip freeze is helpful.
4. What code are you running? Give us a specific commit of a specific repo that we can check out. https://codecov.io/gh/twisted/ldaptor/pull/207/changes
5. What commands did you run? see https://github.com/twisted/ldaptor/pull/207/files#diff-522adf759addbd3b193c74ca85243f7dR73

Expected behavior
I expect to see line 40 of ldaptor/protocols/ldap/ldapconnector.py covered

Additional context
I'm currently trying out tox --parallel to save consumption of GHA parallel builds. Each build runs and then coverage combine runs at the end, as advised here https://tox.readthedocs.io/en/latest/example/basic.html#parallel-mode
see twisted/ldaptor#207

[nedbat]

I'm not sure what can be done about this. Different versions of Python produce slightly different data when tracing the same program, because of optimizations, or because of changed decisions about what the "right" trace is. If you measure on one version, then report using the same version, everything should make sense. But if you measure on one version, then report using a different version, you might get results that don't seem right.

The problem is only made more complicated when combining data from different runs.

But: you mentioned line 40 of ldapconnector.py should be covered. This is what I see:

Coverage.py knows that the "while True" is optimized away. This isn't an issue, since the line isn't marked as uncovered.

[graingert]

@nedbat am I misinterpreting codecov here?

[nedbat]

There are three possible colors for a line: green (executed), red (could have executed but didn't), and while (couldn't have executed). Because the while True is optimized away, it is white. Only the red lines are problems, not the white ones.

[graingert]

ah I see, so it's the correct result for 3.8+?

[nedbat]

Yes :)

[graingert]

@nedbat closed as working as designed?

[nedbat]

That was my interpretation of the discussion. Was I wrong?

[graingert]

That was my interpretation of the discussion. Was I wrong?

That was my interpretation, just double checking for myself and noting for posterity