Core test coverage report doesn't work locally

[saroad2]

Describe the bug
In PR #1643 the coverage configuration of the core library was moved to the core library folder.

Now it seems like I cannot see the actual coverage reprot when I run it locally.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'src/core'
2. Run pip install tox
3. Run tox -e py
4. Run coverage combine
5. Run coverage report
6. Get an error: No source for code: 'C:\Users\sagi\PycharmProjects\toga\src\core\src\toga_dummy\__init__.py'

Expected behavior
Getting coverage report for the core library should work locally.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

• Operating System: Windows
• Python version: 3.10
• Software versions:
  • Briefcase: latest
  • Toga: development
  • ...

Additional context
I tried to move the coverage report created by tox -e py to the root directory and then run coverage combine and coverage report, the same way as it seems like the CI does it, but than I get the following:

Name                                                     Stmts   Miss Branch BrPart    Cover   Missing
------------------------------------------------------------------------------------------------------
src/core/src/toga/__init__.py                               36     36      0      0     0.0%   1-100
src/core/src/toga/app.py                                   230    230     54      0     0.0%   1-660
src/core/src/toga/colors.py                                  1      1      0      0     0.0%   2
src/core/src/toga/command.py                               175    175     50      0     0.0%   1-406
src/core/src/toga/constants/__init__.py                      1      1      0      0     0.0%   1
src/core/src/toga/documents.py                              11     11      0      0     0.0%   1-16
src/core/src/toga/fonts.py                                  32     32      6      0     0.0%   1-102
src/core/src/toga/handlers.py                               45     45     16      0     0.0%   1-78
src/core/src/toga/icons.py                                  57     57     20      0     0.0%   1-118
src/core/src/toga/images.py                                 31     31     16      0     0.0%   1-62
src/core/src/toga/keys.py                                  136    136      0      0     0.0%   1-168
src/core/src/toga/platform.py                               43     43     22      0     0.0%   1-120
src/core/src/toga/sources/__init__.py                        5      5      0      0     0.0%   1-5
src/core/src/toga/sources/accessors.py                      19     19     16      0     0.0%   1-74
src/core/src/toga/sources/base.py                           18     18      4      0     0.0%   1-35
src/core/src/toga/sources/list_source.py                    56     56     12      0     0.0%   1-110
src/core/src/toga/sources/tree_source.py                    92     92     22      0     0.0%   1-154
src/core/src/toga/sources/value_source.py                   14     14      6      0     0.0%   1-18
src/core/src/toga/style/__init__.py                          2      2      0      0     0.0%   1-2
src/core/src/toga/style/applicator.py                       19     19      4      0     0.0%   1-33
src/core/src/toga/style/pack.py                            260    260    112      0     0.0%   1-574
src/core/src/toga/validators.py                            159    159     62      0     0.0%   1-373
src/core/src/toga/widgets/activityindicator.py              23     23      8      0     0.0%   1-64
src/core/src/toga/widgets/base.py                          133    133     60      0     0.0%   1-299
src/core/src/toga/widgets/box.py                            11     11      4      0     0.0%   1-42
src/core/src/toga/widgets/button.py                         45     45     10      0     0.0%   1-142
src/core/src/toga/widgets/canvas.py                        361    361     22      0     0.0%   1-1175
src/core/src/toga/widgets/datepicker.py                     61     61     18      0     0.0%   1-146
src/core/src/toga/widgets/detailedlist.py                   64     64      8      0     0.0%   1-176
src/core/src/toga/widgets/divider.py                        20     20      2      0     0.0%   1-58
src/core/src/toga/widgets/imageview.py                      21     21      6      0     0.0%   1-55
src/core/src/toga/widgets/label.py                          19     19      4      0     0.0%   1-56
src/core/src/toga/widgets/multilinetextinput.py             51     51      6      0     0.0%   1-140
src/core/src/toga/widgets/navigationview.py                 14     14      0      0     0.0%   1-19
src/core/src/toga/widgets/numberinput.py                    89     89     18      0     0.0%   1-203
src/core/src/toga/widgets/optioncontainer.py               188    188     70      0     0.0%   1-460
src/core/src/toga/widgets/passwordinput.py                   4      4      0      0     0.0%   1-9
src/core/src/toga/widgets/progressbar.py                    48     48     10      0     0.0%   1-119
src/core/src/toga/widgets/scrollcontainer.py                75     75     14      0     0.0%   1-150
src/core/src/toga/widgets/selection.py                      42     42     10      0     0.0%   1-102
src/core/src/toga/widgets/slider.py                        109    109     24      0     0.0%   1-244
src/core/src/toga/widgets/splitcontainer.py                 64     64     26      0     0.0%   1-137
src/core/src/toga/widgets/switch.py                         85     85     22      0     0.0%   1-238
src/core/src/toga/widgets/table.py                          91     91     18      0     0.0%   1-256
src/core/src/toga/widgets/textinput.py                      95     95     18      0     0.0%   1-219
src/core/src/toga/widgets/timepicker.py                     62     62     18      0     0.0%   1-157
src/core/src/toga/widgets/tree.py                           54     54      6      0     0.0%   1-156
src/core/src/toga/widgets/webview.py                        55     55      2      0     0.0%   1-166
src/core/src/toga/window.py                                136    136     24      0     0.0%   1-454
src/dummy/src/toga_dummy/__init__.py                         2      2      0      0     0.0%   1-11
src/dummy/src/toga_dummy/app.py                             45     45      2      0     0.0%   1-64
src/dummy/src/toga_dummy/colors.py                           2      2      0      0     0.0%   1-2
src/dummy/src/toga_dummy/command.py                          7      7      0      0     0.0%   1-10
src/dummy/src/toga_dummy/container.py                       19     19      2      0     0.0%   1-26
src/dummy/src/toga_dummy/dialogs.py                         24     24      0      0     0.0%   1-76
src/dummy/src/toga_dummy/documents.py                        5      5      0      0     0.0%   1-7
src/dummy/src/toga_dummy/factory.py                         37     37      0      0     0.0%   1-42
src/dummy/src/toga_dummy/fonts.py                            5      5      0      0     0.0%   1-7
src/dummy/src/toga_dummy/icons.py                            8      8      0      0     0.0%   1-11
src/dummy/src/toga_dummy/images.py                          15     15      6      0     0.0%   1-19
src/dummy/src/toga_dummy/paths.py                           27     27      2      0     0.0%   1-43
src/dummy/src/toga_dummy/test_implementation.py            231    231    134      0     0.0%   1-453
src/dummy/src/toga_dummy/utils.py                          147    147     34      0     0.0%   1-476
src/dummy/src/toga_dummy/widgets/activityindicator.py        8      8      0      0     0.0%   1-12
src/dummy/src/toga_dummy/widgets/base.py                    43     43      0      0     0.0%   1-69
src/dummy/src/toga_dummy/widgets/box.py                      4      4      0      0     0.0%   1-6
src/dummy/src/toga_dummy/widgets/button.py                  10     10      0      0     0.0%   1-15
src/dummy/src/toga_dummy/widgets/canvas.py                  61     61      0      0     0.0%   1-148
src/dummy/src/toga_dummy/widgets/datepicker.py              16     16      0      0     0.0%   1-24
src/dummy/src/toga_dummy/widgets/detailedlist.py            27     27      0      0     0.0%   1-40
src/dummy/src/toga_dummy/widgets/divider.py                  8      8      0      0     0.0%   1-12
src/dummy/src/toga_dummy/widgets/imageview.py                6      6      0      0     0.0%   1-9
src/dummy/src/toga_dummy/widgets/label.py                   10     10      0      0     0.0%   1-15
src/dummy/src/toga_dummy/widgets/multilinetextinput.py      14     14      0      0     0.0%   1-21
src/dummy/src/toga_dummy/widgets/navigationview.py           8      8      0      0     0.0%   1-12
src/dummy/src/toga_dummy/widgets/numberinput.py             20     20      0      0     0.0%   1-30
src/dummy/src/toga_dummy/widgets/optioncontainer.py         41     41      4      0     0.0%   1-57
src/dummy/src/toga_dummy/widgets/passwordinput.py            4      4      0      0     0.0%   1-6
src/dummy/src/toga_dummy/widgets/progressbar.py             12     12      0      0     0.0%   1-18
src/dummy/src/toga_dummy/widgets/scrollcontainer.py         24     24      0      0     0.0%   1-34
src/dummy/src/toga_dummy/widgets/selection.py               16     16      0      0     0.0%   1-24
src/dummy/src/toga_dummy/widgets/slider.py                  20     20      0      0     0.0%   1-30
src/dummy/src/toga_dummy/widgets/splitcontainer.py           8      8      0      0     0.0%   1-12
src/dummy/src/toga_dummy/widgets/switch.py                  14     14      0      0     0.0%   1-21
src/dummy/src/toga_dummy/widgets/table.py                   27     27      0      0     0.0%   1-40
src/dummy/src/toga_dummy/widgets/textinput.py               32     32      0      0     0.0%   1-47
src/dummy/src/toga_dummy/widgets/timepicker.py              16     16      0      0     0.0%   1-24
src/dummy/src/toga_dummy/widgets/tree.py                    21     21      0      0     0.0%   1-31
src/dummy/src/toga_dummy/widgets/webview.py                 24     24      0      0     0.0%   1-35
src/dummy/src/toga_dummy/window.py                          61     61      2      0     0.0%   1-85
------------------------------------------------------------------------------------------------------
TOTAL                                                     4591   4591   1036      0     0.0%

4 files skipped due to complete coverage.

It seems like the coverage is 0%...

[mhsmith]

I'm taking a look at this.

[saroad2]

The problem is still happening after the merge of #1669

[mhsmith]

Yes, I think I understand what's going on now. I'll post a full explanation later, but for now can you try upgrading to coverage version 6.6.0b1, and then repeating the process you described under "Additional context"?

[saroad2]

Ye, it solved the problem :)

Name                                              Stmts   Miss Branch BrPart    Cover   Missing
-----------------------------------------------------------------------------------------------
core/src/toga/app.py                                230     31     54     17    81.7%   18-20, 121, 215, 227-230, 241->246, 257, 275, 278, 285, 288, 293, 300, 307, 314, 328, 384-387, 406, 414, 422, 430, 453, 474, 503->exit, 509, 513, 519->522, 531, 539-540, 558, 578->exit
core/src/toga/command.py                            175      4     50      1    96.9%   138-140, 287->exit, 350
core/src/toga/constants/__init__.py                   1      1      0      0     0.0%   1
core/src/toga/fonts.py                               32      1      6      3    89.5%   41, 95->97, 97->99, 99->102
core/src/toga/handlers.py                            45     24     16      3    45.9%   9, 14-23, 27-33, 56, 60, 66, 72-74
core/src/toga/icons.py                               57      4     20      4    87.0%   17, 62, 102, 113
core/src/toga/keys.py                               136      5      0      0    96.3%   160-164, 168
core/src/toga/platform.py                            43      1     22      1    96.9%   38
core/src/toga/sources/tree_source.py                 92      3     22      0    97.4%   69, 114-115
core/src/toga/style/applicator.py                    19      0      4      1    95.7%   16->15
core/src/toga/style/pack.py                         260     42    112      6    82.3%   85, 248-257, 261, 304-317, 401-408, 424, 470-480, 526-545       
core/src/toga/validators.py                         159      2     62      0    99.1%   17, 50
core/src/toga/widgets/base.py                       133      5     60      7    93.8%   71, 103, 107, 121->120, 152->168, 169, 182->181, 288, 298->exit 
core/src/toga/widgets/detailedlist.py                64      1      8      0    98.6%   162
core/src/toga/widgets/imageview.py                   21      3      6      1    85.2%   43, 54-55
core/src/toga/widgets/navigationview.py              14     14      0      0     0.0%   1-19
core/src/toga/widgets/optioncontainer.py            188      1     70      0    99.6%   105
core/src/toga/widgets/scrollcontainer.py             75      4     14      1    89.9%   58->exit, 63-64, 90-91
core/src/toga/widgets/slider.py                     109      8     24      5    90.2%   85, 161, 174, 179, 180->exit, 234-237, 241-244
core/src/toga/widgets/table.py                       91      2     18      0    98.2%   252, 256
core/src/toga/widgets/textinput.py                   95      0     18      1    99.1%   211->210
core/src/toga/widgets/tree.py                        54      3      6      0    95.0%   116, 126, 146
core/src/toga/widgets/webview.py                     55      2      2      0    96.5%   87, 106
core/src/toga/window.py                             136      0     24      2    98.8%   184->exit, 256->exit
dummy/src/toga_dummy/app.py                          45      7      2      0    85.1%   8, 17, 21, 30, 40, 52, 56
dummy/src/toga_dummy/colors.py                        2      2      0      0     0.0%   1-2
dummy/src/toga_dummy/container.py                    19     19      2      0     0.0%   1-26
dummy/src/toga_dummy/images.py                       15      0      6      1    95.2%   15->exit
dummy/src/toga_dummy/paths.py                        27      2      2      1    89.7%   20-23
dummy/src/toga_dummy/test_implementation.py         231    231    134      0     0.0%   1-453
dummy/src/toga_dummy/utils.py                       147     48     34      1    60.8%   176, 210-217, 226-235, 250-292, 313-322, 333-353, 373-374, 401-402, 420, 447-448, 460-476
dummy/src/toga_dummy/widgets/base.py                 43      6      0      0    86.0%   23, 27, 43, 46, 49, 66
dummy/src/toga_dummy/widgets/datepicker.py           16      1      0      0    93.8%   21
dummy/src/toga_dummy/widgets/detailedlist.py         27      6      0      0    77.8%   12, 15, 18, 21, 24-25
dummy/src/toga_dummy/widgets/imageview.py             6      1      0      0    83.3%   9
dummy/src/toga_dummy/widgets/label.py                10      1      0      0    90.0%   9
dummy/src/toga_dummy/widgets/navigationview.py        8      3      0      0    62.5%   6, 9, 12
dummy/src/toga_dummy/widgets/numberinput.py          20      2      0      0    90.0%   24, 27
dummy/src/toga_dummy/widgets/optioncontainer.py      41      1      4      1    95.6%   26
dummy/src/toga_dummy/widgets/selection.py            16      1      0      0    93.8%   24
dummy/src/toga_dummy/widgets/slider.py               20      1      0      0    95.0%   21
dummy/src/toga_dummy/widgets/table.py                27      4      0      0    85.2%   12, 15, 18, 21
dummy/src/toga_dummy/widgets/textinput.py            32      3      0      0    90.6%   21, 24, 27
dummy/src/toga_dummy/widgets/timepicker.py           16      1      0      0    93.8%   21
dummy/src/toga_dummy/widgets/tree.py                 21      6      0      0    71.4%   12, 15, 18, 21, 24-25
dummy/src/toga_dummy/window.py                       61      3      2      0    95.2%   30, 77, 81
-----------------------------------------------------------------------------------------------
TOTAL                                              4591    510   1036     57    86.5%

[saroad2]

Even though it does seems like the problem was fixed, there is still that thing that if I want to check the coverage locally I need to go through all these steps each and every time:

1. Go to "core" directory
2. Run tox -e py
3. Move the temp coverage file to root directory
4. Run coverage combine
5. Run coverage report

While before the change all you had to do is to run tox -e py...

Another solution could be to create a script that does all of the above, or to add a new environment in the core "tox.ini" file.

[mhsmith]

The original issue was caused by a problem with the [paths] section in core/pyproject.toml. In #1674 I've removed these sections in the subdirectories, in favor of always running coverage combine in the repository root directory, as CI does.

The second case (under "Additional context"), is Windows-specific. coverage combine outputs all paths using forward slashes because that's the syntax used in the pyproject.toml file. But then when you ran that file through coverage report on Windows, although it didn't give any missing file errors, it showed all lines to be uncovered.

This is probably related to nedbat/coveragepy#991, which was referenced in ci.yml. It appears to be fixed in coverage version 6.6.0b1, so I'll update CI to use that.

there is still that thing that if I want to check the coverage locally I need to go through all these steps each and every time

In #1674 I've added a comment at the top of pyproject.toml which gives a simpler sequence of commands. We still need to run coverage combine because it merges the duplicate paths from source and source_pkgs, as explained further down in the file. If anyone can come up with a way to avoid this while still detecting files which are never imported, I'd be happy to hear about it.