set COVERAGE_CORE=sysmon to improve tests' performance in 3.12

[skshetry]

Closes #906.

Running tests without the --cov option is significantly faster on Python 3.12. Similarly, the tests run quite fast on Python 3.11 with or without --cov.

It turns out that this slowdown is caused by pytest-cov, which happens due to a regression in CPython 3.12, see:

• sys.settrace dramatic slowdown in 3.12 python/cpython#107674
• Poor performance in Python 3.12 nedbat/coveragepy#1665

It took me a lot of time to figure out why, because this did not show up on profiling, just that everything was 3x - 4x slower. And profilers do tend to add significant overhead, so I ignored it initially.

With pytest-cov, the specific test is already running quite close to ~5s timeout, that the newer dependencies that datamodel-code-generator brought were enough to increase that to >5s, and started timing out. (For comparison, in Python 3.12, the test without --cov completes in ~1s, whereas in 3.11 with --cov, it completes in <1.5s.)

If I had led with why the test was only failing on 3.12, I might have figured this out quickly, rather than chasing dependencies, xdist and multiprocessing issues.

There is an experimental implementation inside coverage that should be faster, which can be set through COVERAGE_CORE=sysmon envvar, which will hopefully fix this issue. If not, I'll increase the timeout if necessary.

It looks like this change saves more than a minute of runtime on 3.12, and is now the fastest python version among what we test.

Before

After

Also worth mentioning that we do run coverage with branch=true, and it seems this could end slowing things down, but I haven't noticed that at the moment.

• COVERAGE_CORE=sysmon with branch coverage can be 2x slower than default in some cases nedbat/coveragepy#1812

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.66%. Comparing base (e523ccd) to head (16f98f1).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #920   +/-   ##
=======================================
  Coverage   87.66%   87.66%           
=======================================
  Files         130      130           
  Lines       11698    11698           
  Branches     1592     1592           
=======================================
  Hits        10255    10255           
  Misses       1043     1043           
  Partials      400      400           

Flag	Coverage Δ
datachain	87.58% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.