DIS: Cython 3 perf regressions

[lithomas1]

It looks like scikit-learn hasn't updated to Cython 3 yet, but when you do, can you keep an eye out for performance regressions?

On pandas, we updated to Cython 3 but then reverted, since we had issues with some of our benchmarks regressing, and it would be nice to know if this is a bigger problem in Cython.

[glemaitre]

Thanks. We should be able to observe the regression in the continuous benchmark: https://scikit-learn.org/scikit-learn-benchmarks/

[glemaitre]

Actually we don't have any results uploaded since July 12. Maybe @jeremiedbb can help :)

[ogrisel]

Also we don't use Cython 3 in the main branch yet as far as I know. We only test Cython 3 in the scipy-dev build which is not benchmarked.

[glemaitre]

We updated recently the lock file (during EuroSciPy sprint): https://github.com/scikit-learn/scikit-learn/pull/27024/files and thus we are now using Cython 3.0.

We could have a wild check to see if we took an overall slowdown on all test on the CI.

I checked and there is too much variation to resort to the test suite.

[ogrisel]

I missed that. Thanks for the heads up.

[jjerphan]

Actually since there's no upper version pin for Cython, we might have been using Cython 3 since its release.

[lithomas1]

Hm,
I just looked at your asvs and you don't seem to distinguish between Cython 0.29 and Cython 3.0 in yours

See ours for reference https://asv-runner.github.io/asv-collection/pandas/#frame_ctor.FromDicts.time_nested_dict
(In the left sidebar, you can choose between Cython 0.29 and Cython 3.0 while your benchmarks only seem to filter by python and pandas version)

[jjerphan]

I think we need to pin Cython>=0.29.33<3 except for nightly builds and regenerate locks like before we make sure there's no performance regression with Cython 3.

What do you think? I can't look into that now, but could someone else do? 🙏

[glemaitre]

@jeremiedbb is actually looking at the ASV issue and will run before and after #27024 that changed the lock files introducing Cython 3.0