Unpin pytest

[rominf]

Problem Statement

As a Fedora packager I would like to have pytest to not have pin < 7.0.0, so that python-sentry-sdk could be tested in Fedora without workarounds. Also, pytest==7.0.0 was released on Feb 4, 2022.

All actual Fedora releases contain pytest>7.0.0 and installing an older version from PyPI during package build for testing is not possible. Thus, I unpinned it. Since some tests fail with pytest>7.0.0 because of the issue in pytest-forked, I had to find a way how to fix them. Luckily, thanks to this investigation there is a workaround: reorder the tests so that non-forked tests are not last in the file, see patch.

Solution Brainstorm

Of course, the best solution would be to fix the issue in pytest-forked, however it looks like a hard problem.

My suggestions:

• Unpin pytest.
• Reorder the tests.
• Write a note in CONTRIBUTING.md about this quirk. Another option is to add a linter that would inform about the need to reorder tests.

[szokeasaurusrex]

@rominf Thank you for opening this issue, and for linking the investigation! We will place this issue on our internal backlog. However, you are also welcome to open a PR, if you would like – we might be able to implement your suggested changes more quickly that way.

[anthonyryan1]

Gentoo is going to remove support for sentry-python on November 17th if we don't have a release with pytest 8 support before that date.

Reference: https://bugs.gentoo.org/937896

[szokeasaurusrex]

@anthonyryan1 thanks for the heads up, we will try to see whether we can get rid of the pin.

[szokeasaurusrex]

@anthonyryan1, looks like the pin might not be needed on Python 3.8+. Would Gentoo be able to continue shipping the Python SDK if we remove the pin for Python 3.8+, but keep it around for the older Python versions we still support (3.6 and 3.7)?

[anthonyryan1]

Gentoo appears to already be supporting a minimum of Python 3.11 for this package, so there's no change to minimum python version it can be used together with.

[szokeasaurusrex]

The pytest pins across most of our test suite have now been removed. This work was implemented in the following PRs:

• build: Remove pytest pin in requirements-devenv.txt #3696
• test(tox): Unpin pytest on Python 3.8+ gevent tests #3700
• test(tox): Unpin pytest for celery tests #3701
• test(tornado): Unpin pytest for tornado tests #3714

Some pins, however, still remain, because it was not possible to remove these pins with trivial changes to the test code:

• py{3.6,3.7}-{common,gevent} still pin pytest<7.0.0. This might be okay, as these are older Python versions.
• Our Redis tests pin pytest<8.0.0. We should investigate why our Redis tests fail with Pytest ≥8.0.0 and implement a fix, if possible.
  • Unpin Pytest on Redis tests #3715
• Our tornado-v{6.0,6.2} tests pin pytest<8.2. This because these versions of tornado are incompatible with Pytest ≥8.2; tornado v6.4.1 fixes the problem. Our tornado-latest tests therefore do not pin pytest.

This work is likely sufficient to close this issue. I will open a new issue to address the Redis pin.