Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pinned pytest version to 6.2.1 and fix error in test_exceptions.py::test_python_alreadyset_in_destructor #2741

Merged
merged 5 commits into from
Dec 24, 2020

Conversation

YannickJadoul
Copy link
Collaborator

@YannickJadoul YannickJadoul commented Dec 23, 2020

Description

As noted in #2573 (comment), pytest 6 seems to make one of our tests fail. Let's check this, and then figure out if/how to fix that.

Suggested changelog entry:

* Fix issue with a test failing on PyTest 6.2.

@YannickJadoul YannickJadoul marked this pull request as draft December 23, 2020 23:10
@henryiii
Copy link
Collaborator

I think we just need to add a marker to turn this extra pytest feature off for this test. We do want this behavior.

I noticed the dropping of 3.5 in the changelog.

@YannickJadoul
Copy link
Collaborator Author

YannickJadoul commented Dec 23, 2020

It would be nice if we could explicitly turn it off for just that test, though, and not for everything, with the blunt -p no:unraisableexception flags?

@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning") ignores the warning, but still fails our test. Presumably because pytest holds sys.unraisablehook hostage, and doesn't propagate the behavior to the default implemenation, on which we depend in the test.

@bstaletic
Copy link
Collaborator

Can we use the monkeypatch fixture to force our hook and then restore pytest's hook?

@YannickJadoul
Copy link
Collaborator Author

Got something. The original built-in sys.unraisablehook is kept as sys.__unraisablehook__, so we can just use that?

@YannickJadoul YannickJadoul changed the title Test (and hopefully soon fix) test failure with pytest 6 Update pinned pytest version to 6 and fix error in test_exceptions.py::test_python_alreadyset_in_destructor Dec 23, 2020
@YannickJadoul YannickJadoul marked this pull request as ready for review December 23, 2020 23:49
@YannickJadoul YannickJadoul changed the title Update pinned pytest version to 6 and fix error in test_exceptions.py::test_python_alreadyset_in_destructor Update pinned pytest version to 6.2.1 and fix error in test_exceptions.py::test_python_alreadyset_in_destructor Dec 23, 2020
@henryiii
Copy link
Collaborator

henryiii commented Dec 23, 2020

Why make it a string but still error out if unfound, PyTest... needs an if to see if that exception exists.

"ignore::pytest.PytestUnraisableExceptionWarning"
)
else:
ignore_pytest_unraisable_warning = lambda f: f # NOQA: E731
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I could not put a lambda, and put 2 lines def ignore_pytest_unraisable_warning(f): and return f, but then black suddenly adds an empty line after else:, so that's just as ugly.

@YannickJadoul
Copy link
Collaborator Author

Why make it a string but still error out if unfound, PyTest... needs an if to see if that exception exists.

Yeah, indeed ... :-/ Guess they're just following the warnings module?

@@ -50,13 +50,23 @@ def test_python_call_in_catch():
assert d["good"] is True


if hasattr(pytest, "PytestUnraisableExceptionWarning"): # Python >= 3.8 and pytest >= 6
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we move this into the function, and directly use the warnings module. But I'm not sure whether such a filter would get reset after the test or not, this way.

@YannickJadoul
Copy link
Collaborator Author

Green! :-) Ready to be reviewed.

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like I said "a decorator factory" could be a fancy way of avoiding code in global scope, but not a blocker.

tests/test_exceptions.py Outdated Show resolved Hide resolved
def test_python_alreadyset_in_destructor(monkeypatch, capsys):
hooked = False
triggered = [False] # mutable, so Python 2.7 closure can modify it

if hasattr(sys, "unraisablehook"): # Python 3.8+
hooked = True
default_hook = sys.unraisablehook
# Don't take `sys.unraisablehook`, as that's overwritten by pytest
default_hook = sys.__unraisablehook__
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we're happy with this?

If some case where this would be overwritten (by something else than pytest), we're ignoring it as well, ofc, but ... myeah, best I could immediately think of.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems fine to me. We don't want pytest's version here, and it's a test, so nothing else should be fiddling with it.

@YannickJadoul
Copy link
Collaborator Author

Does this need to be marked 2.6.2, still? I guess it's kind of independent from the releases, but ...

Guess it's mostly #2573 that needs to make sure it also installs tests/requirements.txt, as I noted there.

@henryiii henryiii added this to the v2.6.2 milestone Dec 24, 2020
@henryiii henryiii merged commit 830f8ed into pybind:master Dec 24, 2020
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label Dec 24, 2020
@henryiii
Copy link
Collaborator

Great! We don't really control the PyTest version if someone wants to run our tests (a requirements.txt is an internal pinning, not an external one), and a few of our tests use system PyTests, but it's good not to be (too) version specific, tests failing due to PyTest is not ideal.

@YannickJadoul
Copy link
Collaborator Author

Do you want a changelog for this, @henryiii?

@henryiii
Copy link
Collaborator

Yes, I added it to the description.

Here it is in setuptools changelog, for example: https://setuptools.readthedocs.io/en/latest/history.html#misc

@YannickJadoul
Copy link
Collaborator Author

Yes, I added it to the description.

Great! Hadn't spotted that; thanks! :-)

@henryiii henryiii removed the needs changelog Possibly needs a changelog entry label Dec 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants