Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flakiness #25

Merged
merged 1 commit into from
Dec 18, 2022
Merged

Fix flakiness #25

merged 1 commit into from
Dec 18, 2022

Conversation

blazyy
Copy link
Contributor

@blazyy blazyy commented Dec 12, 2022

A test inside test_globals.py was found to be flaky, i.e. test that both pass and fail despite no changes to the code or the tests itself.

Using the pytest-flakefinder plugin, when the tests were re-run more than once, the test test_debug_default_value was failing. Upon looking at the code, the reason was because the value of debug was not being reset back to False on a subsequent run. To fix this, I just added and extra call which initially sets the value of debug to False.

In general, flaky tests are a pain to fix with regards to locating the root of the actual problem, so it's a good idea to fix them when you detect them. I'm aware that the tests might not be re-run at all, but this change makes the entire module more robust and future-proof so it's just a small fix for you to consider.

To reproduce:

Install pytest-flakefinder by following the instructions here.
At root directory, run python3 -m pytest --flake-finder
After implementing the fix, all tests pass successfully, both with and without the pytest-flakefinder plugin being used.

@ganehag ganehag merged commit 8222487 into ganehag:master Dec 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants