-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Convert direct multiprocessing.set_start_method("forkserver") call to a pytest fixture. #4377
chore: Convert direct multiprocessing.set_start_method("forkserver") call to a pytest fixture. #4377
Conversation
Oh, this looks interesting! (I think it's much more than a chore!) Do you already know if that gets rid of the repeat
that popped up under #4376? |
@rwgk I am not sure? This is just functionally the same as it was before, just prevents the code from being run twice by having it be a pytest.fixture . |
All the failures are flakes. Happy to merge pending approval. |
Please give me a moment to try if it helps getting rid of the spurious I believe this PR is good even if it doesn't help, but I want to try first, in case a tweak here is needed for something. TBH I don't really understand this sentence:
Did you mean "called once"? I think I'm missing something fundamental about how fixtures work. Could you please help me (and others) understand? My (maybe incorrect) best-guess understanding is: conftest.py is imported only once, when pytest starts up. There is no direct reference to |
Yep This doc has some really good explanation about the order the fixtures are called and when. a scope='session` is called once when pytest startups and its state is shared among all tests that include it. By adding autouse=True, it is automatically added to all tests affected by conftest.py meaning that it is run regardless of which test is run, and it is only once per session (hence scope="session"). You can also have fixtures run once per module, once per class, etc... to store global state, or to have a piece of code run once before the first instance of the class is created etc... |
Yeah, true, but let's say you just wanted to import code snippits from it outside of a pytest context for debugging, it would still set the multiprocessing state, which may be undesirable. Fixture are also nice because they document how the state of the program is setup for each test. |
Thanks, I need a moment to look. Quick results:
|
Yeah, it should be executed once either way, this just make it more robust and it ensures its' not executed more than once by more edge case things like abusing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please work your latest explanations into the PR description before merging?
Even more importantly: update the title? (because that goes into the commit hash)
I don't think "bugfix" is appropriate.
"best practices" seem more fitting.
But I'd just say what this does in the title and carefully explain in the description.
Suggested title:
Convert direct multiprocessing.set_start_method("forkserver")
call to a pytest fixture.
Nice description! |
Description
multiprocessing
start_method
"forkserver"
#4306. This should be a bit safer since it ensure it can only be called once and if we import anything from conftest for whatever reason, it won't try to run the code snippit again. This is what we should have done at the start as putting any "raw" code in conftest.py is ill advised and bugprone.importlib.reload
or autoreload etc from Jupyter notebooks and better documents what functions are affecting the global state before a test is runSuggested changelog entry: