Skip to content

Wrap all of x.py in if __name__ == '__main__': to avoid problems with multiprocessing #111049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented May 1, 2023

#110427 caused bootstrap.py to start using the multiprocessing module, which unfortunately broke the bootstrap script on some systems.

It seems that in some situations, Python's multiprocessing module needs to be able to safely import a program's entry-point module without executing the program.

Fixes #111046.

@rustbot
Copy link
Collaborator

rustbot commented May 1, 2023

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 1, 2023
@Zalathar
Copy link
Contributor Author

Zalathar commented May 1, 2023

The easier fix would be to just revert #110427, but filing this PR was simple enough that I figured I'd give the option of doing this instead.

@albertlarsan68
Copy link
Member

If you could revert the PR in addition of your changes, it would be great.

@jyn514
Copy link
Member

jyn514 commented May 1, 2023

Why? This fixes the bug, doesn't it? I don't think we gain anything by reverting

@albertlarsan68
Copy link
Member

albertlarsan68 commented May 1, 2023

I (at least) would like to join the pool before continuing and use the os.get_schedaffinity() function instead of the number of CPUs.
If this could be done in this PR, then no need to revert.
The first is especially important, to avoid trying to use the files before they are extracted (may happen with slower CPUs).

EDIT: Per the multiprocessing documentation, the function to use would be len(os.sched_getaffinity(0))

@Zalathar
Copy link
Contributor Author

Zalathar commented May 1, 2023

I'm not interested in fighting over this.

@Zalathar Zalathar closed this May 1, 2023
@Zalathar Zalathar deleted the x-py branch May 31, 2023 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failed to run x
5 participants