Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit available parallelism to 1 per 2GB of RAM #725

Open
riking opened this issue Mar 27, 2024 · 1 comment
Open

Limit available parallelism to 1 per 2GB of RAM #725

riking opened this issue Mar 27, 2024 · 1 comment

Comments

@riking
Copy link

riking commented Mar 27, 2024

This is a proposed solution to the OOM class of issues mentioned in #589 , in particular the pattern where a large number of integration tests are launched in parallel.

Common CPU-to-RAM ratios for cloud provider VMs are 1:2GB, 1:4GB, and 1:8GB. Because there is currently no parallelism limit imposed, use the most generous ratio for the initial implementation.

Proposed Implementation:

src/runner/test.rs is updated to calculate the total memory available to the nearest limiting control group or whole machine, divide this number by 2 gigabytes and provide the result to parallelism-limiting mechanisms.

Each relevant cargo command accepts -jNUM, but cargo test could also specify --test-threads=NUM for the default test runner to limit its parallelism as well.

@riking
Copy link
Author

riking commented Mar 27, 2024

actually, this should only apply to the test runner phase, right? We still want maximum parallel during the build of the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant