Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit Rayon threadpool threads #5871

Merged
merged 23 commits into from
Sep 12, 2019
Merged

Limit Rayon threadpool threads #5871

merged 23 commits into from
Sep 12, 2019

Conversation

sagar-solana
Copy link
Contributor

@sagar-solana sagar-solana commented Sep 10, 2019 •
edited
Loading

Problem

Rayon issues seem to cause local cluster tests to flake due to a lack of cpu compute

Summary of Changes

Put together a hack that limits the local_cluster threadpools to 1 thread each. Keeping cpu usage in check.

Fixes #5350

@codecov
Copy link

codecov bot commented Sep 11, 2019 •
edited
Loading

Codecov Report

Merging #5871 into master will increase coverage by 1.9%.
The diff coverage is 100%.

@@           Coverage Diff            @@
##           master   #5871     +/-   ##
========================================
+ Coverage    73.6%   75.5%   +1.9%     
========================================
  Files         235     236      +1     
  Lines       43161   42087   -1074     
========================================
+ Hits        31778   31789     +11     
+ Misses      11383   10298   -1085

@sagar-solana
Copy link
Contributor Author

test_repairman_catchup is still flaky. :(

@sagar-solana sagar-solana marked this pull request as ready for review September 11, 2019 20:25
mvines
mvines reviewed Sep 11, 2019
View reviewed changes
ci/test-stable.sh Outdated Show resolved Hide resolved
core/src/blocktree_processor.rs Outdated Show resolved Hide resolved
rayon-threadlimit/src/lib.rs Outdated Show resolved Hide resolved
rayon-threadlimit/src/thread_count.rs Outdated Show resolved Hide resolved
core/Cargo.toml Outdated Show resolved Hide resolved
@sagar-solana sagar-solana changed the title Limit threadpool threads and remove serial_test Limit Rayon threadpool threads Sep 11, 2019
@sagar-solana
Copy link
Contributor Author

Any objections to landing this?

mvines
mvines approved these changes Sep 12, 2019
View reviewed changes
Copy link
Member

@mvines mvines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+ from me at least :)

@sagar-solana sagar-solana merged commit c1d7888 into solana-labs:master Sep 12, 2019
@sagar-solana sagar-solana deleted the hack_thread_pools branch September 12, 2019 18:39
@ryoqun ryoqun mentioned this pull request Aug 12, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mvines mvines mvines approved these changes

@sakridge sakridge Awaiting requested review from sakridge

@pgarg66 pgarg66 Awaiting requested review from pgarg66

@carllin carllin Awaiting requested review from carllin

@ParthDesai ParthDesai Awaiting requested review from ParthDesai

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Re-enable local_cluster tests
2 participants
@sagar-solana @mvines