Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make accel "auto flush" threshold configurable #1231

Merged
merged 2 commits into from
May 13, 2024

Conversation

amandalund
Copy link
Contributor

Currently in our local transporter we offload tracks to Celeritas after num_track_slots have been buffered; however, it may be beneficial in some cases to buffer more than the number of available track slots before starting the stepping loop. This change lets the user specify the "auto flush" threshold (it will still default to the number of track slots). I haven't tested many configurations, but for example I saw a 15-25% performance improvement for the CMS run 3, 32 ttbar event problem on an A100 by decreasing the number of track slots from 2^19 to 2^16 but increasing the buffer size to 2^20:
speedup-a100

@amandalund amandalund added app Application front ends performance Changes for performance optimization labels May 11, 2024
@amandalund amandalund requested review from sethrj and esseivaju May 11, 2024 15:20
@sethrj sethrj added the enhancement New feature or request label May 11, 2024
Copy link
Member

@sethrj sethrj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great improvement! Hopefully the performance with number of track slots will scale better with an improved geometry engine.

app/celer-g4/RunInput.hh Outdated Show resolved Hide resolved
src/accel/SetupOptions.hh Outdated Show resolved Hide resolved
src/accel/LocalTransporter.cc Outdated Show resolved Hide resolved
@sethrj sethrj merged commit bd11e58 into celeritas-project:develop May 13, 2024
28 checks passed
@amandalund amandalund deleted the auto-flush branch May 13, 2024 12:16
@esseivaju
Copy link
Contributor

@amandalund If you buffer more primaries than we have track slots, where is that overflow handled?

@amandalund
Copy link
Contributor Author

The primaries are converted to track initializers and stored in that buffer while they are waiting to be transported, so as long as the "auto flush" threshold is less than the initializer capacity there shouldn't be any overflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app Application front ends enhancement New feature or request performance Changes for performance optimization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants