Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CLI parameter to fix the run count #188

Open
GregoryKimball opened this issue Oct 21, 2024 · 0 comments
Open

Add CLI parameter to fix the run count #188

GregoryKimball opened this issue Oct 21, 2024 · 0 comments

Comments

@GregoryKimball
Copy link

nvbench is a great tool for generating profiles for libcudf. I've found that the --profile option with --run-once was a good starting point, but for many operations we need more than one run to generate clean NVTX regions in the profile. For IO operations the first run populates the OS page cache, and for other functions sometime we see lazy library loading CUDA_MODULE_LOADING=LAZY in the first run.

For this reason I've started to use --timeout 0.1 or similar when profiling nvbenchmarks. This approach works well for generating equally-sized blocks of NVTX regions for further analysis. However, this approach starts to break down if the benchmark times vary too much and it is tricky to align NVTX regions to benchmark conditions when the count of NVTX regions per benchmark varies a lot.

I wish there were a way to specify a particular count of benchmark runs. The CLI parameter could be something like --run-fixed 10. This would make it easy to automate profile analysis, and would enable us to index over the NVTX regions and identify the benchmark conditions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant