-
Notifications
You must be signed in to change notification settings - Fork 908
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add validation to reader benchmarks #14137
Conversation
…fea-benchmark-validation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The implementation seems sensible, but why are we doing this in benchmarks rather than tests?
To verify that the benchmarks are doing what they're supposed to. Also, it's helpful in situations where we benchmark different configurations (thread pool size, GDS policy, compression policy...). Basically enabling us to benchmark changes without jumping back to tests for each change. |
Description
Validate that cuIO readers successfully round trip the input tables.
Validation is only done in the first iteration, and it is not included in the timing.
If there is a difference, a warning is logged.
Setting
CUDF_BENCH_OUTPUT_DIFF
environment variable adds diff to the standard output. Valid values areFIRST_ERROR
andALL_ERRORS
.Benchmarks currently report differences for data types that can't be preserved through the given formats. There are not differences caused by data corruption.
Checklist