Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks have errors / problems... #242

Open
JustinCappos opened this issue May 11, 2024 · 0 comments
Open

Benchmarks have errors / problems... #242

JustinCappos opened this issue May 11, 2024 · 0 comments

Comments

@JustinCappos
Copy link
Member

When examining the benchmarks for rustposix, I noticed a few odd measurements that led me to find a few bugs in the benchmark scripts.

The overall problem is that return values were not being adequately validated. (Naively, I thought this was a good idea to limit the overhead of this checking in the benchmark.) This was masking bugs.

A few examples:

In many benchmarks, the file was opened write-only. This means that reads were faster than writes, with Lind far outperforming the kernel because it would error more quickly.

In the (common) case where writes are slower than reads, this means that a benchmark that writes into a file and then reads from it, can easily read to the end of the file and keep requesting bytes. This means iterations at the end will just measure the size of a 0 byte read, regardless of the actual read buffer size.

So, as a result, I will fix those issues in PRs and also update the style guidelines to require assertions in almost all cases so that these sorts of errors can be minimized in the future.

JustinCappos added a commit that referenced this issue May 11, 2024
@JustinCappos JustinCappos mentioned this issue May 11, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant