-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement microbenchmark for FileCache #6610
Implement microbenchmark for FileCache #6610
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## main #6610 +/- ##
============================================
- Coverage 70.79% 70.72% -0.08%
+ Complexity 59121 59107 -14
============================================
Files 4804 4805 +1
Lines 283141 283173 +32
Branches 40815 40816 +1
============================================
- Hits 200462 200268 -194
- Misses 66229 66492 +263
+ Partials 16450 16413 -37
... and 499 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
|
||
@Benchmark | ||
public void get(CacheParameters parameters) { | ||
parameters.fileCache.get(randomKeyInCache(parameters)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious if the benchmark should use blackhole or return the value, the call could be eliminated since the returned value is not used, thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, good call.
My guess is the call would not be eliminated because the JVM doesn't know if there are no side effects to invoking the get()
method. I'm certainly no expert here but as a general practice I think the return values should be blackholed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, that should make it:
public Object get(CacheParameters parameters) {
return parameters.fileCache.get(randomKeyInCache(parameters));
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went with:
@Benchmark
public void get(CacheParameters parameters, Blackhole blackhole) {
blackhole.consume(parameters.fileCache.get(randomKeyInCache(parameters)));
}
Is that not the right way to use the blackhole here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My apologies @andrross, probably the comment went into outdated code version, both approaches are correct, thank you.
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements. Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM): ``` Benchmark (concurrencyLevel) (maximumNumberOfEntries) Mode Cnt Score Error Units FileCacheBenchmark.get 1 65536 thrpt 2243.092 ops/ms FileCacheBenchmark.get 1 1048576 thrpt 950.818 ops/ms FileCacheBenchmark.get 8 65536 thrpt 5651.150 ops/ms FileCacheBenchmark.get 8 1048576 thrpt 2831.012 ops/ms FileCacheBenchmark.put 1 65536 thrpt 2206.027 ops/ms FileCacheBenchmark.put 1 1048576 thrpt 921.248 ops/ms FileCacheBenchmark.put 8 65536 thrpt 4421.122 ops/ms FileCacheBenchmark.put 8 1048576 thrpt 2624.550 ops/ms FileCacheBenchmark.remove 1 65536 thrpt 12387.999 ops/ms FileCacheBenchmark.remove 1 1048576 thrpt 6324.643 ops/ms FileCacheBenchmark.remove 8 65536 thrpt 22161.031 ops/ms FileCacheBenchmark.remove 8 1048576 thrpt 14826.586 ops/ms FileCacheBenchmark.replace 1 65536 thrpt 2146.572 ops/ms FileCacheBenchmark.replace 1 1048576 thrpt 947.612 ops/ms FileCacheBenchmark.replace 8 65536 thrpt 4405.339 ops/ms FileCacheBenchmark.replace 8 1048576 thrpt 2707.204 ops/ms ``` Signed-off-by: Andrew Ross <andrross@amazon.com>
2e4380a
to
1d9aa41
Compare
Gradle Check (Jenkins) Run Completed with:
|
Whitesource fix: #6629 |
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements. Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM): ``` Benchmark (concurrencyLevel) (maximumNumberOfEntries) Mode Cnt Score Error Units FileCacheBenchmark.get 1 65536 thrpt 2243.092 ops/ms FileCacheBenchmark.get 1 1048576 thrpt 950.818 ops/ms FileCacheBenchmark.get 8 65536 thrpt 5651.150 ops/ms FileCacheBenchmark.get 8 1048576 thrpt 2831.012 ops/ms FileCacheBenchmark.put 1 65536 thrpt 2206.027 ops/ms FileCacheBenchmark.put 1 1048576 thrpt 921.248 ops/ms FileCacheBenchmark.put 8 65536 thrpt 4421.122 ops/ms FileCacheBenchmark.put 8 1048576 thrpt 2624.550 ops/ms FileCacheBenchmark.remove 1 65536 thrpt 12387.999 ops/ms FileCacheBenchmark.remove 1 1048576 thrpt 6324.643 ops/ms FileCacheBenchmark.remove 8 65536 thrpt 22161.031 ops/ms FileCacheBenchmark.remove 8 1048576 thrpt 14826.586 ops/ms FileCacheBenchmark.replace 1 65536 thrpt 2146.572 ops/ms FileCacheBenchmark.replace 1 1048576 thrpt 947.612 ops/ms FileCacheBenchmark.replace 8 65536 thrpt 4405.339 ops/ms FileCacheBenchmark.replace 8 1048576 thrpt 2707.204 ops/ms ``` Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 5eed61e) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements. Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM): ``` Benchmark (concurrencyLevel) (maximumNumberOfEntries) Mode Cnt Score Error Units FileCacheBenchmark.get 1 65536 thrpt 2243.092 ops/ms FileCacheBenchmark.get 1 1048576 thrpt 950.818 ops/ms FileCacheBenchmark.get 8 65536 thrpt 5651.150 ops/ms FileCacheBenchmark.get 8 1048576 thrpt 2831.012 ops/ms FileCacheBenchmark.put 1 65536 thrpt 2206.027 ops/ms FileCacheBenchmark.put 1 1048576 thrpt 921.248 ops/ms FileCacheBenchmark.put 8 65536 thrpt 4421.122 ops/ms FileCacheBenchmark.put 8 1048576 thrpt 2624.550 ops/ms FileCacheBenchmark.remove 1 65536 thrpt 12387.999 ops/ms FileCacheBenchmark.remove 1 1048576 thrpt 6324.643 ops/ms FileCacheBenchmark.remove 8 65536 thrpt 22161.031 ops/ms FileCacheBenchmark.remove 8 1048576 thrpt 14826.586 ops/ms FileCacheBenchmark.replace 1 65536 thrpt 2146.572 ops/ms FileCacheBenchmark.replace 1 1048576 thrpt 947.612 ops/ms FileCacheBenchmark.replace 8 65536 thrpt 4405.339 ops/ms FileCacheBenchmark.replace 8 1048576 thrpt 2707.204 ops/ms ``` (cherry picked from commit 5eed61e) Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements. Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM): ``` Benchmark (concurrencyLevel) (maximumNumberOfEntries) Mode Cnt Score Error Units FileCacheBenchmark.get 1 65536 thrpt 2243.092 ops/ms FileCacheBenchmark.get 1 1048576 thrpt 950.818 ops/ms FileCacheBenchmark.get 8 65536 thrpt 5651.150 ops/ms FileCacheBenchmark.get 8 1048576 thrpt 2831.012 ops/ms FileCacheBenchmark.put 1 65536 thrpt 2206.027 ops/ms FileCacheBenchmark.put 1 1048576 thrpt 921.248 ops/ms FileCacheBenchmark.put 8 65536 thrpt 4421.122 ops/ms FileCacheBenchmark.put 8 1048576 thrpt 2624.550 ops/ms FileCacheBenchmark.remove 1 65536 thrpt 12387.999 ops/ms FileCacheBenchmark.remove 1 1048576 thrpt 6324.643 ops/ms FileCacheBenchmark.remove 8 65536 thrpt 22161.031 ops/ms FileCacheBenchmark.remove 8 1048576 thrpt 14826.586 ops/ms FileCacheBenchmark.replace 1 65536 thrpt 2146.572 ops/ms FileCacheBenchmark.replace 1 1048576 thrpt 947.612 ops/ms FileCacheBenchmark.replace 8 65536 thrpt 4405.339 ops/ms FileCacheBenchmark.replace 8 1048576 thrpt 2707.204 ops/ms ``` Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Valentin Mitrofanov <mitrofmep@gmail.com>
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements.
Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM):
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.