crl-release-20.1: vfs: Add vfs.Prefetch, use it for reading ahead data blocks #733
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For sequential-like IO workload where we read data blocks one after the other in quick succession, signalling the OS to asynchronously bring them to cache in advance can deliver significant savings in IOPS dispatched. In IOPS-bound workloads such as backup on an EBS disk, this delivers a 3x speedup. Presumably aggregate queries and compactions will be faster as well, though this hasn't been benchmarked in practice yet. This change maintains a counter for the number of data block reads performed in a singleLevelIterator, and when that count exceeds 2, a readahead system call is made on Linux. RocksDB has almost the exact same behaviour, including the same min/max readahead sizes and read count thresholds. Will address cockroachdb/cockroach#49710 when it lands in cockraoch.
|
This change is |
petermattis
approved these changes
Jun 10, 2020
|
Ah right. I usually do that, but I forgot this time and ended up confusing myself as well. Thanks for the reminder! |
itsbilal
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For sequential-like IO workload where we read
data blocks one after the other in quick succession,
signalling the OS to asynchronously bring them to
cache in advance can deliver significant savings in IOPS
dispatched. In IOPS-bound workloads such as backup on
an EBS disk, this delivers a 3x speedup. Presumably
aggregate queries and compactions will be faster
as well, though this hasn't been benchmarked in
practice yet.
This change maintains a counter for the number of data
block reads performed in a singleLevelIterator, and
when that count exceeds 2, a readahead system
call is made on Linux. RocksDB has almost
the exact same behaviour, including the same
min/max readahead sizes and read count thresholds.
Will address cockroachdb/cockroach#49710
when it lands in cockraoch.