storage: introduce guaranteed durability functionality #76793
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
This change is |
sumeerbhola
commented
Feb 18, 2022
Collaborator
Author
sumeerbhola
left a comment
sumeerbhola
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained
pkg/storage/pebble.go, line 901 at r1 (raw file):
atomic.AddInt64(&p.diskSlowCount, 1) }, FlushEnd: func(info pebble.FlushInfo) {
Note that this does not use the Pebble code introduced in cockroachdb/pebble#1505, and instead uses EventListener as suggested by Peter cockroachdb/pebble#1505 (review)
sumeerbhola
force-pushed
the
eng_durable
branch
from
4d79d8a to
66de60a
Compare
jbowens
approved these changes
Feb 22, 2022
Collaborator
jbowens
left a comment
jbowens
left a comment
There was a problem hiding this comment.
Reviewed 5 of 14 files at r1, 1 of 2 files at r2, all commit messages.
Reviewable status:
Collaborator
Author
|
TFTR! |
This is the CockroachDB plumbing for Pebble's IterOptions.OnlyReadGuaranteedDurable. It is for use in the raftLogTruncator cockroachdb#76215. Since most of the exported interfaces in the storage package use a Reader, we support this via a DurabilityRequirement parameter on Engine.NewReadOnly, and not via an iterator option. There is also a RegisterFlushCompletedCallback method on Engine which will be used to poll certain durable state in the raftLogTruncator. Other than the trivial plumbing, this required some refactoring of the Reader.MVCCGet* code for Pebble and pebbleReadOnly. Even though it is deprecated and primarily/only used in tests, we don't want to have the durability semantics diverge. Release note: None
sumeerbhola
force-pushed
the
eng_durable
branch
from
66de60a to
ac7e0b6
Compare
Collaborator
Author
|
TestTenantLogic timeout must be a flake since nothing other than tests uses this new functionality. |
Collaborator
Author
|
bors r=jbowens |
Contributor
|
Build failed (retrying...): |
Contributor
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the CockroachDB plumbing for Pebble's
IterOptions.OnlyReadGuaranteedDurable. It is for use in
the raftLogTruncator #76215.
Since most of the exported interfaces in the storage
package use a Reader, we support this via a
DurabilityRequirement parameter on Engine.NewReadOnly,
and not via an iterator option.
There is also a RegisterFlushCompletedCallback method
on Engine which will be used to poll certain durable
state in the raftLogTruncator.
Other than the trivial plumbing, this required some
refactoring of the Reader.MVCCGet* code for Pebble
and pebbleReadOnly. Even though it is deprecated and
primarily/only used in tests, we don't want to have
the durability semantics diverge.
Release note: None