Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: add MVCCIterKind parameter to Reader functions #56022

Merged
merged 1 commit into from
Oct 28, 2020

Conversation

sumeerbhola
Copy link
Collaborator

Also changes all callers to try to appropriately set these parameters.
Note that MVCCKeyAndIntentsIterKind would always be correct, but
where possible, we want to set MVCCKeyIterKind.

Release note: None

@sumeerbhola sumeerbhola requested a review from a team as a code owner October 27, 2020 19:05
@sumeerbhola sumeerbhola requested review from miretskiy and removed request for a team October 27, 2020 19:05
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@sumeerbhola sumeerbhola removed the request for review from miretskiy October 27, 2020 19:05
@sumeerbhola sumeerbhola force-pushed the reader_refactor2 branch 2 times, most recently from bdee57d to f5e06c3 Compare October 27, 2020 19:57
Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 17 of 17 files at r1, 44 of 44 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal, @petermattis, and @sumeerbhola)


pkg/kv/kvserver/replica_consistency.go, line 578 at r2 (raw file):

	// Iterate over all the data in the range.
	iter := snap.NewMVCCIterator(storage.MVCCKeyAndIntentsIterKind, storage.IterOptions{UpperBound: desc.EndKey.AsRawKey()})

This raises an interesting question. At first, I thought that this was incorrect because we're already walking over each part of the range in down below (see call to MakeReplicatedKeyRanges). But I think the call to ComputeStatsGo does want to see the intents merged into the MVCC keyspace. So should we not call ComputeStatsGo on the two lock-table key spans?


pkg/kv/kvserver/replica_evaluate.go, line 92 at r2 (raw file):

		return origReqs
	}
	iter := reader.NewMVCCIterator(storage.MVCCKeyAndIntentsIterKind, storage.IterOptions{

I think we can skip intents here. We're just looking for anything in the key range.


pkg/storage/engine.go, line 290 at r2 (raw file):

	// engine. The caller must invoke MVCCIterator.Close() when finished
	// with the iterator to free resources.
	NewMVCCIterator(iterKind MVCCIterKind, opts IterOptions) MVCCIterator

Is it safe to assume that you didn't want to put MVCCIterKind into IterOptions because you wanted to force each caller to specify it?


pkg/storage/mvcc.go, line 1287 at r2 (raw file):

	txn *roachpb.Transaction,
) error {
	iter := newMVCCIterator(rw, timestamp == (hlc.Timestamp{}), IterOptions{Prefix: true})

hlc.Timestamp does have an IsEmpty method if you want to use that.

Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @itsbilal, @nvanbenschoten, and @petermattis)


pkg/kv/kvserver/replica_consistency.go, line 578 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

This raises an interesting question. At first, I thought that this was incorrect because we're already walking over each part of the range in down below (see call to MakeReplicatedKeyRanges). But I think the call to ComputeStatsGo does want to see the intents merged into the MVCC keyspace. So should we not call ComputeStatsGo on the two lock-table key spans?

Glad you noticed that. I've added a TODO, since MakeReplicatedKeyRanges does not currently include any of the lock table ranges.


pkg/kv/kvserver/replica_evaluate.go, line 92 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

I think we can skip intents here. We're just looking for anything in the key range.

Done


pkg/storage/engine.go, line 290 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Is it safe to assume that you didn't want to put MVCCIterKind into IterOptions because you wanted to force each caller to specify it?

Exactly. It is less likely to cause bugs now or in the future.


pkg/storage/mvcc.go, line 1287 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

hlc.Timestamp does have an IsEmpty method if you want to use that.

Done

Also changes all callers to appropriately set these parameters.
Note that MVCCKeyAndIntentsIterKind would always be correct, but
where possible, we want to set MVCCKeyIterKind.

Release note: None
@sumeerbhola
Copy link
Collaborator Author

bors r+

@craig
Copy link
Contributor

craig bot commented Oct 28, 2020

Build succeeded:

@craig craig bot merged commit 70fb9a5 into cockroachdb:master Oct 28, 2020
@sumeerbhola sumeerbhola deleted the reader_refactor2 branch November 2, 2020 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants