-
Notifications
You must be signed in to change notification settings - Fork 494
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Internal Pagination: Adds Merge Proofing #2084
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bchong95
requested review from
ealsur,
FabianMeiswinkel,
j82w,
khdang,
kirankumarkolli,
kirillg and
sboshra
as code owners
December 18, 2020 16:55
ealsur
reviewed
Dec 18, 2020
Microsoft.Azure.Cosmos/src/FeedRange/FeedRanges/FeedRangeEpk.cs
Outdated
Show resolved
Hide resolved
Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CrossPartition/PartitionMapper.cs
Show resolved
Hide resolved
Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CrossPartition/PartitionMapper.cs
Show resolved
Hide resolved
Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CrossPartition/PartitionMapper.cs
Outdated
Show resolved
Hide resolved
Microsoft.Azure.Cosmos/src/Pagination/CrossPartitionRangePageAsyncEnumerator.cs
Show resolved
Hide resolved
sboshra
reviewed
Dec 23, 2020
Microsoft.Azure.Cosmos/src/FeedRange/FeedRanges/FeedRangeEpk.cs
Outdated
Show resolved
Hide resolved
sboshra
reviewed
Dec 23, 2020
Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CrossPartition/PartitionMapper.cs
Show resolved
Hide resolved
sboshra
previously approved these changes
Dec 23, 2020
sboshra
approved these changes
Jan 11, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Internal Pagination: Adds Merge Proofing
Description
This PR merge proofs the pagination library which covers the pull models for ReadFeed, ChangeFeed, and Query.
For ReadFeed and ChangeFeed, we don't do anything special to the continuation token on top of what the pagination library handles. These operations were merge proof just from epk range handling.
For Query, we only emit the state of the partition that we resumed from. This makes it difficult to resume the query on a merge, since we have no context on what partitions were there in the past. The algorithm is documented in PartitionMapper.cs, but I will run through an example here:
Suppose we have the following partitions:
[(A, B), (B, C), (E, F), (H, I), (I, L)]
And the following continuation token:
[{"range": {"min": "I", "max": "K"}, "token": "blah"}]
Then we can resume the cross partition state like so:
Extensive testing has been added that tests the cross product of:
{UseContinuationTokens, SplitPartitions, MergePartitions}
. This should cover all the scenarios for splits, merges, and continuation tokens.InMemoryContainer needed to be updated to support merge proofing the token it generates for the SDK.
The follow up work item is adding integration tests for splits and merge to validate that there is no soft contract violations with the NetworkAttachedDocumentContainer class.