Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query: Fixes a bug where max page size is not being honored after the first 2 pages #2776

Merged
merged 5 commits into from
Oct 6, 2021

Conversation

sourabh1007
Copy link
Contributor

@sourabh1007 sourabh1007 commented Oct 6, 2021

Description

Query results were not honouring MaxItemCount after 2 pages. This causing the query to be slow and cost a lot of RUs.
As part of this PR fixing the bug.

Analysis:

It was found that we are loosing configured MaxItemCount somewhere in the flow and getting set as Int32 Max i.e. 2,147,483,647 which was leading to above issue.
So in this OrderByQueryPartitionRangePageAsyncEnumerator, We have InnerEnumerator which stores QueryPaginationOptions and there we have private field and one public field to store the same information. In the constructor of this, we were setting only private field and getter was getting value from the public field. So changed the code to set the public variable and remove the private variable.

Why it was affecting only after 2 pages?

For first page it gets the results with correct maxitemcount value and then merge the other partitions (multiple partition to single one e.g (A, B) (B, C) => (A,C)) due to which 2nd page makes a call but throws Split Exception and which goes to this code,

Where we are re-initializing OrderByQueryPartitionRangePageAsyncEnumerator with public variable of uninitializedEnumerator.QueryPaginationOptions which was null all the time. So here we are losing this value.

Type of change

  • [] Bug fix (non-breaking change which fixes an issue)

Closing issues

To automatically close an issue: closes #2756

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors|Removes) Description"

Internal should be used for PRs that have no customer impact. This flag is used to help generate the changelog to know which PRs should be included. Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

@sourabh1007 sourabh1007 changed the title [In development] Query Issue : fixes a bug where max page size is not being honored after the first 2 pages Query Issue : fixes a bug where max page size is not being honored after the first 2 pages Oct 6, 2021
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors|Removes) Description"

Internal should be used for PRs that have no customer impact. This flag is used to help generate the changelog to know which PRs should be included. Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

@j82w j82w changed the title Query Issue : fixes a bug where max page size is not being honored after the first 2 pages Query: Fixes a bug where max page size is not being honored after the first 2 pages Oct 6, 2021
@github-actions github-actions bot dismissed stale reviews from themself October 6, 2021 14:51

All good!

@j82w j82w merged commit 75f57bc into master Oct 6, 2021
@j82w j82w deleted the users/sourabhjain/2756issue branch October 6, 2021 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cross Partition ORDER BY Queries consume large RUs
3 participants