Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query: MaxBufferedItemCount is not being honored #877

Open
sourabh1007 opened this issue Oct 6, 2021 · 1 comment
Open

Query: MaxBufferedItemCount is not being honored #877

sourabh1007 opened this issue Oct 6, 2021 · 1 comment
Assignees

Comments

@sourabh1007
Copy link
Contributor

The MaxBufferedItemCount is not being honored. If you set the MaxBufferedItemCount to 0 if you look at fiddler the SDK will still drain the entire query in the background. The problem with this is anyone doing the paging model will be paying to drain the entire query even though it is only using the first page.

Here is an example that reproduces the issue.

using DocumentClient client = new DocumentClient(
                new Uri(https://cosmosdbdotnetperf.documents.azure.com:443/),
                "",
                new ConnectionPolicy(){
                    ConnectionMode = ConnectionMode.Gateway,
                    ConnectionProtocol = Protocol.Https,
                },
                ConsistencyLevel.Eventual);
            
            string dbName = "testdb";
            string containerName = "tempdb";

            Uri collectionLink = UriFactory.CreateDocumentCollectionUri(dbName, containerName);
            for(int i = 0; i < 20; i++)
            {
                await client.CreateDocumentAsync(collectionLink, new { id = Guid.NewGuid().ToString(), pk = "Test" + Guid.NewGuid().ToString(), random = "Test"  });
            }

            var query = client.CreateDocumentQuery(collectionLink,
                "select * from T where T.random = 'Test'",
                new FeedOptions()
                {
                    MaxItemCount = 5,
                    MaxDegreeOfParallelism = 1,
                    MaxBufferedItemCount = 0,
                    EnableCrossPartitionQuery = true,
                }).AsDocumentQuery();

            while (query.HasMoreResults)
            {
                var result = await query.ExecuteNextAsync();
                string s = result.SessionToken;
                Console.WriteLine(s);
            }

@sourabh1007 sourabh1007 self-assigned this Oct 6, 2021
@sourabh1007 sourabh1007 changed the title MaxBufferedItemCount is not being honored Query: MaxBufferedItemCount is not being honored Oct 6, 2021
@sourabh1007
Copy link
Contributor Author

sourabh1007 commented Oct 18, 2021

MaxBufferedItemCount gets honoured only when we have cross partition query (or parallel execution) So basically it is applicable for below query contexts:

  1. CrossPartitionQueryExecutionContext (line number 178)
  2. ParallelDocumentQueryExecutionContext
  3. OrderByDocumentQueryExecutionContext

Along with this, below code decides if it is cross partition query : https://msdata.visualstudio.com/CosmosDB/_git/CosmosDB?path=/Product/SDK/.net/Microsoft.Azure.Documents.Client/Query/DocumentQueryExecutionContextFactory.cs&version=GBmaster&line=253&lineEnd=279&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

Here, we are checking of it is set as 0 then set it as DefaultMaximumBufferSize : https://msdata.visualstudio.com/CosmosDB/_git/CosmosDB?path=/Product/SDK/.net/Microsoft.Azure.Documents.Client/Query/CrossPartitionQueryExecutionContext.cs&version=GBmaster&line=178&lineEnd=178&lineStartColumn=13&lineEndColumn=70&lineStyle=plain&_a=contents

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants