-
Notifications
You must be signed in to change notification settings - Fork 491
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feed iterator enters an infinite loop after a physical partition split has occurred #4326
Comments
This is a known issue with the hierarchical pk feature. There is a temporary fix that should solve the issue for now until a version of the SDK is released with the fix. Rather than having the PK included in the query request options, filtering on top level hierarchical Pks should be done through where clauses. CURRENT QUERY THAT HANGS: var queryDefinition = new QueryDefinition(
$"SELECT * FROM c ");
var partitionKey = new PartitionKeyBuilder().Add(topLevelPkValue).Build();
var requestOptions = new QueryRequestOptions();
requestOptions.PartitionKey = pk;
List<JObject> results = new List<JObject>();
using var iterator = container.GetItemQueryIterator<JObject>(queryDefinition, requestOptions: requestOptions);
while (iterator.HasMoreResults)
{
var response = await iterator.ReadNextAsync();
} FIX: var queryDefinition = new QueryDefinition(
$"SELECT * FROM c "
+ $"WHERE c.topLevelPkValue = @pk1")
.WithParameter("@pk1", topLevelPkValue);
using var iterator = container.GetItemQueryIterator<JObject>(queryDefinition);
while (iterator.HasMoreResults)
{
var response = await iterator.ReadNextAsync();
} |
Would've been lovely to be able to see a known issue such as this one in the known issues list. |
Describe the bug
When I run a targeted multi-partition query with a partition key that includes the first level of subpartitioning, the
FeedIterator<T>.ReadNextAsync
method never completes after a physical partition split has occurred. The iterator enters an infinite loop reading partition key ranges and the application eventually runs out of memory.To Reproduce
I wrote a console application to reproduce the issue. Please download the solution, open the
Program.cs
file and specify a connection string to an Azure Cosmos DB for NoSQL account. The application will create a new database with one container and one physical partition. It will then upsert about 20 GB of data and attempt to read the first 10 documents from two logical partitions. At this point there should be two physical partitions. You can run the application multiple times. It will not upsert data if the container already exists.The data consists of 40,000 documents (articles) with the following properties:
There are two customers. Each customer has 20,000 articles organized in 2,000 sections (10 articles per section).
The container uses a hierarchical partition key
/customerId, /sectionId
, so there are 4,000 hierarchical logical partitions with 10 documents.Finally, the application will execute a query
SELECT * FROM c OFFSET 0 LIMIT 10
, using the customer ID as the partition key in the request options. When a response is returned, it displays the number of documents and the total request charge.Expected behavior
The
FeedIterator<T>.ReadNextAsync
method returns a response with the first 10 documents.Actual behavior
The
FeedIterator<T>.ReadNextAsync
method never completes and the application eventually runs out of memory.Environment summary
SDK Version: 3.38.1 (version 3.37.1 has the same issue)
OS Version: Windows 11 Enterprise (10.0.22631 Build 22631)
Additional context
You can examine the trace and the diagnostic string from another incident. It happens in both
Direct
andGateway
mode.The text was updated successfully, but these errors were encountered: