Change Feed Processor - "Quick" poll after ready current batch of changes?

[johngallardo]

We have a near-real time system that is trying to synchronize changes from one CosmosDB container to another. From looking at the CFP code, it appears that the algorithm it roughly uses to poll for changes is:

• (A) Read Current Set of Changes
• (B) Dispatch them to observer
• (C) Wait Poll interval, Goto (A)

This means that any changes that occur during (B) will need to wait Poll interval before being processed.

Has it been considered to instead implement as:

• (A) Read Current set of Changes
• If None, Wait Poll Interval, Goto (A)
• Dispatch Them to observer
• Goto (A)

This would allow changes that were "in flight" during processing of the previous window to be more quickly processed, and the poll interval would only be hit in the event that we had a zero-read "no changes".

Maybe there is some way to achieve this with the current CFP?

[ealsur]

That is not correct, the process is explained here: https://docs.microsoft.com/en-us/azure/cosmos-db/sql/change-feed-processor#processing-life-cycle

The wait time is only used when we receive a HTTP 304 / Not Modified from reading new changes.

After doing a read, the iterator has a HasMoreResults flag, and we loop until that is false: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/ChangeFeedProcessor/FeedProcessing/FeedProcessorCore.cs#L71