fix discard of packets so the next packet to expect is updated correctly #302
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Comment on lines
+436
to
+440
| // we need to update the next expected seqno even if the iterator returns nothing, since it may have discarded some packets. | ||
| // iterators do not support ref arguments, so we use a simple wrapper class to work around this limitation | ||
| MutableLong nextPacketToReceive = new MutableLong() { Value = current.NextPacketToReceive.seqNo }; | ||
|
|
||
| await foreach ((EventData eventData, PartitionEvent[] events, long seqNo) in this.blobBatchReceiver.ReceiveEventsAsync(this.taskHubGuid, packets, current.ErrorHandler.Token, nextPacketToReceive)) |
There was a problem hiding this comment.
We also call ReceiveEventsAsync in LoadMonitorProcessor and EventHubsTransport. What is the reason why we don't need to use update the expected seqno in there?
There was a problem hiding this comment.
Those two are used by clients and the load monitor, respectively. The latter do not care about the order or about duplication of received events, so they are not tracking the sequence numbers.
sebastianburckhardt
added a commit
that referenced
this pull request
Oct 11, 2023
sebastianburckhardt
added a commit
that referenced
this pull request
Oct 17, 2023
sebastianburckhardt
added a commit
that referenced
this pull request
Oct 23, 2023
* Update CreationRequestReceived.cs * remove the faster-alternate data store and fix broken deserialization. (#308) * fix and instrument commitlog serialization and deserialization. (#305) * fix discard of packets so the next packet to expect is updated correctly (#302) * New EventHubs performance tests (#178) * rebase, and remove changes to scale monitor * add hello cities test that prints the history of a nested orchestration * implement a watchdog that terminates CompletePending quickly if it hangs (#318) * add unique id to scale monitor constructor (#316) * fix bugs from PR #302 (#315) * fix cache size reporting (#321) * added a comment * update durable task package references to 2.15.1 and 2.12.0 (#317) * update DT and DF package references * update to latest * sync dev w/ main (#324) * Update GH automation (#303) * initial commit (#290) * Revert "initial commit (#290)" (#314) This reverts commit 3a2d193. * Bump Azure.Identity from 1.7.0 to 1.10.2 in /samples/TokenCredentialDTFx (#323) Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.7.0 to 1.10.2. - [Release notes](https://github.com/Azure/azure-sdk-for-net/releases) - [Commits](Azure/azure-sdk-for-net@Azure.Identity_1.7.0...Azure.Identity_1.10.2) --- updated-dependencies: - dependency-name: Azure.Identity dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Terminate partition when FASTER refuses to checkpoint for over a minute (#301) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Varshitha Bachu <vabachu@microsoft.com> Co-authored-by: Sebastian Burckhardt <sburckha@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Sebastian Burckhardt <sburckha@microsoft.com> Co-authored-by: David Justo <david.justo.1996@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes a bug where discarding events from other taskhubs causes the expected sequence number to be wrong (not updated to reflect the discarded packets) which then triggers spurious errors when receiving more events.
This code is not tested yet.