[FEATURE REQ] Dead-letter and retry mechanism for Event Hub

[kacheng2018]

The customer experiences exception failure and application abort due to failures in event hubs for sending / receiving messages, and the messages will be overflown if the application cannot be recovered in-time. Currently, the event hub does not provide a mechanism for dead-letter handling and re-try policy where it will incur risk of messages lost. Requesting to add dead-letter and retry mechanism for event hub in the SDK

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl.

Issue Details

---

The customer experiences exception failure and application abort due to failures in event hubs for sending / receiving messages, and the messages will be overflown if the application cannot be recovered in-time. Currently, the event hub does not provide a mechanism for dead-letter handling and re-try policy where it will incur risk of messages lost. Requesting to add dead-letter and retry mechanism for event hub in the SDK

Author:	kacheng2018
Assignees:	-
Labels:	Client, Event Hubs, Service Attention, customer-reported, feature-request, question
Milestone:	-

[conniey]

This is a feature that AFAIK isn't available in the Azure Event Hubs service. @JamesBirdsall would be able to provide more accurate information.

[JamesBirdsall]

Event Hubs does not have the concept of dead-lettering because messages are not consumed destructively. Messages only become unavailable when they exceed the retention period and are purged. At any time, an application can open a receiver to consume any message that has not expired, and can re-open receivers to re-receive a particular message as many times as necessary until it has been processed properly.

Speaking very generally, checkpointing is a common pattern, in which an application persistently stores the offset (or sequence number) of the last message processed successfully for each partition of the event hub. Normally it is not necessary for the application to re-receive the message because it can simply keep trying to process the copy of the message that it already has, but if it is necessary to restart the receiver for any reason (application shutdown and restart, application crash and restart, VM crash and restart, etc.) then the checkpointed offset indicates the position where the restarted receiver should begin in the partition.

The producing side is simpler. If the send to the event hub succeeds, then our service has persisted the message and the producing application can move on to the next message. If the send fails, our SDKs will automatically retry if the error is transient, up to the timeout specified. If it is still failing, the SDK will throw and what to do then is up to your code, but you probably want to retry at that level too. If it keeps failing, you should have some sort of application monitoring so you can open a support ticket.

[ramya-rao-a]

Thanks for the clarification @JamesBirdsall

@kacheng2018 We hope that have answered your queries.
Please do open a new issue if you have any other feedback for us.