Combine blocking and non-blocking error handling

[ghost]

Hello 🙂

I am looking for a possibility to combine blocking and non-blocking error handling:

• non-blocking (but not fatal) exceptions should be retried -> I am already using @RetryableTopic for this purpose
• non-blocking (but fatal, e.g. DeserializationException) exceptions should be routed to the DLT directly -> should be possible using exception classification
• blocking exceptions (e.g. a database is currently not available) should not be routed to a retry topic since all following messages would be routed there as well. The consumer should remain at the current offset -> should be possible using DefaultErrorHandler?

Now my question is: How can I configure the error handling accordingly, to use either non-blocking or blocking error handling depending on the thrown exception? How do I need to combine the configuration of RetryableTopic and DefaultErrorHandler?

Thanks in advance!

[garyrussell]

You can satisfy the first two bullets by adding retryable exceptions using the include and exclude properties (or by using the .retryOn() and noRetryOn() builder methods when using the configuration builder).

There is no support for falling back to blocking retries based on an exception.

An interesting idea, but I am not sure how difficult it would be to implement.

cc/ @tomazfernandes

As a work around for now, you could stop the container (with the stopImmediate container property true).

[ghost]

Thanks for the quick response! Then I'll have a look if I can use the workaround. I came up with the idea of distinguishing exceptions after reading this article. Maybe it's also interesting for you

[ghost]

As a follow-up question: Would you mind if I open an issue as a feature request for this? @garyrussell

[garyrussell]

Sure; but I can't guarantee when we will get to it.

[tomazfernandes]

Please do @mariecuriiee - I should be able to take a look into this soon. Thanks.

[ghost]

Sorry, haven't found the time for it in the last few days. The issue is here 🙂

[tomazfernandes]

That sure looks like an interesting feature, we'd be able to benefit from the best of both worlds.

We already have a DefaultErrorHandler and its blocking retries setup for the retryable topics, but currently it's hardcoded with no retries and we only use its classification for not getting stuck in endless fatal exceptions loops in the DLT.

By leveraging that we'd be able to execute the blocking retries before we get into the retry topics' logic. Not sure it's doable without changing some of the current classification behavior though.

OTOH it should be simple enough to add new logic to the DLPRF to handle this on the retry topics' side, so we can just go that way too.

[garyrussell]

Somewhat related issue: https://stackoverflow.com/questions/71165794/how-to-avoid-hitting-all-retryable-topics-for-fatal-by-default-exceptions/71165954#71165954