Fatal exceptions using RetryTopicConfiguration

[einarjohnson]

Hi,
Is it possible to set a retriable topic configuration up in such a way that certain fatal exceptions go straight to the DLQ topic/handler?

[garyrussell]

For normal retry processing, see https://docs.spring.io/spring-kafka/docs/current/reference/html/#seek-to-current

Starting with version 2.3, the SeekToCurrentErrorHandler considers certain exceptions to be fatal, and retries are skipped for such exceptions; the recoverer is invoked on the first failure. The exceptions that are considered fatal, by default, are:

DeserializationException

MessageConversionException

ConversionException

MethodArgumentResolutionException

NoSuchMethodException

ClassCastException

since these exceptions are unlikely to be resolved on a retried delivery.

You can add more exception types to the not-retryable category, or completely replace the map of classified exceptions. See the Javadocs for SeekToCurrentErrorHandler.setClassifications() for more information, as well as those for the spring-retry BinaryExceptionClassifier.

See

spring-kafka/spring-kafka/src/main/java/org/springframework/kafka/listener/FailedRecordProcessor.java

Lines 74 to 92 in ad1ae60

	/**
	* Set an exception classifications to determine whether the exception should cause a retry
	* (until exhaustion) or not. If not, we go straight to the recoverer. By default,
	* the following exceptions will not be retried:
	* <ul>
	* <li>{@link DeserializationException}</li>
	* <li>{@link MessageConversionException}</li>
	* <li>{@link MethodArgumentResolutionException}</li>
	* <li>{@link NoSuchMethodException}</li>
	* <li>{@link ClassCastException}</li>
	* </ul>
	* All others will be retried.
	* When calling this method, the defaults will not be applied.
	* @param classifications the classifications.
	* @param defaultValue whether or not to retry non-matching exceptions.
	* @see BinaryExceptionClassifier#BinaryExceptionClassifier(Map, boolean)
	* @see #addNotRetryableExceptions(Class...)
	*/
	public void setClassifications(Map<Class<? extends Throwable>, Boolean> classifications, boolean defaultValue) {

to replace the defaults, and

spring-kafka/spring-kafka/src/main/java/org/springframework/kafka/listener/FailedRecordProcessor.java

Lines 162 to 181 in ad1ae60

	/**
	* Add exception types to the default list. By default, the following exceptions will
	* not be retried:
	* <ul>
	* <li>{@link DeserializationException}</li>
	* <li>{@link MessageConversionException}</li>
	* <li>{@link ConversionException}</li>
	* <li>{@link MethodArgumentResolutionException}</li>
	* <li>{@link NoSuchMethodException}</li>
	* <li>{@link ClassCastException}</li>
	* </ul>
	* All others will be retried.
	* @param exceptionTypes the exception types.
	* @since 2.6
	* @see #removeNotRetryableException(Class)
	* @see #setClassifications(Map, boolean)
	*/
	@SafeVarargs
	@SuppressWarnings("varargs")
	public final void addNotRetryableExceptions(Class<? extends Exception>... exceptionTypes) {

[garyrussell]

Ah - I see; set on the annotation itself.

But, shouldn't we provide a way to set the classifierBuilder on the RetryTopicConfigurationBuilder for when programmatic configuration is used?

[tomazfernandes]

@garyrussell, for programmatic configuration we can use something like this:

@Bean
public RetryTopicConfiguration myRetryTopic(KafkaTemplate<String, MyOtherPojo> template) {
    return RetryTopicConfigurationBuilder
            .newInstance()
            .notRetryOn(MyDontRetryException.class)
            .traverseCauses()
            .create(template);
}

[garyrussell]

Doh - sorry; got it now.

Great work, once again!

[tomazfernandes]

Thanks a lot @garyrussell! Glad to see users bringing questions :)

[einarjohnson]

@garyrussell and @tomazfernandes , thanks for your help here. I will try the notRetryOn on the builder.