Introduce exponential backoff for ErrorStrategy retry delay

[georgeharley]

Feature description

At present the @ErrorStrategy available for a @KafkaListener using the "retry on error" strategy offers users the option of setting a fixed time delay between event consumption attempts.

It would be helpful if the option also existed to configure an exponential backoff approach with users able to configure initial and maximum delay values.

[breader124]

Hi, it sounds interesting, if that's possible, I'd like to start working on it

[breader124]

I tried to start working on it, but build is failing for me locally. Errors I'm getting are exactly the same as in case of the failing builds on GitHub. @graemerocher, could you please give it a look?

[larsduelfer]

I also tried to get the build running locally to draft another PR but was also not yet successful. Help would be much appreciated @graemerocher

[graemerocher]

@breader124 @larsduelfer if you can use the 4.5.x branch in the interum as master is in the middle of a Micronaut 4 migration https://github.com/micronaut-projects/micronaut-kafka/tree/4.5.x

Thanks

[larsduelfer]

@breader124 Does the branch build locally on your machine? I have a handful of failing tests on the 4.5.x branch at the moment.

[breader124]

No, when I try to run ./gradlew check just after cloning the repository, I'm getting lots of errors related to problems with spinning up Kafka container and missing DNS resolver-related classes (it's exactly this error described here). They both happen quite often to me, because I work on a computer powered with M1 from Apple. I managed to replace all the Kafka image names to point to the ones built specifically for ARMs and also include the newest version of micrometer-core to override transitive dependency to the older one introduced by micronaut-micrometer. Even after these modifications around 2-3 tests are still failing. Tried to run them on the branch suggested above and on master as well, because I noticed successful build there yesterday.

[larsduelfer]

@breader124 I have the same setup (Apple with M1). Using the 4.5.x branch, I have one failing / toggling test in the kafka sub-project and a few more in the kafka-streams subproject. Need to further investigate, how to fix that.

[breader124]

@larsduelfer, it seems that I managed to configure the 4.5.x branch the way that it passes the build. Changes I introduced:

• Replaced Kafka Docker images built for x86 with the ones built for ARM
• In kafka-streams module I added direct dependency on kafka-streams:3.3.1 to force Gradle to resolve this one instead of the default kafka-streams:2.8.2. kafka-streams:2.8.2 has a dependency on obsolete RocksDB and suffers from the problem described here. This problem was solved one time in Upgrade Kafka to 3.0.0 #474, but the solution didn't find its way to branch 4.5.x.
• In kafka module I added direct dependency on micrometer-registry-statsd:1.10.2. This way I got rid of errors related to DNS resolution. Even without this change build was passing, but still wanted to get rid of those exceptions
• Replaced @IgnoreIf with @Ignore here. Originally this test was ignored only if GITHUB_WORKFLOW env variable was set, what was the case for build on GitHub, but wasn't for my computer.

You can find all the necessary changes in this commit. I hope it'll help you and it seems that now I'm able to continue working on the original issue.

[breader124]

@graemerocher, I opened a PR, could you give it a look?