-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Micronaut Kafka Listener commits messages at shutdown that should not be commited with offset strategy SYNC #315
Comments
Guys this can potentially lead to data loss! Any opinion about this? |
@burtbeckwith can you please prioritise this issue |
@JoernSchimmelpfeng you checked "example on GitHub" - do you have a test app? |
Sorry for my stupidity. I somehow checked the box and could not uncheck it again. Perhaps a browser issue. |
@JoernSchimmelpfeng PRs welcome! |
…s messages at shutdown
I think I've noticed similar behavior. I see JoernSchimmelpfeng added a commit. Is there anything outstanding regarding his proposed fix? |
@seancarroll can you test the proposed PR and potentially provide a test? it hasn't been merged because it doesn't include a test |
@graemerocher, given the number of commits behind the PR is where does it make sense to potentially add a test to? |
You could either rebase the PR or create another PR based off that one and leave the rebase to us |
Sounds good. I think I have a test that shows for one scenario but it does require an exception to be thrown. In another test that only has a wakeup i'm still seeing behavior in which commits are still taking place. Its unclear to me right now why on a WakeUpException we would ever want to do a commit sync. Or perhaps more specifically why doing a commit in the finally block is required. There also might be something else going in the scenario in which I'm only calling wakeup that is causing commits which I dont think should be occurring. Still attempting to understand the behavior. In particular one example (test) is when using batch and a WakeUpException occurs in the middle of processing. |
With OffsetStrategy set to SYNC or ASYNC the KafkaListner should not commit messages when an Exception occurs. However during shutdown all messages get committed regardless if there was an Exception or not.
Task List
Steps to Reproduce
Expected Behaviour
The KafkaListener should not commit messages with offset strategy SYNC (or ASYNC) at shutdown time that could not be processed due to an Exception.
Actual Behaviour
All events get committed. Only with offset strategy DISABLED this set is skipped.
The text was updated successfully, but these errors were encountered: