Max Retries #337
Open
Max Retries #337
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Is there a Jira associated with this PR? The reason I ask, is that it would be useful to understand the impact of setting a max-retry and it failing in terms of the state of the system and recovery. I suspect this would be worthy of explaining and getting feedback in a Jira. |
|
There is already a issue here. Seems this project is no longer maintained. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
maxRetriesoption that limits the amount of times a request will be attempted before the request is dropped. When set to the default value of -1, batches will be retried indefinitely which matches the current behaviour.This provides a simple means to drop failed messages after a configurable number of attempts when such behaviour is desired, which is currently not very easy to implement.
This also addresses an issue with Flink Statefun where a pipeline will get stuck if a remote function fails deterministically and the issue cannot be caught inside the handler. Normally, such issues can and should be caught inside the handler code, but this is not possible if the server deterministically crashes, e.g. with
log.Fatalwin golang. Setting a maxRetries limit prevents such behaviour from blocking pipelines indefinitely.