-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add: Error Handling Environment Bundle to Bento configuration #174
Conversation
6b2fdcb
to
fa81b67
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would just change the key strict_mode
to something like on_error
with the values being an enum of, continue
(default) / strict_mode
.
So that we can add new modes later on if required, or to add a new mode to change the retry behaviour later? i.e. a new on_error: exponential_backoff
.
I think that adding this field to the manager level also makes sense when running in streams mode, it will be ever stream that is created will have the same strict_mode / on_error mode - will simplify understanding?
fa81b67
to
26bf1a7
Compare
26bf1a7
to
2b2e3fa
Compare
Co-authored-by: Jem Davies <131159520+jem-davies@users.noreply.github.com>
Motivation
We need global config to allow for a processing step to return and log an error, with the subsequent processors not being run.
Related to #166 -- where we opted for nacking at the failed
processor
instead of at the output layer.Can be used as follows:
Changes
error_handling
in the top-level that can be one ofreject
ornone
output
.generate
input component always enablingauto_replay_nacks
.TODO
Add more tests.Populate this config at the manager level to allow for nestedinput.processors
andoutput.processors
to also have their results checked for errors.input
to stop new data being read in.Add documentation for this feature.