Expire silences after one day instead of one year #194
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The cronjob renews the silence every 4 hours. I don't see any added benefits in creating silences with 1 year lifetime instead of one day.
However there is one major downside. If the silence functionality is used to suppress certain alerts temporarily, the silence stays active for 1 year after it's removed from the config hierarchy. To enable the alert again it, the silence has to be expired manually on all clusters.
Reducing this to one day is a compromise that allows to remove silences and ensure they're active again in a reasonable timeframe without having to manually expire all silences. Happy to discuss other values for the duration.
Checklist
changelog.
The PR has a meaningful description that sums up the change. It will be
linked in the changelog.
bug
,enhancement
,documentation
,change
,breaking
,dependency
as they show up in the changelog.