-
Notifications
You must be signed in to change notification settings - Fork 800
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increased timeout value for delete history event task #5888
Conversation
Codecov Report
Additional details and impacted files
... and 10 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
Pull Request Test Coverage Report for Build 018ec3d7-42d3-4387-8cf7-9808097685aeDetails
💛 - Coveralls |
@@ -110,7 +110,10 @@ func (t *timerStandbyTaskExecutor) Execute( | |||
case persistence.TaskTypeWorkflowBackoffTimer: | |||
return t.executeWorkflowBackoffTimerTask(ctx, timerTask) | |||
case persistence.TaskTypeDeleteHistoryEvent: | |||
return t.executeDeleteHistoryEventTask(ctx, timerTask) | |||
// special timeout for delete history event | |||
deleteHistoryEventContext, deleteHistoryEventCancel := context.WithTimeout(context.Background(), taskDeleteHistoryEventContextTimeout) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know the existing ctx
has 3 sec timeout and doesn't listen for queue shut down signals.
With the new 60s timeout for history deletions it is going to be more likely to end up hanging a shutdown. We should consider making these operations responsive to shut down signals.
@Shaddoll what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed offline. We will hook up Stop signals to task executor in a follow up PR.
In this PR, let's make this 60s dynamically configurable
What changed?
We increased the timeout value for delete history event task to dynamic config with default at 30s.
Why?
The timeout value for delete history event is too low causing large history deletion to constantly timeout
How did you test it?
Potential risks
Release notes
Documentation Changes