Check update stagger #884
Check update stagger #884
Conversation
|
You might want to look at the existing randomStagger: https://github.com/hashicorp/consul/blob/master/command/agent/util.go#L40 |
|
Hah, yeah, I actually pulled the logic from that function but for some reason though it was in a different package. Updated that. @ryanuber and @armon, let me know what you think about it being configurable. I added it so that the behavior doesn't change for existing users unexpectedly but it may be a small change that would actually help everyone. |
|
So the situation this would help under would be a server with a very large amount of checks, all with frequently changing output and stable statuses. Since this is such a minor optimization, I don't think we need to introduce a configuration option to enable/disable it. |
|
Dropped the stagger on/off config. |
|
Thanks! |
The AKS nightly tests panicked twice in the last 3 days after the 40m timeout. The tests used to have a 40m timeout per package, but a recent PR changed them to all run together. So, it needs a lot longer timeout to run all the tests together. This commit increases the test timeout to 2h. It also increases the no_output_timeout to slightly longer than that since there's no output until the tests finish.
Adding an optional stagger to the CheckUpdateInterval so that when consul runs in an environment with very frequency checks, all of those checks are not flushed at the same time when the CheckUpdateInterval expires. The stagger has a configurable on/off as "check_update_stagger" in the config and staggers between .5 and 1.5 of the CheckUpdateInterval.