Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NonSlidingUntil for deployment #692

Merged
merged 1 commit into from
Jan 11, 2022

Conversation

jklaw90
Copy link
Contributor

@jklaw90 jklaw90 commented Jan 6, 2022

Use NonSlidingUntil over Until in order to account for function duration.
Trying to make the run times on more of an interval in order to make executions times more predictable.

Using NonSlidingWindow will make the deployment behave more like the cronjob timing wise.

@k8s-ci-robot
Copy link
Contributor

Hi @jklaw90. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 6, 2022
@jklaw90
Copy link
Contributor Author

jklaw90 commented Jan 6, 2022

/assign @seanmalloy

Copy link
Contributor

@damemi damemi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ok-to-test
Sounds good to me, thanks for the suggestion @jklaw90 !

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 6, 2022
@@ -116,7 +116,7 @@ func RunDeschedulerStrategies(ctx context.Context, rs *options.DeschedulerServer
ignorePvcPods = *deschedulerPolicy.IgnorePVCPods
}

wait.Until(func() {
wait.NonSlidingUntil(func() {
Copy link
Contributor

@ingvagabund ingvagabund Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent of the jitter factor is to make sure all clients sending a request to the apiserver do not sent the request at the same time or too close to each other in time. Given the running time of all the strategies is unknown in advance, using the sliding version helps to increase the randomness.

Trying to make the run times on more of an interval in order to make executions times more predictable.

@jklaw90 I wonder what's the motivation for making the execution time more predictable and if there are measurements which might tell us if using the non-sliding version has any benefits for the overall resource utilization (e.g. API requests) of the cluster.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what's the motivation for making the execution time more predictable

I think the use case could be for example: you expect descheduler to run every 30 minutes, but you have a giant (hypothetical) cluster where each run takes 20 minutes. If the timer doesn't start until the first run completes, that means your 2 runs will actually appear 50 minutes apart

@ingvagabund
Copy link
Contributor

@jklaw90 thank you for opening the topic of the descheduler improvements. Is your motivation to improve the debugability, performance or other aspect of the descheduler?

@jklaw90
Copy link
Contributor Author

jklaw90 commented Jan 10, 2022

@ingvagabund debugging and testability are why i think we should use the non sliding window.
we have smoke tests that were testing the old cronjob which always ran on an exact interval. The Until method causes the "interval" to not be respected. From our findings when we set the interval to 10m and we were seeing runs ~12 - 15 minutes apart.

Here is an example of the two since the documentation wasn't the clearest for NonSlidingUntil.

func main() {
	wait.NonSlidingUntil(func() {
		log.Println("running")
		time.Sleep(8 * time.Second)
	}, 10*time.Second, nil)
}
// 2022/01/10 08:47:54 running
// 2022/01/10 08:48:04 running
// 2022/01/10 08:48:14 running
// 2022/01/10 08:48:24 running

func main() {
	wait.Until(func() {
		log.Println("running")
		time.Sleep(8 * time.Second)
	}, 10*time.Second, nil)
}
// 2022/01/10 08:48:50 running
// 2022/01/10 08:49:08 running
// 2022/01/10 08:49:26 running
// 2022/01/10 08:49:44 running

@ingvagabund
Copy link
Contributor

ingvagabund commented Jan 10, 2022

@jklaw90 @damemi the use cases you mentioned make clear this change improve the user experience as well. Which is also valuable.

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ingvagabund, jklaw90

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 10, 2022
@damemi
Copy link
Contributor

damemi commented Jan 11, 2022

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 11, 2022
@k8s-ci-robot k8s-ci-robot merged commit 796f347 into kubernetes-sigs:master Jan 11, 2022
briend pushed a commit to briend/descheduler that referenced this pull request Feb 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants