Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VPA: make parameters oomBumpUpRatio and oomMinBumpUp configurable #5275

Merged
merged 7 commits into from
Jan 30, 2023

Conversation

navinjoy
Copy link
Contributor

Which component this PR applies to?

vertical-pod-autoscaler/recommender

What type of PR is this?

/kind feature

What this PR does / why we need it:

This is a small change for adding the parameters for oomBumpUpRatio and oomMinBumpUp.

We have executed multiple Load tests for VPA and since VPA has to terminate the pods and new recommendations are applied during recreation of a pod and when OOM occurs during load, 20% increase is not sufficient. Therefore we want this to be configurable and for our service and tests we did we doubled the memory. Having these flags configurable will help for different types of services.

Which issue(s) this PR fixes:

No issue exists for this.

Special notes for your reviewer:

Validated the change in my local dev cluster k8s 1.25.

% kubectl get node
NAME                       STATUS   ROLES           AGE   VERSION
vpa-k8s125-control-plane   Ready    control-plane   24h   v1.25.0
% k describe deploy vpa-recommender
Name:                   vpa-recommender
Namespace:              kube-system
CreationTimestamp:      Wed, 19 Oct 2022 16:36:22 -0700
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision: 3
Selector:               app=vpa-recommender
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           app=vpa-recommender
  Service Account:  vpa-recommender
  Containers:
   recommender:
    Image:      <updated image with change>
    Port:       8942/TCP
    Host Port:  0/TCP
    Args:
      --oom-bump-up-ratio=1.5
      --oom-min-bump-up=524288000

Does this PR introduce a user-facing change?

NA

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NA

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 26, 2022
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Oct 26, 2022
@navinjoy navinjoy changed the title Oom params VPA: make parameters oomBumpUpRatio and oomMinBumpUp configurable Oct 26, 2022
Copy link
Collaborator

@jbartosik jbartosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please mention those flags in README/Starting multiple recommenders

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 31, 2022
@navinjoy
Copy link
Contributor Author

navinjoy commented Nov 1, 2022

@jbartosik I have addressed the review comments, could you please check if it looks good now. Thank you.

Copy link
Collaborator

@jbartosik jbartosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is close to ready

vertical-pod-autoscaler/pkg/recommender/main.go Outdated Show resolved Hide resolved
@navinjoy
Copy link
Contributor Author

@jbartosik please review when you get a chance.

Copy link
Collaborator

@jbartosik jbartosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was unavailable for a while.

Please add a release note to the description.

vertical-pod-autoscaler/README.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@jbartosik jbartosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please fix the presubmit error:

vertical-pod-autoscaler/README.md:298:86: "occured" is a misspelling of "occurred"

Copy link
Contributor

@voelzmo voelzmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions regarding the wording for the sake of consistency and clarity.

vertical-pod-autoscaler/README.md Outdated Show resolved Hide resolved
vertical-pod-autoscaler/README.md Outdated Show resolved Hide resolved
vertical-pod-autoscaler/README.md Outdated Show resolved Hide resolved
vertical-pod-autoscaler/README.md Outdated Show resolved Hide resolved
vertical-pod-autoscaler/pkg/recommender/main.go Outdated Show resolved Hide resolved
Accepted the suggested changes and commited.

Co-authored-by: Marco Voelz <voelzmo@users.noreply.github.com>
@navinjoy
Copy link
Contributor Author

navinjoy commented Dec 5, 2022

@jbartosik @voelzmo I committed the suggested changes for the PR. Can you please review and approve this. Thank you.

@navinjoy navinjoy requested review from jbartosik and removed request for kgolab December 12, 2022 08:29
@voelzmo
Copy link
Contributor

voelzmo commented Dec 12, 2022

Thanks for including the wording changes! At this point, I'm wondering if this is good to go, or if we should have some tests showing that specifying a custom value for oomMinBumpUp and oomBumpUpRatio actually does what it should do? @jbartosik, WDYT? Unfortunately, I didn't see any tests for the other cases in which we moved out configuration like this from being hardcoded into a flag (like e.g. targetCPUPercentile)

@navinjoy
Copy link
Contributor Author

@jbartosik could you please review and let me know if its ready to merge. Thank you.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 9, 2023
@jbartosik jbartosik mentioned this pull request Jan 10, 2023
@jbartosik
Copy link
Collaborator

@navinjoy looks good to me. Can you rebase the PR?

@jbartosik
Copy link
Collaborator

@navinjoy please squash commits and rebase the PR

@navinjoy
Copy link
Contributor Author

Sure will do that. I was out last 2weeks.

@jbartosik
Copy link
Collaborator

Thanks, please ping me when you do

@jbartosik
Copy link
Collaborator

Do you think you can squash & rebase this week? I'd like to resume working on the release next week

@navinjoy
Copy link
Contributor Author

@jbartosik I will do this today. Got struck with my tasks.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. area/addon-resizer area/cluster-autoscaler area/helm-charts and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 23, 2023
@Shubham82 Shubham82 mentioned this pull request Jan 24, 2023
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 26, 2023
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 26, 2023
@navinjoy
Copy link
Contributor Author

@jbartosik please check if the PR is ready to merge.

Copy link
Collaborator

@jbartosik jbartosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 30, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jbartosik, navinjoy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 30, 2023
@k8s-ci-robot k8s-ci-robot merged commit 65c098b into kubernetes:master Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/vertical-pod-autoscaler cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants