Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configs/installer: increase the resource limits for all image builds #3588

Merged

Conversation

abhinavdahiya
Copy link
Contributor

With addition of azure terraform plugins 1, looks like the installer builds are failing because of OOMKilled 2

/cc @wking @staebler

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 24, 2019
@stevekuznetsov
Copy link
Contributor

Is it valid to add limits to builds? @smarterclayton @bparees

@bparees
Copy link
Contributor

bparees commented Apr 24, 2019

Is it valid to add limits to builds? @smarterclayton @bparees

the build api supports it, if that's what you're asking.

whatever you put here:
https://github.com/openshift/api/blob/master/build/v1/types.go#L71

will be translated onto the build pod that's created.

@smarterclayton
Copy link
Contributor

Memory limits yes, helps ensure the correct thing gets oomkilled. Something is going to die, we want it to be guy who went bad

@bparees
Copy link
Contributor

bparees commented Apr 24, 2019

Something is going to die, we want it to be guy who went bad

https://www.youtube.com/watch?v=d73XkPQPKiY

requests:
cpu: "3"
memory: 5Gi
upi-installer:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how these keys tie us to the openshift-install compilation step. Maybe we want to restore just the bin key I seem to have accidentally removed in 3f2f01c (#1677)? Which was maybe originally from 0b1cdda (#891)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using it similar to unit

Yup, although I'd be surprised if wee needed 5 GiB memory for our unit tests. And I'm still not clear how these are supposed to work, but I worry that this line is getting limits/requests for our UPI jobs (which are probably fine already) instead of for the compilation job that the UPI tests depend on. Does the CI output show this worked?

Copy link
Contributor Author

@abhinavdahiya abhinavdahiya Apr 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the CI output show this worked?

yes the build jobs for rehearse had the correct resource requests/limits for build pods.

@openshift-ci-robot
Copy link
Contributor

@abhinavdahiya: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/rehearse/openshift/installer/master/e2e-openstack c589ac7 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-rhel-scaleup c589ac7 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-vsphere c589ac7 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-aws c589ac7 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-metal c589ac7 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-libvirt c589ac7 link /test pj-rehearse
ci/prow/pj-rehearse c589ac7 link /test pj-rehearse

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@wking
Copy link
Member

wking commented Apr 25, 2019

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 25, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit c218372 into openshift:master Apr 25, 2019
@openshift-ci-robot
Copy link
Contributor

@abhinavdahiya: Updated the following 2 configmaps:

  • ci-operator-master-configs configmap in namespace ci using the following files:
    • key openshift-installer-master.yaml using file ci-operator/config/openshift/installer/openshift-installer-master.yaml
  • ci-operator-master-configs configmap in namespace ci-stg using the following files:
    • key openshift-installer-master.yaml using file ci-operator/config/openshift/installer/openshift-installer-master.yaml

In response to this:

With addition of azure terraform plugins 1, looks like the installer builds are failing because of OOMKilled 2

/cc @wking @staebler

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants