Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update AWS CCM tags #12104

Merged
merged 3 commits into from
Sep 9, 2021
Merged

Update AWS CCM tags #12104

merged 3 commits into from
Sep 9, 2021

Conversation

rifelpet
Copy link
Member

@rifelpet rifelpet commented Aug 5, 2021

WIP pending any action from kubernetes/cloud-provider-aws#244

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 5, 2021
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2021
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/addons and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 4, 2021
@rifelpet
Copy link
Member Author

rifelpet commented Sep 4, 2021

kubernetes/cloud-provider-aws#267 has the new location for images (including going back to 1.18)

/test ?

@rifelpet
Copy link
Member Author

rifelpet commented Sep 4, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@rifelpet rifelpet changed the title WIP Add AWS CCM v1.21.0 tag Add AWS CCM v1.21.0 tag Sep 4, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 4, 2021
@rifelpet
Copy link
Member Author

rifelpet commented Sep 4, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@rifelpet
Copy link
Member Author

rifelpet commented Sep 5, 2021

/retest

olemarkus
olemarkus reviewed Sep 5, 2021
View reviewed changes
upup/models/cloudup/resources/addons/aws-cloud-controller.addons.k8s.io/k8s-1.18.yaml.template Outdated
@@ -28,7 +28,7 @@ spec:
serviceAccountName: aws-cloud-controller-manager
containers:
- name: aws-cloud-controller-manager
image: {{ if .ExternalCloudControllerManager.Image }}{{ .ExternalCloudControllerManager.Image }}{{ else }}gcr.io/k8s-staging-provider-aws/cloud-controller-manager:{{AWSCCMTag}}{{ end }}
image: {{ if .ExternalCloudControllerManager.Image }}{{ .ExternalCloudControllerManager.Image }}{{ else }}us.gcr.io/k8s-artifacts-prod/provider-aws/cloud-controller-manager:{{AWSCCMTag}}{{ end }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should move this + the tag logic to pkg/model/components. Then we can more easily fall back to latest for newer k8s versions.

Also, should the repository for prod be k8s.gcr.io/provider-aws/cloud-controller-manager?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to move it to pkg/model/components but the cluster spec as seen by AWSCloudControllerManagerOptionsBuilder:

kops/pkg/model/components/awscloudcontrollermanager.go

Line 39 in 8dee92b

eccm := clusterSpec.ExternalCloudControllerManager

shows cloudControllerManager as empty, even if the actual cluster spec includes the cloudProvider subfield.

I think a related symptom is that the integration test input includes the CCM struct and fields:

kops/tests/integration/update_cluster/many-addons-ccm/in-v1alpha2.yaml

Lines 19 to 20 in 8dee92b

cloudControllerManager:
provider: aws

but the completed cluster spec as stored in the state store has an empty CCM struct:

kops/tests/integration/update_cluster/many-addons-ccm/data/aws_s3_bucket_object_cluster-completed.spec_content

Line 21 in 8dee92b

cloudControllerManager: {}

I don't know why that is happening but I'm guessing it is related. Any help would be appreciated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I run kops get cluster -o yaml --full I do see:

  cloudControllerManager:
    allocateNodeCIDRs: true
    cloudProvider: aws
    clusterCIDR: fd00:10:96::/64
    clusterName: test.kops-dev.srsandbox.io
    configureCloudRoutes: false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With some rough testing, this only seems to be happening with the terraform tests, but why ... it doesn't make any sense to me.

@rifelpet
Copy link
Member Author

rifelpet commented Sep 5, 2021

/retest

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 8, 2021
@rifelpet
Copy link
Member Author

rifelpet commented Sep 8, 2021

I just pushed what ./hack/update-expected.sh is setting for the integration test files. I don't understand why one manifest has image: null and another manifest hasimage: removed altogether.

@rifelpet
Copy link
Member Author

rifelpet commented Sep 8, 2021

The external cloud controller manager data seems to be lost during decoding:

kops/cmd/kops/create.go

Line 123 in d69106e

o, gvk, err := kopscodecs.Decode(section, nil)

The input yaml has the fields but the outputted object does not.

@rifelpet rifelpet changed the title Add AWS CCM v1.21.0 tag Update AWS CCM tags Sep 8, 2021
@rifelpet
Copy link
Member Author

rifelpet commented Sep 8, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@rifelpet
Copy link
Member Author

rifelpet commented Sep 8, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@rifelpet
Copy link
Member Author

rifelpet commented Sep 9, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@rifelpet
Copy link
Member Author

rifelpet commented Sep 9, 2021

/test pull-kops-e2e-cni-kuberouter

@rifelpet
Copy link
Member Author

rifelpet commented Sep 9, 2021

@olemarkus it turns out the decoding issues were due to the manifests having provider: instead of cloudProvider:. The test outputs now look as expected and e2e is passing

@rifelpet
Copy link
Member Author

rifelpet commented Sep 9, 2021

/test pull-kops-e2e-ipv6-conformance

@rifelpet
Copy link
Member Author

rifelpet commented Sep 9, 2021

/test pull-kops-e2e-cni-kuberouter

@hakman
Copy link
Member

hakman commented Sep 9, 2021

/retest

@hakman
Copy link
Member

hakman commented Sep 9, 2021

/test pull-kops-e2e-aws-cloud-controller-manager

@hakman
Copy link
Member

hakman commented Sep 9, 2021

/test pull-kops-e2e-ipv6-conformance

@kubernetes kubernetes deleted a comment from k8s-ci-robot Sep 9, 2021
@hakman hakman mentioned this pull request Sep 9, 2021
Merged
@hakman
Copy link
Member

hakman commented Sep 9, 2021

/test pull-kops-e2e-cni-kuberouter

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 9, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: olemarkus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 9, 2021
@k8s-ci-robot k8s-ci-robot merged commit 5a917b5 into kubernetes:master Sep 9, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.22 milestone Sep 9, 2021
@hakman hakman mentioned this pull request Sep 12, 2021
Merged
k8s-ci-robot added a commit that referenced this pull request Sep 12, 2021
@k8s-ci-robot
Merge pull request #12318 from hakman/automated-cherry-pick-of-#12104-#…
d80e56b 
…12291-upstream-release-1.22

Automated cherry pick of #12104: Fix cluster spec typo in CCM integration tests
#12291: Default to latest staging image for AWS CCM
@johngmyers johngmyers modified the milestones: v1.22, v1.23 Oct 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@olemarkus olemarkus olemarkus approved these changes

@johngmyers johngmyers Awaiting requested review from johngmyers

@mikesplain mikesplain Awaiting requested review from mikesplain

Assignees

@olemarkus olemarkus

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/addons cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v1.23
Development

Successfully merging this pull request may close these issues.
5 participants
@rifelpet @hakman @k8s-ci-robot @olemarkus @johngmyers