Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kube-router to v2.1.0 #16440

Merged
merged 3 commits into from
Apr 1, 2024
Merged

Conversation

aauren
Copy link
Contributor

@aauren aauren commented Mar 31, 2024

@hakman for review

Changes:

  • Bump kube-router version to v2.1.0 from v1.6.0
  • Enable hairpin mode for kube-router by default - From the e2e tests this seems to be expected functionality of a CNI
  • Add the runtime endpoint into the container so that user's are able to utilize DSR functionality
  • Mount /etc/iproute2/rt_tables into the container as well which is used by kube-router to configure source routing for DSR
  • Add hostPID: true functionality for DSR
  • Add environment var POD_NAME so that user's can easily enable load-balancing mode for kube-router
  • Add additional RBAC for the following:
    • leases / services/status - For loadbalancer controller which can be optionally enabled
    • endpointslices - This is needed in kube-router v2.X and on so that kube-router is able to handle dual-stack endpoints

I found that in the current main branch of kops that kube-router's old method of adding instance modify permissions to the nodes no longer works correctly. I believe that this permission should be coming in when kube-router is chosen as the networking provider:

However, this no longer seems to be applying to nodes correctly and when this is missing kube-router is not able to forward pod traffic to other nodes.

After poking around a bit, I also noticed that other CNIs like Calico have special logic in the IAM builder like the following to add similar functionality in the IAM node builder code: https://github.com/kubernetes/kops/blob/master/pkg/model/iam/iam_builder.go#L467-L469

So I added this functionality for kube-router as well and found that it works correctly and enables kube-router to disable the src-dest check in AWS.

Additionally removed all of the skip_regex.go exceptions that were kept for kube-router as with this version update kube-router now passes all of the upstream e2e conformance tests.

* Update version v1.6.0 -> v2.1.0
* expose container runtime socket to kube-router
* kube-router loadbalancer controller
  * Add access / information that is needed by the load balancer
    controller in kube-router.
* add access to endpoint slices for services controller
* enable hairpin mode on kube-router
  * There are integration tests in kops that expect that hairpin
    functionality is always enabled, as such we now enable hairpin-mode
    in kube-router by default.
* enable hostPID access
  * kube-router now requires access to the host's process namespace. See
    cloudnativelabs/kube-router#1584 for more
    information.
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 31, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @aauren. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hakman
Copy link
Member

hakman commented Mar 31, 2024

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 31, 2024
@aauren aauren force-pushed the kube-router_v2.X branch from af0a292 to da0ce7c Compare March 31, 2024 17:07
@aauren
Copy link
Contributor Author

aauren commented Mar 31, 2024

I went ahead and re-added the skip regex for the 2 failing kube-router tests TCP close/wait which appears to be flaky as this one isn't failing consistently in my similar test-bed and the kube-proxy URLs one as something appears to have changed in the test package that causes this to fail.

I'll look more into these later, but this should hopefully get it through.

aauren added 2 commits March 31, 2024 13:16
* kube-router enable service.kubernetes.io tests
* kube-router enable load-balancer tests
* kube-router enable endpointslice tests
@aauren aauren force-pushed the kube-router_v2.X branch from da0ce7c to 821ab18 Compare March 31, 2024 18:16
@hakman
Copy link
Member

hakman commented Apr 1, 2024

Nice bump. Thanks @aauren!
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 1, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 1, 2024
@hakman hakman changed the title kube-router v2.1.0 Update kube-router to v2.1.0 Apr 1, 2024
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Apr 1, 2024

@aauren: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-e2e-cni-cilium-ipv6 821ab18 link false /test pull-kops-e2e-cni-cilium-ipv6
pull-kops-e2e-cni-calico-ipv6 821ab18 link false /test pull-kops-e2e-cni-calico-ipv6

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hakman
Copy link
Member

hakman commented Apr 1, 2024

/test pull-kops-e2e-cni-amazonvpc

@hakman
Copy link
Member

hakman commented Apr 1, 2024

/test pull-kops-e2e-k8s-aws-calico

@k8s-ci-robot k8s-ci-robot merged commit 393aac4 into kubernetes:master Apr 1, 2024
29 of 31 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.29 milestone Apr 1, 2024
k8s-ci-robot added a commit that referenced this pull request Apr 1, 2024
…-upstream-release-1.29

Automated cherry pick of #16440: Update kube-router to v2.1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/addons cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants