Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce configuration changes for the ipv6 support #673

Merged
merged 4 commits into from
Feb 3, 2021

Conversation

mikhail-sakhnov
Copy link
Contributor

@mikhail-sakhnov mikhail-sakhnov commented Jan 29, 2021

Pass CIDR as separate ENV variables for calico-node
Change default node-cidr-mask-size-ipv6 to be a smaller size value
Enable CALICO_IPV6POOL_NAT_OUTGOING for dualstack setup
Change kube-proxy to ipvs in case of dual-stack setup

closes #394

The changes here enable to create ipv6\ipv4 dualstack setup

Limitations:

  • Connections to the outer world work (e.g. curl -6 ipv6.google.com will work from any pod)
  • It seems like currently k8s doesn't assign internal ipv6 address automatically on bare-metal, but could be done by workaround with external cloud provider and manual node status update as described in No way to configure ExternalIP addresses for a node with kubelet kubernetes/kubernetes#42125
  • each pod gets ipv6 address from the CNI
  • cross pod connectivity exists across nodes

Additional links used during this PR:
https://www.projectcalico.org/dual-stack-operation-with-calico-on-kubernetes/
https://docs.projectcalico.org/networking/ipv6
https://kubernetes.io/docs/tasks/network/validate-dual-stack/ (this one mentions how to create dual\single stack services)
https://kubernetes.io/docs/concepts/services-networking/dual-stack/

In case of testing this on AWS the source\destination checks must be disabled on the interface.

k0s.yaml used:

spec:
  network:
    calico:
      mode: "bird"
    dualStack:
      enabled: true
      IPv6podCIDR: "fd00::/108"
      IPv6serviceCIDR: "fd01::/108"

@mikhail-sakhnov mikhail-sakhnov requested a review from a team as a code owner January 29, 2021 16:44
@mikhail-sakhnov mikhail-sakhnov requested a review from kke January 29, 2021 16:44
@mikhail-sakhnov mikhail-sakhnov changed the title WIP: Introduce configuration changes for the ipv6 support Introduce configuration changes for the ipv6 support Feb 2, 2021
@mikhail-sakhnov mikhail-sakhnov force-pushed the ipv6-support branch 3 times, most recently from ff54708 to 31aef81 Compare February 2, 2021 10:32
jasmingacic
jasmingacic previously approved these changes Feb 2, 2021
Copy link
Member

@jnummelin jnummelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few minor tweaks needed

docs/dual-stack.md Show resolved Hide resolved
pkg/apis/v1beta1/dualstack.go Outdated Show resolved Hide resolved
pkg/apis/v1beta1/dualstack.go Outdated Show resolved Hide resolved
pkg/apis/v1beta1/network.go Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
docs/dual-stack.md Outdated Show resolved Hide resolved
.github/workflows/go.yml Outdated Show resolved Hide resolved
@unixfox
Copy link

unixfox commented Feb 2, 2021

Could you allow dual stack mode for custom network provider? Currently, I can't do that due to this error: network dual stack is supported only for calico mode bird.

My config:

network:
    provider: custom
    dualStack:
      enabled: true
      IPv6podCIDR: "fd00::/108"
      IPv6serviceCIDR: "fd01::/108"

EDIT: I got it working by specifying bird calico config even though I'm using a custom network provider. Like this:

network:
    provider: custom
    dualStack:
      enabled: true
      IPv6podCIDR: "fd00::/108"
      IPv6serviceCIDR: "fd01::/108"
    calico:
      mode: "bird"

But that's not the ideal solution.

unixfox
unixfox previously approved these changes Feb 2, 2021
Copy link

@unixfox unixfox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great! Thank you for your work!

I tested:

  • reaching the exposed port of a nodeport from a pod using the nginx image
  • pod interconnectivity with pods in other nodes
  • connectivity to the internet from a pod

unixfox
unixfox previously approved these changes Feb 2, 2021
pkg/apis/v1beta1/network.go Outdated Show resolved Hide resolved
@mikhail-sakhnov
Copy link
Contributor Author

pported only for calico mode bird

@unixfox should be fixed in latest version

Mikhail Sakhnov added 3 commits February 3, 2021 13:25
Pass CIDR as separate ENV variables for calico-node
Change default node-cidr-mask-size-ipv6 to be a smaller size value
Enable CALICO_IPV6POOL_NAT_OUTGOING for dualstack setup
Change kube-proxy to ipvs in case of dual-stack setup

Signed-off-by: Mikhail Sakhnov <msakhnov@mirantis.com>
Add documentation

Signed-off-by: Mikhail Sakhnov <msakhnov@mirantis.com>
Fix unit tests for kubelet profile
Go mod tidy
Signed-off-by: Mikhail Sakhnov <msakhnov@mirantis.com>
…ult values

Signed-off-by: Mikhail Sakhnov <msakhnov@mirantis.com>
@mikhail-sakhnov
Copy link
Contributor Author

@unixfox thank you for the help with testing things!

@mikhail-sakhnov mikhail-sakhnov merged commit f8fa0ba into k0sproject:main Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for dual-stack Kubernetes cluster
4 participants