Network Policy issue with deploying gotk with k3s

[onedr0p]

I recently chatted with @stefanprodan and he came to the conclusion that k3s might behave differently than k8s in terms of network policies.

k3s version: 1.19.2-k3s1

Basically the bootstrap process times out.

◎ waiting for cluster sync
✗ context deadline exceeded

I was able to get this to work with Calico and kube-proxy by removing the gotk network policy kubectl delete networkpolicy/deny-ingress -n gotk-system

However it didn't work on my tests when kube-proxy was disabled when using Calico w/ eBPF or Cilium and kube-router.

More information may be be found here in the slack channel.

Related: k3s-io/k3s#1308 and k3s-io/k3s#947 which also links to a kube-router issue.

[onedr0p]

@stefanprodan awesome! I'll give this a shot and report back if I see any issues with k3s

[stefanprodan]

You need to use the v0.1.4 CLI, it should be available in a couple of minutes

[onedr0p]

Seems like my issue was Cilium and setting masquerade to false in their helm chart values, along with the default network policy that is set. I don't know why this would have an effect on gotk completing the bootstrap process or not. I have turned back on masquerading AND disabled the network policy. gotk finishes the bootstrap command, all is well.

https://docs.cilium.io/en/v1.8/concepts/networking/masquerading/

Such a strange issue that took me trying out so many different things.