Inter pod / Node connectivity issue with Flannel 0.15 and Centos 8 Machine.

[tittuvarghese]

I have a running K8 cluster of version v1.22.2 on Centos 8, 4.18.0-193.14.2.el8_2.x86_64. Recently I tried to update my kernel and other dependencies with yum update. Kernel go updated to 4.18.0-305.19.1.el8_4.x86_64.

After this the inter pod communication stopped working. In TCPDUMP I can see traffic is reaching to the nodes, but it's not going beyond that.

If I revert the kernel version to older one, it's working smoothly, without any issues.

I'm using vxlan as backend for flannel.

Expected Behavior

1. Inter pod communication should happen.
2. DNS resolution should be working without any issues.

Current Behavior

1. Response for dns resolution on Kernel version - 4.18.0-305.19.1.el8_4.x86_64

kubectl exec -it -n dnstest dnsutils -- nslookup kubernetes.default
;; connection timed out; no servers could be reached

2. Response for dns resolution on Kernel version - 4.18.0-193.14.2.el8_2.x86_64

kubectl exec -it -n dnstest dnsutils -- nslookup kubernetes.default
Server:         172.89.0.10
Address:        172.89.0.10#53

Name:   kubernetes.default.svc.cluster.local
Address: 172.89.0.1

No changes other than rebooting from the old kernel.

Possible Solution

1. I don't think it's a DNS issue, as the same setup works if I'm using the old kernel version.

Your Environment

• Flannel version: v0.15
• Backend used (e.g. vxlan or udp): vxlan
• Kubernetes version (if used): v1.22.2
• Operating System and version: Centos 8

[chris93111]

@tittuvarghese do you have found solution ? i have same problem in rhel 8.4

k3s-io/k3s#5013

i use k3s 1.22.5 and flannel 0.15.1 is embedded, i have already the problem

[chris93111]

confirm downgrade to 4.18.0-193.13.2.el8_2.x86_64 work , i thinks they are problem with module nf_conntrack on > 4.18.0-193.
module nf_conntrack_ipv4 is only present in 4.18.0-193

[tittuvarghese]

Downgrade worked for me as well.

[komuta]

This is likely due to #1155, which has been fixed in v0.15.1.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.