Infinite recursion when setting up Multus (delegates to itself)

[werdnum]

Trying to install on an arm64 k3s cluster.

Looks like maybe this part of thin_entrypoint should be skipping to the next entry if the first one is Multus?

https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/cmd/thin_entrypoint/main.go#L302-L316

What happened:

Server crashes because of a Multus forkbomb.

Examining the 00-multus.conflist file, I see that Multus is delegating to itself: https://gist.github.com/werdnum/de21d625ae7c4428ed55914e8da9ef0e

What you expected to happen:

Should delegate to the first non-Multus CNI instead of to itself

How to reproduce it (as minimally and precisely as possible):

Install this on a k3s cluster, label one of the nodes with network-implementation: multus https://gist.github.com/werdnum/9f36b4084f39924b74d27824ee410966

Anything else we need to know?:

Environment:

• Multus version

sha256:35accc0395c03a8f0706304d79d670602debf4b31228f9da76b8e415c0523387
https://github.com/k8snetworkplumbingwg/multus-cni/pkgs/container/multus-cni/112115274?tag=snapshot

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3+k3s1", GitCommit:"fe9604cac119216e67f8bd5f14eb5608e3bcd3cf", GitTreeState:"clean", BuildDate:"2023-06-26T15:36:47Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/arm64"}

• Primary CNI for Kubernetes cluster:

Flannel

• OS (e.g. from /etc/os-release):

Ubuntu 22.04

• File of '/etc/cni/net.d/'

udo ls -l /var/lib/rancher/k3s/agent/etc/cni/net.d
total 312
-rw------- 1 root root  24576 Jul 27 22:02 00-multus.conf.new
-rw------- 1 root root 284581 Jul 27 22:02 00-multus.conflist
-rw-r--r-- 1 root root    406 Jul 27 20:34 10-flannel.conflist
drwxr-xr-x 2 root root   4096 Jul 27 22:02 multus.d

• File of '/etc/cni/multus/net.d'

Empty

• NetworkAttachment info (use kubectl get net-attach-def -o yaml)

N/A

• Target pod yaml info (with annotation, use kubectl get pod <podname> -o yaml)

N/A

• Other log outputs (if you use multus logging)

Logs from kube-multus-ds pod: https://gist.github.com/werdnum/dea719c11d9d9c9297d38b9c544fb0e6

[roosmaa]

Am experiencing similar issues, also on k3s (amd64, debian 12). With bitnami/multus-cni helm chart (that installs multus 4.0.2). And primary CNI being Cilium.

Here's a quick summary of what I observed:

1. With the --cleanup-config-on-exit=true argument, I observed the infinite loop scenario like the OP. This loop doesn't seem to have any exit conditions, or blocking statements, so multus goes crazy infinitely delegating to itself.

2. Without the cleanup flag, multus runs fine ... until it gets restarted (or some such) and then it does delegation to itself and breaks all networking.

[werdnum]

I was able to come up with a patch that fixes this (by refusing to delegate to another Multus configuration).

I'll clean it up and send it as a PR in the next day or two.

[ma-sattari]

Hi @werdnum

I have the same problem With bitnami/multus-cni helm chart (that installs multus 4.0.2) when installing to k8s 1.27.3. Will this PR go to snapshot-thick tag or a new version will be released?