Losing contact with an instance after a while

[thehappycoder]

Describe the bug
I've installed Kubernetes on 3 instances on an Apple M1 machine, and after connecting these instances in a Kubernetes cluster via kubeadm join, I lost contact to all 3 instances.

I've managed to regain access by restarting multipass:

sudo launchctl unload /Library/LaunchDaemons/com.canonical.multipassd.plist && sudo launchctl load /Library/LaunchDaemons/com.canonical.multipassd.plist

But then the instances freeze after a while. It also happens when I run only 1 instance.

Logs
I get this in /Library/Logs/Multipass/multipassd.log when I lose contact with my instances:

[2022-03-31T07:28:21.508] [debug] [k8s-worker2] QMP: {"timestamp": {"seconds": 1648675701, "microseconds": 508459}, "event": "NIC_RX_FILTER_CHANGED", "data": {"path": "/machine/unattached/device[7]/virtio-backend"}}

Additional info
Apple M1
macOS 12.3
multipass 1.8.1+mac

[Saviq]

Hi @thehappycoder, we've not seen this before I don't think. This message suggests the kernel reinitializes the virtual network (you should see it when it's first booting as well). I was actually unable to get that message even unloading the network module, are you certain the message only shows when networking goes down?

If you launch an instance without k8s, does it happen as well?

[thehappycoder]

@Saviq

Yes, it may have something to do with k8s running or some related commands I ran on an instance because a brand new instance seems to be not affected with the freeze happens. Here are the commands that I ran 1643991933135-Building a Kubernetes Cluster.pdf

Also I am not sure if it's normal or not, but memory usage of qemu processes seems suspiciously high for some instances

.

[Saviq]

@thehappycoder there's a lot of networking foo happening there, it's quite possible something there is messing it up.

You could try using a bridged network for the k8s bits, I just posted a pre-release package here: #2364 (comment)

I'll close this, then.

[Saviq]

Ah @thehappycoder on memory use, I recently explained some of it here:

#2494 (comment)

[thehappycoder]

Turns out something in the software running on these virtual machines is messing up SSH because I also experience this when I connect to virtual machines via SSH that are running in UTM.