Support Docker Desktop Kubernetes (Mac)

[mldevpants]

Is your feature request related to a problem? Please describe.

The main caretta pod cannot be started with the following error:
2023/03/26 06:45:23 Warning: couldn't retrieve the owner of docker-desktop - Unsupported kind for lookup - Node. This might happen when starting up
2023/03/26 06:45:33 Couldn't load probes - error loading BPF objects from go-side. field HandleSockSetState: program handle_sock_set_state: apply CO-RE relocations: load kernel spec: no BTF found for kernel version 5.15.49-linuxkit: not supported

Describe the solution you'd like
A clear and concise description of what you want to happen.
I'd like caretta to run under Docker-Dekstop Kubernetes in MacOS
My HW:
CPU - Apple M1 Max
Arch: ARM64
OS: MacOS Monterey 12.6.3

Describe alternatives you've considered
Kind support, cloud integration, I am testing a lot of stuff in my local environment and I'd like caretta to run under the same e environment, first to test it and then use it of course.

Additional context
This project looks awesome, introduced by company's Dev team in KDC2023.

Thanks

[udiz]

hey @mldevpants, thanks for the feedback!

Right now, as the log message says, Docker on Mac isn't supported because the linuxkit image does not provide BTF support. Linuxkit is the linux VM Docker on Mac is running, and its kernel is the kernel used by the containers.
There's an open issue linuxkit/linuxkit#3755 on linuxkit to enable BTF but with no roadmap currently.

We're looking into a possible workaround for Caretta without BTF, specifically on linuxkit, but currently the best alternative would be to run another linux VM to run the containers on (it shouldn't have a performance impact as the default Docker on Mac is already using a VM).