[spire and k8s] hostpath required is a security issue

[raffaelespazzoli]

hello, I'm looking at spire as a way to attestate the identity of pod workloads. But I see that mounting a hostpath volume is required for spire to work on kubernetes. This is a problem in k8s as hostpath opens the system to vulnerability and should be used only for trusted workloads (agents etc...). so we have a casse where in order to add more security on the identity of the workload we decrease the security of the system (in particular we open to workload to node attacks).
I wonder is anyone is thinking about how to fix this issue.
In istio we had a similar problem that was fixed with the introduction of a CNI plugin to execute the privileged commands (in this case mounting the host socket). Maybe a similar approach could be used here.

[mchurichi]

Hi @raffaelespazzoli, there is a SPIFFE k8s CSI driver that provides a mechanism to inject the agent API into workload pods.

We definitely need to document this better. Thanks for bringing it up, and hope it helps.

[raffaelespazzoli]

understood, thanks. The next improvement could be to inject the spire volume based on an annotation instead of making the user author the pod spec. Any thoughts?

[raffaelespazzoli]

I installed this csi driver in OpenShift and it does not work. I can see the unix socket mounted in the container, but I get this error:

drwxr-xr-x.   2 root root   60 Oct 29 20:28 spiffe-workload-api
drwxr-xr-x.   2 root root    6 Jun 21  2021 srv
dr-xr-xr-x.  13 root root    0 Oct 29 19:14 sys
drwxrwxrwx.   2 root root   58 Oct 26 11:23 tmp
drwxr-xr-x.  12 root root  144 Oct 26 11:09 usr
drwxr-xr-x.  19 root root  249 Oct 26 11:09 var
sh-4.4$ ls -la spiffe-workload-api
ls: cannot open directory 'spiffe-workload-api': Permission denied
sh-4.4$ id
uid=1000690000(1000690000) gid=0(root) groups=0(root),1000690000

as you can see, based on the group, I should be able to ls that directory. I suspect selinux might be in play here. Any suggestions?

[raffaelespazzoli]

also this is failing:

/opt/spire $ /opt/spire/bin/spire-agent api fetch -socketPath $SPIFFE_ENDPOINT_SOCKET
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /opt/spire/unix:/spiffe-workload-api/spire-agent.sock: connect: no such file or directory"

[mchurichi]

Hi @raffaelespazzoli, would you mind opening an issue in the CSI driver repository? if you could also include the relevant manifests would be great

[azdagron]

If you are using SELinux, make sure you are using v0.2.0 of the SPIFFE CSI driver and that the volume itself is marked readonly inside the PodSpec.

Example:
https://github.com/spiffe/spiffe-csi/blob/17c8e758441770822cccf3e717d3b9a985d0dd9a/example/config/workload.yaml#L21

[raffaelespazzoli]

no one is answering on the csi driver project, but I found proof that selinux is denying access:

time->Fri Nov  4 17:52:55 2022
type=PROCTITLE msg=audit(1667584375.484:68): proctitle=6C73002D6C61002F7370696666652D776F726B6C6F61642D617069
type=SYSCALL msg=audit(1667584375.484:68): arch=c000003e syscall=2 success=no exit=-13 a0=7ffedde6edda a1=98000 a2=0 a3=0 items=0 ppid=266580 pid=266767 auid=4294967295 uid=1000680000 gid=0 euid=1000680000 suid=1000680000 fsuid=1000680000 egid=0 sgid=0 fsgid=0 tty=pts0 ses=4294967295 comm="ls" exe="/bin/busybox" subj=system_u:system_r:container_t:s0:c15,c26 key=(null)
type=AVC msg=audit(1667584375.484:68): avc:  denied  { read } for  pid=266767 comm="ls" name="agent-sockets" dev="tmpfs" ino=1257099 scontext=system_u:system_r:container_t:s0:c15,c26 tcontext=system_u:object_r:container_var_run_t:s0 tclass=dir permissive=0