Use k3s with selinux set to enforce

[mak3r]

Describe the bug
We would like to set SELinux to enforce with k3s but are unable to get this working.

To Reproduce
I created a config.toml.tmpl to include the Boolean to enable it in containerd.

worker# grep selinux k3s/agent/etc/containerd/config.toml
enable_selinux = true

My nginx deployment has an seLinuxOptions.

grep -A5 securityContext nginx-deployment.yaml

securityContext:
seLinuxOptions:
  user: system_u
  role: object_r
  type: container_t
  level: s0:c100,c200

But when the pods run, they wind up using the spc_t (super privileged container SELinux type). All pods seem to run with spc_t unfortunately.

worker# ps -efZ | grep nginx

unconfined_u:system_r:spc_t:s0-s0:c0.c1023 root 21234 21217  0 10:08 ? 00:00:00 nginx: master process nginx -g daemon off;

unconfined_u:system_r:spc_t:s0-s0:c0.c1023 101 21326 21234  0 10:08 ?  00:00:00 nginx: worker process

kubectl exec nginx-5499b86ff-r7pvv -- runcon

unconfined_u:system_r:spc_t:s0-s0:c0.c1023

Host where the pod is running is setup with SELinux.

worker# getenforce
Enforcing

Expected behavior
I would have expected the container_t type for running pods. If I switch to use Docker, pods have problems launching. They complain about CNI and net namespace. If I change SELinux to be permissive, then pods launch using Docker and show up with the expected container_t type.

[westurner]

See: "Add support for SELinux" #1372

[davidnuzik]

PR: #1411

[erikwilson]

Merged #1448

[westurner]

Thanks!

…

On Mon, Feb 24, 2020, 6:52 PM Erik Wilson ***@***.***> wrote: Merged #1448 <#1448> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#533>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMNSZ3EEBXRSYGT5Q2S5TRERMS5ANCNFSM4HYHLKWQ> .

[westurner]

- [ ] remove the "Set SELinux to disabled state" task from the ansible

prereq role: #1372 (comment) Is this safe to do yet?

…

On Mon, Feb 24, 2020, 7:49 PM Wes Turner ***@***.***> wrote: Thanks! On Mon, Feb 24, 2020, 6:52 PM Erik Wilson ***@***.***> wrote: > Merged #1448 <#1448> > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#533>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAMNSZ3EEBXRSYGT5Q2S5TRERMS5ANCNFSM4HYHLKWQ> > . >

[Fodoj]

See #1666

[ShylajaDevadiga]

Verified the issue is resolved.
k3s version v1.18.6+k3s1 (6f56fa1)

k3s/agent/etc/containerd/config.toml has selinux enabled by default and correct labels are applied.

 cat /etc/redhat-release 
CentOS Linux release 7.8.2003 (Core)

getenforce
Enforcing

cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml  |grep enable
  enable_selinux = true

kubectl get pods -A |grep nginx
default       nginx-deployment-6b474476c4-2tw5j        1/1     Running     0          14m
default       nginx-deployment-6b474476c4-9wq4q        1/1     Running     0          14m

ps -eZ |grep nginx
system_u:system_r:container_t:s0:c418,c611 19205 ? 00:00:00 nginx
system_u:system_r:container_t:s0:c147,c840 19226 ? 00:00:00 nginx

Redhat 7.8

cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.8 (Maipo)

ps -eZ|grep nginx
system_u:system_r:container_t:s0:c237,c399 30433 ? 00:00:00 nginx
system_u:system_r:container_t:s0:c237,c399 30457 ? 00:00:00 nginx

kubectl get pods 
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6b474476c4-hnb28   1/1     Running   0          28m
nginx-deployment-6b474476c4-mtx5h   1/1     Running   0          28m