Does "containerized socket activation" improve security?

[eriksjolund]

The live video streaming and chat server software owncast makes use of two listening TCP sockets (one for the web server and one for RTMP). Using socket activation for multiple different sockets requires the use of the environment variable LISTEN_FDNAMES to distinguish the different of sockets from each other. I wanted to try it out, so I created a feature request and wrote a PR. The functionality seems to work. (I've only verified it by checking that the web server provides a web page when accessing it).

Unfortunately the feature request was closed with the motivation that there is no need for the requested feature.
Now I wonder if I maybe should have put more emphasis on the security advantage of using containerized socket activation.
(At least to me there seems to be a security advantage).

This was one of the rationale arguments I provided in the feature request:

If you have a containerized server that takes requests from the network via a listening TCP port, adding socket activation support opens up the possibility to run the container with the --network=none command-line option. This will restrict the container from accessing the network except via the TCP socket it has been passed by systemd via Podman. Outgoing connections would not be allowed. Running the container with --network=none improves security. I'm not sure if this is relevant in the case of owncast though.

(I hope that was a correct description of how it works)

Has anyone given any thoughts to how "containerized socket activation" relates to security? Any pros and cons?

A side-note regarding SELinux:
container-selinux needs to be at least version v2.179.0
to avoid having to pass the option --security-opt label=disable to podman run.

By the way, what is a suitable name for the functionality?
"containerized socket activation" or maybe better "socket activation to container" or something else?

[rhatdan]

I like the way you talk about this from a security point of view. The --network=none with socket activation seems like a great idea.

I could see this being useful with a display only webserver for example. Where you could allow incoming connections not not allow connections back out. But this would only work with TCP connections.

[eriksjolund]

@rhatdan Thanks for the feedback!

But this would only work with TCP connections.

I'm not sure how to parse that sentence. Could you explain a bit more? Do you mean TCP in comparison to Unix sockets, or maybe TCP in comparison to UDP?

[rhatdan]

Yes TCP, and Unix Domain Sockets would work, but not udp, I believe.

[eriksjolund]

I made a quick test with ListenDatagram=127.0.0.1:3000 and a socket activated container. It worked with --network=none.

I should go through the test more carefully to see that I have not done any mistake.

[eriksjolund]

New plan: I'll publish a minimal demo about using UDP but it might take a few weeks.

[rhatdan]

If you could just give a few examples of each use case, we could write a cool blog on it.
Unix Domain Socket, TCP Socket, UDP Socket.

[rhatdan]

If you have examples of each, I will write up the Blog.

[eriksjolund]

Sure, that would be cool! Right now I have an echo server that upcase the string it receives

TCP

$ echo hello | socat - tcp:127.0.0.1:3000
HELLO

UDP

$  echo hello | socat -t 5 - udp:127.0.0.1:3000
HELLO

I'll send you something during the weekend.

[rhatdan]

Yes a demo showing a leaked socket working and then exec into the container and showing that you can not connect out using curl or point ...

[eriksjolund]

@rhatdan See
https://github.com/eriksjolund/asio/blob/systemd_socket_activation/socket_activation.md

(I hope to clean the code up a bit). I wrote you an email too (from erik.sjolund@gmail.com).

[rhatdan]

Thanks, this is looking really good. I updated containers-selinux to allow socket activation for tcp_sockets and udp_sockets from systemd and users including unconfined_t.

[eriksjolund]

Thanks, after upgrading to container-selinux-2.181.0-1.fc35.noarch from https://bodhi.fedoraproject.org
systemd-socket-activate now works without --security-opt label=disable!

Some changes:
I've

• deleted https://github.com/eriksjolund/asio and instead created the new repository https://github.com/eriksjolund/socket-activate-echo where I put the source code.
• added a GitHub Actions workflow that builds the multi-arch container image ghcr.io/eriksjolund/socket-activate-echo (archs: linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7)
• modified the documentation and the service file to make use of that container image.
• added a troubleshooting section where I discussed the socat timeout configuration and the possibility of using --pull=never and pulling the image beforehand.

[rhatdan]

Excellent, now time to throw together a blog on running containers more securely by turning off network access, but using systemd socket activation to allow limited network connectivity.

[eriksjolund]

Thanks!

[eriksjolund]

Instead of redhat.com/sysadmin, do you think it would be possible to add the document under
https://github.com/containers/podman/tree/main/docs/tutorials
and https://docs.podman.io/en/latest/Tutorials.html
?
(I thought maybe the document will then be used and reused more)
I created a branch where I made the document to be more of a tutorial and less of a blog. (I understand a proper review process would need to follow before the document could land upstream). The document is not quite ready yet but for the most part.

[rhatdan]

I would prefer it in both. sysadmin gets eyeballs. While adding it to tutorials would be useful for others to find it in the future.

One would be written from a security point of view for sysadmin, the other would be to explain how to use socket activation in containers.

[eriksjolund]

SGTM, I'll split it into two. I'l try to do it over the weekend.

[eriksjolund]

I split it into 3 documents

1. https://github.com/eriksjolund/podman-socket-activation/blob/rework_text/blog.md
   Draft blog text for redhat/sysadmin

2. https://github.com/eriksjolund/podman-socket-activation/blob/rework_text/README.md
   Possible starting point for a podman.io tutorial.

3. https://github.com/eriksjolund/podman-socket-activation/blob/rework_text/RestrictAddressFamilies.md
   Probably in the unsupported-use-of-Podman-territory, so I guess I could put the text somewhere locally

[eriksjolund]

I created another example
https://github.com/eriksjolund/socket-activate-httpd
(A web server that can't establish outgoing connections)

Yes, I agree, time for a blog. Tell me if you would like help.

[eriksjolund]

Edit 1

Another reflection regarding using podman and socket activation:

The data communication in the socket does not go through slirp4netns when running rootless Podman.
Hopefully this means better network performance (throughput).

I haven't done any benchmarking to verify this theory though.

I opened up a new discussion thread about the network performance

Edit 2: fixed link 2 September 2024

Does "containerized socket activation" improve network performance?

[zeronumbers]

I believe you wrote wrong link since it points to this very page where we are now, the one you wanted to link is probably: #14068

I didn't read whole discussion, and am still learning what this socket stuff is, but I guess another important question is whether "containerized socket activation" could be exploited in any way, or there are literally no downsides (for security) guaranteed?

[eriksjolund]

Thanks, for catching this. I've now fixed the link.

Here are some other links. I wrote two blog posts:

• How to limit container privilege with socket activation
• How to restrict network access in Podman with systemd

I didn't read whole discussion, and am still learning what this socket stuff is, but I guess another important question is whether "containerized socket activation" could be exploited in any way, or there are literally no downsides (for security) guaranteed?

SELinux didn't allow containerized socket activation at first, but after relaxing the rules somewhat, it now works.

• Fail early when "socket activation" provides a Unix socket and SELINUX is active and there is no --security-opt label=disable container-selinux#171
• SELinux prevents passing TCP sockets via Podman/systemd socket activation container-selinux#172
• SELinux prevents passing TCP and UDP sockets via systemd-socket-activate/podman (socket activation) container-selinux#174
• Socket activation AF_VSOCK (SOCK_STREAM): Client on the host, server on the host container-selinux#175
• Socket activation AF_VSOCK (SOCK_STREAM): Client on the host, server in a VM container-selinux#176
• systemd-socket-activate problem. audit2allow shows "allow container_t unconfined_t:tcp_socket { setopt shutdown };" container-selinux#179

Maybe this could be seen as a downside, although I don't see it that way.