Running addons with --privileged

[pschmitt]

Hi.

I'm trying to create an addon for pilight: https://github.com/pschmitt/hassio-addons/tree/master/pilight
Unfortunately for this to work I'd need to run the container with --privileged. AFAIK this is not possible right now. This and --pid=host may be useful for other addons as well (dind, zabbix-agent)

[gollo]

Is the device mapping in config.json (eg "devices": [ "/dev/vchiq:/dev/vchiq:rwm" ]) not sufficient?

[pschmitt]

No. I tried both with --device /dev/mem:/dev/mem:rw and --cad_add SYS_RAWIO. The service starts okay, I can connect to it and all but nothing happens when I connect ask the daemon to issue commands (they are logged as successful - but that's all). Something else must be missing since starting the container with --privileged makes it work as expected.

If you happen to have a RF transmitter you can try it for yourself:

# Do not work
docker run -it --rm --name pilight --device /dev/mem --cap-add SYS_RAWIO pschmitt/hassio-addon-armhf-pilight
docker exec -it pilight pilight-send -p pollin -u 1 -s 1 -t
# Works
docker run -it --rm --name pilight --privileged pschmitt/hassio-addon-armhf-pilight
docker exec -it pilight pilight-send -p pollin -u 1 -s 1 -t

EDIT: Whatever. This issue was more intended to be a feature request / discussion than me trying to get help for my addon

[gollo]

Sure. I guess the next question is why this is an addon and not just a component/platform in main HA (which runs with --privileged). I suppose I'm just establishing a definitive requirement to then base the discussion on. I'm assuming the default position is to avoid having to do this if at all possible (so a 3rd party addon doesn't kill the the entire appliance), but others know more/better than me...

[pschmitt]

I guess pilight could be implemented as a component, but that is a ton of work: just take a look at the source code.
There already is a client component in hass. Why not just use it?

Anyway this --privileged requirement will certainly come up for other addons. Like for example zabbix-agent. Ideally the agent would need to run with --privileged and --pid=host which is not available to addons as well.

[pvizeli]

For zabbix and official docs (https://hub.docker.com/r/zabbix/zabbix-agent/)
docker run --name some-zabbix-agent -v /dev/sdc:/dev/sdc -d zabbix/zabbix-agent:latest

Privileged is only a other word for "I don't know what he need so I give him all right". So we have no privileged options for addon while we work with capability (like capsh can help). At the moment we support 3 but If you need some one, we can add this. That allow us later to give a user a hint what the addon can do on his system.
https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
http://man7.org/linux/man-pages/man7/capabilities.7.html

You need only find the correct cap to run inside docker.

For host pid I will look. We need find a mix that a addon can not damage a system and protect the user and give him aware for addons vs. development of add-ons.

[pvizeli]

For good debain multiarch image look into this:
https://github.com/home-assistant/hassio-addons/blob/master/google_assistant/build.json

[pschmitt]

@pvizeli You missed the "Privileged mode" paragraph in the official documentation of zabbix-agent you linked: https://hub.docker.com/r/zabbix/zabbix-agent/

By default, Docker containers are "unprivileged" and do not have access to the most of host resources. Zabbix agent is designed to monitor system resources, to do that Zabbix agent container must be privileged or you may mount some system-wide volumes

[gollo]

Perhaps a clearer definition of what hassio (and perhaps moreso, addons) is intended to do is required (or may exist, I can't find anything right now). IMHO, hassio is used to create an appliance, plug and play, all user friendly and everything happens automatically (eg updates, including the OS) or via the web interface. Anything that can potentially break this has to be closely controlled/limited, hence the restrictions on addons. Really the hassio box is just homeassistant, and whatever helpers it may need to talk to the outside world, not also a monitoring hub/server (but maybe a client of that, yes.)

I think the zabbix use case isn't really a hassio one. I think you would be better off using standalone homeassistant on a "normal" OS and run zabbix alongside of it, if you desire to host both on the same machine. For now this seems the same would be the best approach for pilight as well (I think it would be great for homeassistant to have closer integration with this, but an add-on for hassio probably isn't the answer.
Maybe @pvizeli can help clarify the hassio scope/vision. @pschmitt you've got the discussion you wanted :-)

[pvizeli]

@pschmitt privileged or you may mount some system-wide volumes I think the or have a big ident there. That mean you can add /dev/sdc:/dev/sdc:rwm to your device lists and it will work like you give privileged access.

Same with pilight, if you know with cap he need (I think SYS_ADMIN) it will be run. The Problem is with privileged Modus it is a very bad praxtic. In some case it could be make sense but it also destroy any things that will be docker stand for it.

I think this discusion is very imported and I'm happy to read other opinions.

[pschmitt]

That does not scale. How am I supposed to pass ALL devices to my agent? When packaging I cannot know the amount or even the path to said devices. Will it be /dev/sda only? /dev/vda? Will there be 5 disks /dev/sd[abcde]?

There is no way to correctly guess how the hassio host will be monitored (what template will be applied)

[pschmitt]

I'd suggest the following: implement --privileged but show a big red warning on the addon page with a corresponding message. "Please contact the developer. Running privileged addons is strongly discouraged"

[pschmitt]

Regarding pilight I managed to get it working using docker run -it --rm --cap-add SYS_RAWIO --device /dev/mem -v /sys:/sys --name pilight pschmitt/hassio-addon-armhf-pilight

Sadly there isn't any way to issue a --mount / --volume command in an addon's config.json.

[pvizeli]

moby/moby#25885

I add a new options gpio to config to allow access to gpio interface with sys but we give not the hole sys free to docker, that is very danger.

I think we should go out at evening with that update and you can use pilight as add-on 👍

[pvizeli]

0.65 with gpio support is out

[pschmitt]

Nice

[pschmitt]

Thanks for the reactivity @pvizeli. The bad news is that pilight seems to require broader access to /sys. I have no idea what it may be using on there. But /sys/class/gpio simply isn't enough :(

EDIT: From my tests it appears to be something within /sys/devices

[pvizeli]

Do you run it with /dev/mem and SYS_RAWIO?

/sys/devices/virtual/gpio was the old place of gpio interface on linux kernel 3.x

[pvizeli]

Can you try that? Also what he need on devices. So I can make a better solution.

For privileged modues. I see PR for docker they will merged in next ce release and dedicate the privileged mode. They switch to a preset like system. I think we can do the same now and defined presets they have the access where we need.

[pschmitt]

Works:
docker run -it --rm --cap-add SYS_RAWIO --device /dev/mem -v /sys:/sys --name pilight pschmitt/hassio-addon-armhf-pilight

Does not:
docker run -it --rm --cap-add SYS_RAWIO --device /dev/mem -v /sys/class/gpio:/sys/class/gpio --name pilight pschmitt/hassio-addon-armhf-pilight

I'll try to figure out what is needed

[pschmitt]

Got it! It's /sys/devices/platform/soc

docker run -it --rm --cap-add SYS_RAWIO --device /dev/mem -v /sys/class/gpio:/sys/class/gpio -v /sys/devices/platform/soc:/sys/devices/platform/soc --name pilight pschmitt/hassio-addon-armhf-pilight

[pvizeli]

Okay. I will not make it hard to implement things like this. I will add a user-friendly or developer friendly access system.

raspberrypi/linux#1117

I make fix and release it in 3-4 hours. Thanks to help for make it better.

[pvizeli]

Short question for your monitor software problem. Do work:
-v /dev:/dev:ro

? So you have all device inside container but no privileged rights. Maybe you need SYS_ADMIN too

[pschmitt]

Bind-mounting /dev would have a ton a side-effects since this is a special dir, both for the host and the container. It'd certainly break /dev/stdin and /dev/stdout since these are supposed to be writable.

I tried that nonetheless and as expected the container didn't even start:

root@hassistant:~# docker run -it --rm -v /dev:/dev -p 10056:10050 --cap-add SYS_ADMIN pschmitt/hassio-addon-armhf-zabbix-agent
docker: Error response from daemon: invalid character 'c' looking for beginning of value.

[pvizeli]

True. I spoke with paulus and we add a user device selector for add-ons. That will be control over API. I hope we find a UI developer that have time to make that nice on it.

That make you able to modify the device list dynamic. So you can provide a add-on with basic and imported devices for runtime. Later can the user also add more device (in your case for monitoring) that he want to control with it.

But I look too for more solutions

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GabrielePicco]

I have the same problem with a Wireguard client add-on that i trying to build. Any idea on how to solve it ?

For the Wireguard client the error is: Setting key "net.ipv6.conf.default.forwarding": Read-only file system.

I added in the configuration

"kernel_modules": true,
"full_access": true, 

but it is not equal to running the container with --privileged

[frenck]

You can't do that @GabrielePicco, as was already discussed above.
Furthermore, this issue has been closed for 11 months already. Please don't drag up old issues, instead, just create new ones.

Thanks!

[GabrielePicco]

I know I can't run an add-on with --privileged. But it seemed pertinent to the discussion for the alternative solutions proposed in this issue. In any case, I will specify it better in the new issue.

Thanks