draft: feat: Run as non-root by default to allow easier deployment on the restricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed #308

blackandred · 2023-03-03T10:24:18Z

Hi,

The container does not necessarily start as root and then switches permissions to nginx user.
I simplified it by starting everything as nginx from the beginning.
Additionally I reduced the end image size and memory usage (~20 MB less) by migrating from supervisord to multirun which is a container-native init process, that does not depend on Python.

Benefits:

Increased security by decreasing the privileges
Less memory usage (by ~20 MB)
Less image size (398MB -> 348MB for all-in-one variant)
Better compatibility with restricted environments (OpenShift, Kubernetes)
Less attack surface by removing supervisord and Python

I was testing it initially with docker-compose, I will test also on Kubernetes and will then "undraft" the PR. I'm creating the PR at this moment to know what do you think about such a big change.

…stricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed

blackandred · 2023-03-03T10:38:01Z

This may probably resolve issues like #194, because no any process will be running with other privileges than the application, so there will be no chance of accidentially creating a file as root.

…h proper permissions / mriedmann#308

ArchBlood · 2023-03-03T23:03:29Z

Dockerfile

@@ -59,7 +59,7 @@ ARG RUNTIME_DEPS="\
    php7-xmlwriter \
    php7-zip \
    sqlite \
-    supervisor \
+    multirun \


I'm not sure if HumHub has been fully tested with multirun, you should probably run some tests before switching.

https://docs.humhub.org/docs/admin/asynchronous-tasks/

In this case crond is used instead of supervisor.

blackandred · 2023-03-04T07:26:11Z

After some more testing I see volume mount points are having non-deterministic permissions, sometimes I get nginx:nginx and other times (more often) I get 101:nginx (why 101?).

Here is my testing script:

HUMHUB_VERSION=1.13.0

build-prod:
	docker build . --target humhub_nginx --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:stable-nginx
	docker build . --target humhub_phponly --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:stable-phponly

rm-prod:
	docker-compose -f docker-compose.prod.yml rm -s -v -f
	docker volume rm -f humhub-docker_assets humhub-docker_modules humhub-docker_mysql humhub-docker_themes humhub-docker_uploads

run-prod: build-prod rm-prod
	docker-compose -f docker-compose.prod.yml up

build-aio:
	docker build . --target humhub_allinone --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:latest

make run-prod

ArchBlood · 2023-03-04T07:34:45Z

After some more testing I see volume mount points are having non-deterministic permissions, sometimes I get nginx:nginx and other times (more often) I get 101:nginx (why 101?).

Here is my testing script:

HUMHUB_VERSION=1.13.0

build-prod:
	docker build . --target humhub_nginx --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:stable-nginx
	docker build . --target humhub_phponly --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:stable-phponly

rm-prod:
	docker-compose -f docker-compose.prod.yml rm -s -v -f
	docker volume rm -f humhub-docker_assets humhub-docker_modules humhub-docker_mysql humhub-docker_themes humhub-docker_uploads

run-prod: build-prod rm-prod
	docker-compose -f docker-compose.prod.yml up

build-aio:
	docker build . --target humhub_allinone --build-arg HUMHUB_VERSION=${HUMHUB_VERSION} -t docker.io/mriedmann/humhub:latest

make run-prod

Is it running on a reverse proxy?

blackandred · 2023-03-04T07:35:27Z

Yes, I'm testing the variant with separate nginx and php-fpm if you are asking abut this.

ArchBlood · 2023-03-04T07:50:03Z

Yes, I'm testing the variant with separate nginx and php-fpm if you are asking abut this.

Are there any noticeable differences in the logs between ngnix:nginx and 101:nginx after each run?

From my understanding of HumHub itself, it can be picky with reverse proxies. 🤔

blackandred · 2023-03-04T07:55:30Z

Yeah, with nginx:nginx it starts. With 101:nginx the installation fails for assets directory.

blackandred · 2023-03-04T08:06:28Z

On all-in-one variant it looks pretty fine I see, at least in docker-compose, I will have to check on Kubernetes on CRI-O.

$ docker volume ls
DRIVER    VOLUME NAME
$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Startup logs

humhub_1  | /docker-entrypoint.sh: Creating database...
humhub_1  | /docker-entrypoint.sh: Installing...
humhub_1  | total 776
humhub_1  | drwxr-xr-x    1 nginx    nginx         4096 Mar  3 05:56 .
humhub_1  | drwxr-xr-x    1 root     root          4096 Mar  3 05:56 ..
humhub_1  | -rw-rw-r--    1 nginx    nginx          413 Dec 21 16:01 .codeclimate.yml
humhub_1  | -rw-rw-r--    1 nginx    nginx         1588 Dec 21 16:01 .htaccess.dist
humhub_1  | -rw-rw-r--    1 nginx    nginx          312 Dec 21 16:01 .php_cs.dist
humhub_1  | -rw-rw-r--    1 nginx    nginx       123496 Dec 21 16:01 CHANGELOG.md
humhub_1  | -rw-rw-r--    1 nginx    nginx         1889 Dec 21 16:01 CONTRIBUTING.md
humhub_1  | -rw-rw-r--    1 nginx    nginx         5973 Dec 21 16:01 Gruntfile.js
humhub_1  | -rw-rw-r--    1 nginx    nginx          328 Dec 21 16:01 LICENSE
humhub_1  | -rw-rw-r--    1 nginx    nginx        34524 Dec 21 16:01 LICENSE.AGPL-3.0-or-later
humhub_1  | -rw-rw-r--    1 nginx    nginx          832 Dec 21 16:01 README.md
humhub_1  | -rw-rw-r--    1 nginx    nginx          240 Dec 21 16:01 SECURITY.md
humhub_1  | drwxrwxr-x    2 nginx    users         4096 Mar  4 08:02 assets
humhub_1  | -rw-rw-r--    1 nginx    nginx         5253 Mar  2 21:46 composer.json
humhub_1  | -rw-rw-r--    1 nginx    nginx       420847 Dec 21 16:01 composer.lock
humhub_1  | -rw-rw-r--    1 nginx    nginx         1331 Dec 21 16:01 index-test.php
humhub_1  | -rw-rw-r--    1 nginx    nginx          876 Dec 21 16:01 index.php
humhub_1  | -rw-rw-r--    1 nginx    nginx       102926 Mar  2 21:46 package-lock.json
humhub_1  | -rw-rw-r--    1 nginx    nginx          433 Mar  2 21:46 package.json
humhub_1  | drwxr-xr-x    1 nginx    nginx         4096 Mar  2 21:17 protected
humhub_1  | -rw-rw-r--    1 nginx    nginx           23 Dec 21 16:01 robots.txt
humhub_1  | drwxrwxr-x    9 nginx    nginx         4096 Mar  2 21:45 static
humhub_1  | drwxrwxr-x    1 nginx    nginx         4096 Mar  2 21:45 themes
humhub_1  | drwxrwxr-x    4 nginx    users         4096 Mar  4 08:02 uploads

(...)

humhub_1  | Validating: User Module - ContentContainer (1 entries)
humhub_1  | Validating: User Module - Users (1 entries)
humhub_1  | Validating: User Module - Password (1 entries)
humhub_1  | Validating: User Module - Profile (1 entries)
humhub_1  | Validating: User Module - Mentioning (0 entries)
humhub_1  | Validating: User Module - Follow (0 entries)
humhub_1  | Validating: User Module - Content container (1 entries)
humhub_1  | 
humhub_1  | *** All integrity checks done
humhub_1  | 
humhub_1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/60-nginx-config.sh
humhub_1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/70-nginx-allowedips.sh
humhub_1  | Setting HUMHUB_REVERSEPROXY_WHITELIST
humhub_1  | Added 127.0.0.1 to Reverseproxy Whitelist
humhub_1  | /docker-entrypoint.sh: Configuration complete; ready for start up
humhub_1  | /docker-entrypoint.sh: Entrypoint finished! Launching ...
humhub_1  | crond: crond (busybox 1.34.1) started, log level 8
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: using the "epoll" event method
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: nginx/1.20.2
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: OS: Linux [redacted]
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: getrlimit(RLIMIT_NOFILE): 1024:1024
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: start worker processes
humhub_1  | 2023/03/04 08:03:27 [notice] 72#72: start worker process 73
humhub_1  | [04-Mar-2023 08:03:28] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
humhub_1  | [04-Mar-2023 08:03:28] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
humhub_1  | [04-Mar-2023 08:03:28] NOTICE: fpm is running, pid 69
humhub_1  | [04-Mar-2023 08:03:28] NOTICE: ready to handle connections
humhub_1  | - -  04/Mar/2023:08:03:56 +0000 "GET /ping" 200

$ curl localhost:8080 -L -s  |grep HumHub|grep "<title>"
    <title>Login - HumHub</title>

mriedmann · 2023-03-07T10:31:19Z

Hi! Thank you both so much for your work. I will try to have a look asap and comment/approve it.

blackandred · 2023-03-07T17:41:15Z

I marked is as draft so it is not an asap, take your time. I will play with it little bit more and will let you know as I don't want to break somebody's instance with this change 😃

blackandred · 2023-03-14T06:15:56Z

I started making a Helm Chart (https://github.com/riotkit-org/k8s-humhub) at first I will support root image, then I will experiment with non-root after I will get the root image working enough stable.

mriedmann · 2023-11-21T16:27:54Z

Dockerfile

@@ -159,7 +174,13 @@ RUN chmod +x /usr/local/bin/php-fpm-healthcheck \
 && adduser --uid 100 -g 101 -S nginx

 EXPOSE 9000
+USER nginx
+CMD ["multirun", "/usr/sbin/php-fpm7 --fpm-config /etc/php-fpm.d/pool.conf -O -F", "tail -f /var/www/localhost/htdocs/protected/runtime/logs/app.log", "crond -f -L /proc/self/fd/2"]


tailing on this file is not the best idea ... it still writes to the top-layer potentially filling up the nodes local storage. We should use a symlink to /proc/self/fd/2 for that (if possible).

mriedmann · 2023-11-21T16:30:21Z

Dockerfile

+    mkdir -p /var/www/localhost/htdocs/uploads /var/www/localhost/htdocs/assets /var/www/localhost/htdocs/protected/modules /var/www/localhost/htdocs/themes /var/www/localhost/htdocs/protected/config && \
+    mkdir -p /run/nginx /run/php-fpm && \
+    touch /var/www/localhost/htdocs/protected/runtime/logs/app.log && \
+    chown 100:101 -R /var/www/localhost/htdocs/protected/runtime/logs /run/nginx /run/php-fpm \


for an openshift setup using the group 0 might be better than 101 because openshift gives the user a "random" high-id and the group 0. If we just allow the group 0 to read/write most parts of the files this should work in most scenarios.

see https://catalog.redhat.com/software/containers/ubi9/nginx-120/61a609f2bfd4a5234d596287?q=nginx&architecture=amd64&image=654d19137095b1690e733af3&container-tabs=dockerfile

mriedmann · 2023-11-22T13:18:56Z

I would really like to see this change and the helm chart come to (more) life, but I have no time to finish it. Is anyone interested in helping with this? Is there even a need for it? Please Emoji-Vote if you would be interested in using this, thx.

chroniclesofdave · 2023-12-30T02:03:19Z

I would really like to see this change and the helm chart come to (more) life, but I have no time to finish it. Is anyone interested in helping with this? Is there even a need for it? Please Emoji-Vote if you would be interested in using this, thx.

I'm going to be honest, I have no idea what you are doing with the code, but I use the docker container in truecharts with truenas scale, and I love it. If sorry you have no time, but if I could upvote it a million times I would. I'm having an issue with my deployment because of supervisord troubles and if this would fix those, it would light up my world.

mriedmann · 2024-01-04T17:04:35Z

I had another look and I will try to get this going. I am still no big fan of multirun but reducing the size and complexity and getting a non-root container is a big improvement. I hope I can get to it over the weekend.

blackandred · 2024-01-14T21:56:18Z

Wow, sorry for abadoning the topic. I "slept" for almost a year. My instance still works on a feature branch, damn it 😁
Happy to see that somebody took over it.

blackandred · 2024-01-14T21:56:35Z

I must at least finish the Helm Chart :)

blackandred · 2024-01-14T21:58:58Z

Regarding the non-root container - I feel there could be some pitfalls on container startup, like chmod/chown or cache wipe, let's check it more strictly.

ArchBlood · 2024-01-15T02:01:41Z

Regarding the non-root container - I feel there could be some pitfalls on container startup, like chmod/chown or cache wipe, let's check it more strictly.

I've also considered this as well for my own Dockerfile, currently this is how I've implemented it;
GreenMeteor/humhub-docker#23

feat: Run as non-root by default to allow easier deployment on the re…

afd6192

…stricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed

blackandred mentioned this pull request Mar 3, 2023

Cache has wrong permissions #194

Closed

blackandred mentioned this pull request Mar 3, 2023

update to php8 #268

Merged

blackandred added 4 commits March 3, 2023 13:03

ci: Increase verbosity

13fb159

ci: Increase verbosity

94d6aac

fix: Set initial volume permissions by creating empty directories wit…

e2f4980

…h proper permissions / mriedmann#308

feat: Increase verbosity / mriedmann#308

ff2ea1c

ArchBlood reviewed Mar 3, 2023

View reviewed changes

blackandred mentioned this pull request Mar 14, 2023

Support non-root container riotkit-org/k8s-humhub#5

Open

mriedmann reviewed Nov 21, 2023

View reviewed changes

mriedmann mentioned this pull request Jan 5, 2024

feat: run non-root (use multirun instread of supervisor) #337

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

draft: feat: Run as non-root by default to allow easier deployment on the restricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed #308

draft: feat: Run as non-root by default to allow easier deployment on the restricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed #308

blackandred commented Mar 3, 2023 •

edited

Loading

blackandred commented Mar 3, 2023

ArchBlood Mar 3, 2023

ArchBlood Mar 3, 2023

blackandred Mar 4, 2023

blackandred commented Mar 4, 2023

ArchBlood commented Mar 4, 2023

blackandred commented Mar 4, 2023

ArchBlood commented Mar 4, 2023

blackandred commented Mar 4, 2023

blackandred commented Mar 4, 2023 •

edited

Loading

mriedmann commented Mar 7, 2023

blackandred commented Mar 7, 2023

blackandred commented Mar 14, 2023 •

edited

Loading

mriedmann Nov 21, 2023

mriedmann Nov 21, 2023 •

edited

Loading

mriedmann commented Nov 22, 2023

chroniclesofdave commented Dec 30, 2023

mriedmann commented Jan 4, 2024

blackandred commented Jan 14, 2024

blackandred commented Jan 14, 2024

blackandred commented Jan 14, 2024

ArchBlood commented Jan 15, 2024

draft: feat: Run as non-root by default to allow easier deployment on the restricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed #308

Are you sure you want to change the base?

draft: feat: Run as non-root by default to allow easier deployment on the restricted environments like OpenShift, Kubernetes and any corporate (or just secure) environments where root privileges are not allowed #308

Conversation

blackandred commented Mar 3, 2023 • edited Loading

blackandred commented Mar 3, 2023

ArchBlood Mar 3, 2023

Choose a reason for hiding this comment

ArchBlood Mar 3, 2023

Choose a reason for hiding this comment

blackandred Mar 4, 2023

Choose a reason for hiding this comment

blackandred commented Mar 4, 2023

ArchBlood commented Mar 4, 2023

blackandred commented Mar 4, 2023

ArchBlood commented Mar 4, 2023

blackandred commented Mar 4, 2023

blackandred commented Mar 4, 2023 • edited Loading

mriedmann commented Mar 7, 2023

blackandred commented Mar 7, 2023

blackandred commented Mar 14, 2023 • edited Loading

mriedmann Nov 21, 2023

Choose a reason for hiding this comment

mriedmann Nov 21, 2023 • edited Loading

Choose a reason for hiding this comment

mriedmann commented Nov 22, 2023

chroniclesofdave commented Dec 30, 2023

mriedmann commented Jan 4, 2024

blackandred commented Jan 14, 2024

blackandred commented Jan 14, 2024

blackandred commented Jan 14, 2024

ArchBlood commented Jan 15, 2024

blackandred commented Mar 3, 2023 •

edited

Loading

blackandred commented Mar 4, 2023 •

edited

Loading

blackandred commented Mar 14, 2023 •

edited

Loading

mriedmann Nov 21, 2023 •

edited

Loading