Issues when using a custom uid/gid with docker-compose

[damien-git]

This is related to #641 (which is closed).

The mysql image is supposed to support using a custom uid/gid (see Running as an arbitrary user).
In practice, this doesn't work with docker-compose, because the /var/lib/mysql directory is owned by the mysql image user with id 999. /var/lib/mysql is defined as a volume in the image, so it makes sense to define a named volume to use it with docker-compose. But even when permissions to the volume are fixed, within a container they will always be the permissions set by the image, and the custom user defined with user: in the docker-compose file will not have access.

To work around this issue, I tried to run the container as root (without using user:) and define a custom entrypoint in docker-compose with entrypoint:, which was fixing permissions in the container and executing the image's entrypoint with the custom user. That didn't work, because I needed to use sudo and apparently this is not part of buster-slim.

I then tried another workaround: using --datadir in a custom entrypoint, and a named volume pointing to a directory different from /var/lib/mysql. That works, but a new volume with a random name is created every time the application starts because of the volume defined in the image for /var/lib/mysql. So I have to create an "unused" named volume for /var/lib/mysql to work around that... That's rather ugly.

A possible fix for these issues would be to use environment variables instead of user:, change the Dockerfile to install sudo, and change the mysql image entrypoint to fix permissions to /var/lib/mysql if necessary and use something like sudo -u "#$MYSQL_USER_UID" -g "#$MYSQL_USER_GID" to continue.

To explain the context, I looked into using a custom user because I needed to share the socket file with the host to allow a backup with it (the backup software we use requires a socket file). But I didn't want user 999 on the host to have access to it.

[tianon]

Arg, this is why we chmod 777 /var/lib/mysql:

mysql/8.0/Dockerfile

Lines 74 to 75 in b0f81a3

	# ensure that /var/run/mysqld (used for socket and lock files) is writable regardless of the UID our mysqld instance ends up having at runtime
	&& chmod 777 /var/run/mysqld

But, I've tested and confirmed it is not working: 😞

version: '3.8'

services:
  mysql:
    image: mysql:8.0
    volumes:
      - mysql:/var/lib/mysql
    user: 1234:5678
    environment:
      MYSQL_ROOT_PASSWORD: example

volumes:
  mysql:

$ docker-compose -p example up
Creating network "example_default" with the default driver
Creating volume "example_mysql" with default driver
Creating example_mysql_1 ... done
Attaching to example_mysql_1
mysql_1  | 2020-09-01 16:03:08+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
mysql_1  | 2020-09-01 16:03:08+00:00 [Note] [Entrypoint]: Initializing database files
mysql_1  | mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (OS errno 13 - Permission denied)
mysql_1  | 2020-09-01T16:03:08.106514Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.21) initializing of server in progress as process 22
mysql_1  | 2020-09-01T16:03:08.107896Z 0 [ERROR] [MY-010460] [Server] --initialize specified but the data directory exists and is not writable. Aborting.
mysql_1  | 2020-09-01T16:03:08.107899Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
mysql_1  | 2020-09-01T16:03:08.107933Z 0 [ERROR] [MY-010119] [Server] Aborting
mysql_1  | 2020-09-01T16:03:08.108012Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.21)  MySQL Community Server - GPL.
example_mysql_1 exited with code 1

[tianon]

Oh, that Dockerfile bit is /var/run/mysqld, not /var/lib/mysql -- that's for the PID file and socket, not the data.

I think our stance in the past has been that if you want to run as an arbitrary user, you become responsible for ensuring the data directory is sufficiently writable by that user. That being said, I think there's some room for adjustment here (and hopefully @yosifkit and @ltangvald won't disagree 😅).

[damien-git]

@tianon thanks for providing an example and log, I think it sums it up.

The only way this example could work is if /var/lib/mysql was not created in the Dockerfile. Which makes me wonder: is this an option ? Maybe we could just create it in the entrypoint if needed, and fix permissions if it's executed as root ?

[yosifkit]

Made a PR to improve it: #710