VirtioFS is not handling permissions as expected. All mount permissions are owned by root regardless of chown.

[spurin]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: 20220319160755

Expected behavior

Have encountered an issue in VirtioFS. Docker Desktop for Mac should handle bind mounted file permissions as you would expect on a generic Linux distribution. If a bind mount is chown'd to a specific user, it should be visible as being owned by the target user in question.

As an example. If you execute the following with or without the new virtualisation framework and VirtioFS disabled -

% pwd; sudo rm -rf tmphome; mkdir tmphome; docker run --mount type=bind,source=/Users/james/virtiofs-issue/tmphome,target=/home/testuser,bind-propagation=rprivate --rm ubuntu /bin/sh -c "id; useradd -m testuser; chown -R testuser /home/testuser; ls -altrh /home/testuser"
/Users/james/virtiofs-issue
uid=0(root) gid=0(root) groups=0(root)
useradd: warning: the home directory /home/testuser already exists.
useradd: Not copying any file from skel directory into it.
total 4.0K
drwxr-xr-x 2 testuser root   64 Mar 19 15:45 .
drwxr-xr-x 1 root     root 4.0K Mar 19 15:45 ..

The userid of testuser is as expected for the directory ownership.

Actual behavior

If VirtioFS is enabled and Docker Desktop is restarted, and then, the attempt is re-issued, the permissions for that directory are incorrectly owned by root -

% pwd; sudo rm -rf tmphome; mkdir tmphome; docker run --mount type=bind,source=/Users/james/virtiofs-issue/tmphome,target=/home/testuser,bind-propagation=rprivate --rm ubuntu /bin/sh -c "id; useradd -m testuser; chown -R testuser /home/testuser; ls -altrh /home/testuser"
/Users/james/virtiofs-issue
uid=0(root) gid=0(root) groups=0(root)
useradd: warning: the home directory /home/testuser already exists.
useradd: Not copying any file from skel directory into it.
total 4.0K
drwxr-xr-x 2 root root   64 Mar 19 15:51 .
drwxr-xr-x 1 root root 4.0K Mar 19 15:51 ..

For the projects I manage, I make extensive use of users and volumes and at present, this renders the environment unusable owing to the individual userid's having incorrect permissions.

Information

• macOS Version: 12.3
• Intel chip or Apple chip: Apple M1 (Mac Mini)
• Docker Desktop Version: 4.6.0 GA

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

Am unsure why the diagnostics are failing but, all is okay when using Docker as per the examples above -

% /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check
Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0031: does the Docker API work?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0011: are the LinuxKit services running?
[FAIL] DD0016: is the LinuxKit VM running? vm is not running: vm has not started
[PASS] DD0001: is the application running?
[PASS] DD0018: does the host support virtualization?
[FAIL] DD0017: can a VM be started? vm has not started: vm has not started
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0007: is the backend responding?
[PASS] DD0014: are the backend processes running?
[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[PASS] DD0012: is the VM networking working?
[PASS] DD0032: do Docker networks overlap with host IPs?
[SKIP] DD0030: is the image access management authorized?
[PASS] DD0019: is the com.docker.vmnetd process responding?
[PASS] DD0033: does the host have Internet access?

Please investigate the following 1 issue:

1 : The test: can a VM be started?
    Failed with: vm has not started: vm has not started

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

Steps to reproduce the behavior

1. Disable VirtioFS, run the command example listed, check that the directory is owned by testuser
2. Enable VirtioFS, repeat the test, if the issue occurs the directory is owned by root

[spurin]

Tested on 4.7.0 (76176), issue still persists

[fredericdalleau]

@spurin thanks, we are investigating.
On a side note, could you elaborate a bit on your use case? It seems to me that a volume in the sense of docker volume create could be a better choice as you would have real unix semantics ? Any reason not do so ?

[spurin]

Thanks @fredericdalleau

I very much appreciate you taking the time to respond to this. The environment that I noticed the issue with is open sourced under https://github.com/spurin/diveintoansible-lab

Essentially it provides a nice lab environment allowing those learning Ansible (and Linux in general), to have six vm like containers (ubuntu and centos) and an Ansible control host alongside web terminals and sshd.

I opted for bind mounted directories that correspond to the 'ansible' and 'root' users to cater for those new to unix who at this point, may struggle with the likes of vim or other areas (the course is aimed from a beginner up with no prior knowledge).

With this, all of the user directories are also easily accessible on the host system should they need them.

It the user does have difficulties, they can use a local editor on their system and/or easily copy/save the work they may have done during the course.

If you were curious, you can quickly try the environment on Google Cloud Shell using a standard google id -

https://diveinto.com/p/playground

[jaspertandy]

I'm experiencing this in particular with postgres images where I mount a data dir from host into /var/lib/postgresql/data - the mounted directory gets root ownership, which I can't change. Changing the user in docker-compose.yml does alter the permissions but I still get a lot of permissions errors when trying to write to the database, either by creating databases or inserting data. I'll try to create a replicable docker-compose.yml

Edit: I can't replicate this with a brand new container. Going to try trashing the offending container in my broken project and seeing if that fixes it.

[jaspertandy]

Ah I was just over-thinking it. Given this docker-compose.yml:

version: '3'
services:
  db:
    image: postgres:13.2
    environment:
      POSTGRES_PASSWORD: root
    volumes:
      - ./postgres-data:/var/lib/postgresql/data

Running docker compose up gives me this:

[+] Running 1/1
 ⠿ Container virtio-db-1  Recreated                                                                                                                                                       0.1s
Attaching to virtio-db-1
virtio-db-1  | The files belonging to this database system will be owned by user "postgres".
virtio-db-1  | This user must also own the server process.
virtio-db-1  |
virtio-db-1  | The database cluster will be initialized with locale "en_US.utf8".
virtio-db-1  | The default database encoding has accordingly been set to "UTF8".
virtio-db-1  | The default text search configuration will be set to "english".
virtio-db-1  |
virtio-db-1  | Data page checksums are disabled.
virtio-db-1  |
virtio-db-1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
virtio-db-1  | creating subdirectories ... ok
virtio-db-1  | selecting dynamic shared memory implementation ... posix
virtio-db-1  | selecting default max_connections ... 100
virtio-db-1  | selecting default shared_buffers ... 128MB
virtio-db-1  | selecting default time zone ... Etc/UTC
virtio-db-1  | creating configuration files ... ok
virtio-db-1  | running bootstrap script ... 2022-03-21 18:07:23.423 UTC [38] LOG:  could not open file "pg_wal/000000010000000000000001": No such file or directory
virtio-db-1  | 2022-03-21 18:07:23.423 UTC [38] FATAL:  could not open file "pg_wal/000000010000000000000001": No such file or directory
virtio-db-1  | child process exited with exit code 1
virtio-db-1  | initdb: removing contents of data directory "/var/lib/postgresql/data"
virtio-db-1 exited with code 1

Presumably because it's trying to do bootstrapping in the postgres container with directories owned by root.

If I change my docker-compose.yml to add a user, I get a different error message:

version: '3'
services:
  db:
    image: postgres:13.2
    user: postgres
    environment:
      POSTGRES_PASSWORD: root
    volumes:
      - ./postgres-data:/var/lib/postgresql/data

[+] Running 1/0
 ⠿ Container virtio-db-1  Created                                                                                                                                                         0.0s
Attaching to virtio-db-1
virtio-db-1  | chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted
virtio-db-1  | The files belonging to this database system will be owned by user "postgres".
virtio-db-1  | This user must also own the server process.
virtio-db-1  |
virtio-db-1  | The database cluster will be initialized with locale "en_US.utf8".
virtio-db-1  | The default database encoding has accordingly been set to "UTF8".
virtio-db-1  | The default text search configuration will be set to "english".
virtio-db-1  |
virtio-db-1  | Data page checksums are disabled.
virtio-db-1  |
virtio-db-1  | initdb: error: could not change permissions of directory "/var/lib/postgresql/data": Operation not permitted
virtio-db-1 exited with code 1

Sorry @spurin for crashing into your issue but I think we're experiencing symptoms of the same problem.

[palgalik]

I think I have the same issue, only this happens with mysql.

My compose file looks like this:

version: '3.5'

services:
  mysql:
    image: mysql/mysql-server:8.0
    volumes:
      - ./data:/var/lib/mysql
    ports:
      - "3306:3306"

My console log:

test-mysql-1  | [Entrypoint] MySQL Docker Image 8.0.28-1.2.7-server
test-mysql-1  | [Entrypoint] Starting MySQL 8.0.28-1.2.7-server
test-mysql-1  | 2022-03-21T20:36:10.700110Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.28) starting as process 1
test-mysql-1  | 2022-03-21T20:36:10.715582Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
test-mysql-1  | 2022-03-21T20:36:10.814497Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
test-mysql-1  | 2022-03-21T20:36:10.848186Z 1 [ERROR] [MY-012585] [InnoDB] Linux Native AIO interface is not supported on this platform. Please check your OS documentation and install appropriate binary of InnoDB.
test-mysql-1  | 2022-03-21T20:36:10.848563Z 1 [Warning] [MY-012654] [InnoDB] Linux Native AIO disabled.
test-mysql-1  | 2022-03-21T20:36:11.162386Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
test-mysql-1  | 2022-03-21T20:36:11.855332Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
test-mysql-1  | 2022-03-21T20:36:11.856044Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
test-mysql-1  | 2022-03-21T20:36:11.860685Z 0 [ERROR] [MY-010259] [Server] Another process with pid 91 is using unix socket file.
test-mysql-1  | 2022-03-21T20:36:11.860827Z 0 [ERROR] [MY-010268] [Server] Unable to setup unix socket lock file.
test-mysql-1  | 2022-03-21T20:36:11.861910Z 0 [ERROR] [MY-010119] [Server] Aborting
test-mysql-1  | 2022-03-21T20:36:12.737014Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.28)  MySQL Community Server - GPL.

Here is my uploaded diagnostics id:
1AFB312E-95AE-48E1-908A-A8E51B0CFD65/20220321202816

[spurin]

No problem by the way @jaspertandy - I welcome your input. We're all in the same boat with this issue 🙂

[doprdele]

I also have this issue. Will upload.

[RikkiMasters]

I also have the issue with MariaDB. Took so long to find that the cause was VirtioFS!

ERROR: 29 File './mysql/' not found (Errcode: 13 "Permission denied")

db:
  image: 'mariadb:10.4.21'
  restart: unless-stopped
  volumes:
    - './docker/mariadb:/var/lib/mysql'

[spurin]

I also have the issue with MariaDB. Took so long to find that the cause was VirtioFS!

ERROR: 29 File './mysql/' not found (Errcode: 13 "Permission denied")

db:

  image: 'mariadb:10.4.21'

  restart: unless-stopped

  volumes:

    - './docker/mariadb:/var/lib/mysql'

Hi @RikkiMasters

Yep, looks like the same issue. Had a look at the Dockerfile for this version of Mariadb and my guess it's also expecting specific user ownership as per the line here -

https://github.com/MariaDB/mariadb-docker/blob/5b93a88ae340de53d621125bef89e3571a325cfa/10.4/Dockerfile#L119

[palgalik]

docker/roadmap#7 (comment)

You can work it around, by adding the following operator to the mysqld command:
command: mysqld --socket=/tmp/mysql.sock

[spurin]

docker/roadmap#7 (comment)

You can work it around, by adding the following operator to the mysqld command:

command: mysqld --socket=/tmp/mysql.sock

Thanks for sharing this workaround for MySQL/MariaDB!

The underline issue of virtiofs not handling permissions as expected is the main context of the issue I raised.

Hopefully when resolved this will resolve the issue I've encountered as well as avoid the need for workarounds or changes to container images like you've had to do 👍