Mount with 9p not working on Arch with docker

[gudn]

Hello, I'd met with the same issue. Cluster was started with minikube start with default docker daemon. Then I'd tried minikube mount .:/app

Logs:
minikube_mount.log
logs.txt

Docker version output:

Client:
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.18.3
 Git commit:        100c70180f
 Built:             Sat Jun 11 23:27:28 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.3
  Git commit:       a89b84221c
  Built:            Sat Jun 11 23:27:14 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1.m
 runc:
  Version:          1.1.3
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Minikube version:

minikube version: v1.26.0
commit: f4b412861bb746be73053c9f6d2895f12cf78565-dirty

Kernel version is 5.15.57-2-MANJARO

Originally posted by @gudn in #14325 (comment)

[afbjorklund]

mount: /app: mount(2) system call failed: Bad address.

Do you have 9p fs enabled in your kernel ?

CONFIG_9P_FS

As previous issue, using docker --mount (at start) is preferred.

[gudn]

Yes, all 9P kernel configs are enabled, most of them as modules. Modules were loaded too.

[afbjorklund]

Maybe it doesn't work to mount 9p, even from a privileged container ? Haven't tried it myself.

[afbjorklund]

Nope, must be something else. It mounted fine in Ubuntu.

docker@minikube:~$ findmnt /app
TARGET SOURCE       FSTYPE OPTIONS
/app   192.168.49.1 9p     rw,relatime,sync,dirsync,dfltuid=1000,dfltgid=999,acc
docker@minikube:~$ uname -r
5.4.0-122-generic

[gudn]

Yes, I've just tested on clean Debian vm with 5.10 kernel, and it works fine.

docker inspect output on my laptop:

docker inspect minikube

[
  {
    "Id": "98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157",
    "Created": "2022-07-31T07:17:35.553662686Z",
    "Path": "/usr/local/bin/entrypoint",
    "Args": [
      "/sbin/init"
    ],
    "State": {
      "Status": "running",
      "Running": true,
      "Paused": false,
      "Restarting": false,
      "OOMKilled": false,
      "Dead": false,
      "Pid": 98955,
      "ExitCode": 0,
      "Error": "",
      "StartedAt": "2022-07-31T07:17:36.531697035Z",
      "FinishedAt": "0001-01-01T00:00:00Z"
    },
    "Image": "sha256:ff7b11088f07b5cc3be2087261aee1627a466228279acbafdb95902df26942d2",
    "ResolvConfPath": "/var/lib/docker/containers/98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157/resolv.conf",
    "HostnamePath": "/var/lib/docker/containers/98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157/hostname",
    "HostsPath": "/var/lib/docker/containers/98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157/hosts",
    "LogPath": "/var/lib/docker/containers/98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157/98c644b3dc651bd07a88007e14bfb3df227855a6bb91083961febf318cacb157-json.log",
    "Name": "/minikube",
    "RestartCount": 0,
    "Driver": "overlay2",
    "Platform": "linux",
    "MountLabel": "",
    "ProcessLabel": "",
    "AppArmorProfile": "",
    "ExecIDs": null,
    "HostConfig": {
      "Binds": [
        "/lib/modules:/lib/modules:ro",
        "minikube:/var"
      ],
      "ContainerIDFile": "",
      "LogConfig": {
        "Type": "json-file",
        "Config": {}
      },
      "NetworkMode": "minikube",
      "PortBindings": {
        "22/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": ""
          }
        ],
        "2376/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": ""
          }
        ],
        "32443/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": ""
          }
        ],
        "5000/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": ""
          }
        ],
        "8443/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": ""
          }
        ]
      },
      "RestartPolicy": {
        "Name": "no",
        "MaximumRetryCount": 0
      },
      "AutoRemove": false,
      "VolumeDriver": "",
      "VolumesFrom": null,
      "CapAdd": null,
      "CapDrop": null,
      "CgroupnsMode": "private",
      "Dns": [],
      "DnsOptions": [],
      "DnsSearch": [],
      "ExtraHosts": null,
      "GroupAdd": null,
      "IpcMode": "private",
      "Cgroup": "",
      "Links": null,
      "OomScoreAdj": 0,
      "PidMode": "",
      "Privileged": true,
      "PublishAllPorts": false,
      "ReadonlyRootfs": false,
      "SecurityOpt": [
        "seccomp=unconfined",
        "apparmor=unconfined",
        "label=disable"
      ],
      "Tmpfs": {
        "/run": "",
        "/tmp": ""
      },
      "UTSMode": "",
      "UsernsMode": "",
      "ShmSize": 67108864,
      "Runtime": "runc",
      "ConsoleSize": [
        0,
        0
      ],
      "Isolation": "",
      "CpuShares": 0,
      "Memory": 4294967296,
      "NanoCpus": 0,
      "CgroupParent": "",
      "BlkioWeight": 0,
      "BlkioWeightDevice": [],
      "BlkioDeviceReadBps": null,
      "BlkioDeviceWriteBps": null,
      "BlkioDeviceReadIOps": null,
      "BlkioDeviceWriteIOps": null,
      "CpuPeriod": 0,
      "CpuQuota": 0,
      "CpuRealtimePeriod": 0,
      "CpuRealtimeRuntime": 0,
      "CpusetCpus": "",
      "CpusetMems": "",
      "Devices": [],
      "DeviceCgroupRules": null,
      "DeviceRequests": null,
      "KernelMemory": 0,
      "KernelMemoryTCP": 0,
      "MemoryReservation": 0,
      "MemorySwap": 8589934592,
      "MemorySwappiness": null,
      "OomKillDisable": null,
      "PidsLimit": null,
      "Ulimits": null,
      "CpuCount": 0,
      "CpuPercent": 0,
      "IOMaximumIOps": 0,
      "IOMaximumBandwidth": 0,
      "MaskedPaths": null,
      "ReadonlyPaths": null
    },
    "GraphDriver": {
      "Data": {
        "LowerDir": "/var/lib/docker/overlay2/15263647c06c316eecf18e00431221562ccd164282cdf8b6db4b14dd5a3886a7-init/diff:/var/lib/docker/overlay2/a0f6584d591146310abee78a5d1366ea579497c3464a99591616a0556ab65bd5/diff:/var/lib/docker/overlay2/ed6ff90cf135cca8241a229d4ce2a01837d5871ee8c5cb9e2bffee6f94180a6a/diff:/var/lib/docker/overlay2/9658635d7fe5e86762732ff1635c516369a1a04b475923593fd4302736867323/diff:/var/lib/docker/overlay2/10390aed329f87b7558359ae42f9cb012ffbaf60cd92dd2ecf98f07975b25766/diff:/var/lib/docker/overlay2/da0fedbb4d1fa38be2b3b2b29a6add135553a83f989c27583b7392d6d86c86b2/diff:/var/lib/docker/overlay2/1a3e8c6f03a7494fa08461f20c5ec33c29f46cbc685eecbebe8e36ccd663702c/diff:/var/lib/docker/overlay2/3dec6973b6682fd8a8f3120bdc9aac20235f4dee96189e9cf722c70e2caa1c40/diff:/var/lib/docker/overlay2/b33924f8cd76118aa1a52ce21aa13fadb04d15e259b4ea081eb53057b3772b24/diff:/var/lib/docker/overlay2/537302b62086e30ffb0b4926efa9e4e384769beeb712614765aa05c1cb6550dd/diff:/var/lib/docker/overlay2/82e999800323eb727ea6cca324eb68b706ca4c05f74483abdcfab28eeffbfe59/diff:/var/lib/docker/overlay2/cfd5e07f57e02191d31be3672cfff73b91d6db2dca558df8ac269cda912d1b94/diff:/var/lib/docker/overlay2/9770ec47f93cde0400121b791c873ed4db9aecfcb6e66825bc8e39dfce329f0b/diff:/var/lib/docker/overlay2/6097c439c421ac10e81b7b5f410239bc73e68f1a5e7a845c9bccccdacf046254/diff:/var/lib/docker/overlay2/a38f6af8e86f32853d810bc3b9b3e419ecb3ce0a38e9eb58b47b7975e4ab80b4/diff:/var/lib/docker/overlay2/158bf915490adbe9ffdf6729afdccfcab53e62797e7a6386ba00d5e7b2c28c4f/diff:/var/lib/docker/overlay2/2d75d65888a540639c09157fc38fef3597bf5a4fef026e33f6b5a82d4f63e327/diff:/var/lib/docker/overlay2/2b5ce5c1411090b7f22f95b09dfeaa55d0903a5dbb5f332a16646a5b359c1632/diff:/var/lib/docker/overlay2/1b45847212dd70b5bf642e1b75a6469477c4b5d0e7e7a4cb90993b5370018892/diff:/var/lib/docker/overlay2/b81072ac79f705bc439fb5968ca888315fbb2742db78f9f8e8a1af2b63fe0b0d/diff:/var/lib/docker/overlay2/e36f7bbc0a74c31c36f6f623fabbb046a5e9c03315c52c910f0bb91f834f82e1/diff:/var/lib/docker/overlay2/b805a8ecef6ec1f1826d2d7a66a1168d1aa3798dc9da259a5d34f0e7ad9897ab/diff:/var/lib/docker/overlay2/d4347672cda3fc1f490b9a379669dc621007a3a579bc2eeccf9dc965d73796fb/diff:/var/lib/docker/overlay2/8924c2eeaf8cec879ddd4a9020c3d968440a6f9cb7a5a11f811c77308a245af2/diff:/var/lib/docker/overlay2/138e7e86af0bd4d08bc1a1c7c338de8f9a9cc5a025ede49a78916d947989bf2b/diff:/var/lib/docker/overlay2/f64d7a773f571b77e75b03601fba54e69a300db99bafba454c9cc9a179ae0d7e/diff:/var/lib/docker/overlay2/2ac2803d55be24871cf1aff702574cb656f93fc391a977c99dec9bd8abb38516/diff:/var/lib/docker/overlay2/c8cf4c726ac07fb9184e6894d23b6f239dcd8cfdfe68264a9fc491396fe30a7e/diff:/var/lib/docker/overlay2/b61bf5dd3255bea16787074218108f5e403a7577f41917c027780416803c2bb6/diff:/var/lib/docker/overlay2/c9fed89a0fdc1a80a7f34fb33b98c003656c0322a429f38065a092ef66d14a1c/diff:/var/lib/docker/overlay2/db7fca70b9a79586dfa9191c475ba422cdeb57011e82c9be56746c990ea3c051/diff:/var/lib/docker/overlay2/d7e53e41d603e67da1c1c50824ec9ffbb44c963bc68fc99e3da0313ad4d75950/diff:/var/lib/docker/overlay2/e4aed4996351b4fffb9839882c6724a98ace86e0e4711d042af1d68204b61a98/diff:/var/lib/docker/overlay2/f6b9454e348577bcf6cdd9affed270583e207c97cbeddb61a1487ef56e4f830c/diff:/var/lib/docker/overlay2/8c96d77bb0ccc1acc5d6c3e6c4e4696dbd15f95c414e79a41163d8137861717c/diff:/var/lib/docker/overlay2/5fb9cb9f903f22ec37d69f784f82e4419a6fab99b48f44c36fbae389eeb1f747/diff:/var/lib/docker/overlay2/e332afb3bded0e2fedcf2fa12aee62b217fa4324069158ca428820fcb9e8b04d/diff:/var/lib/docker/overlay2/36abcaca77c0fb1538a683a79be4d5707d60c8a76bff00a8478b1a77807d1af9/diff:/var/lib/docker/overlay2/3eafcef78335fa5e14bdf4b22b0ffa12edf3669419abac10e739a1fdb9eb8d1b/diff:/var/lib/docker/overlay2/b3652076a5935ef265fe7326504c885653a6aa477db92f42a17d9ab6c29e7fcb/diff:/var/lib/docker/overlay2/2eaef56a057528b87f6511560e48d44e53e751ce671eee4e601c9e6681054dc5/diff:/var/lib/docker/overlay2/85411f5e096c02b1b23d233e72471e0c8c55484345f33e7629feb75dbb548dff/diff",
        "MergedDir": "/var/lib/docker/overlay2/15263647c06c316eecf18e00431221562ccd164282cdf8b6db4b14dd5a3886a7/merged",
        "UpperDir": "/var/lib/docker/overlay2/15263647c06c316eecf18e00431221562ccd164282cdf8b6db4b14dd5a3886a7/diff",
        "WorkDir": "/var/lib/docker/overlay2/15263647c06c316eecf18e00431221562ccd164282cdf8b6db4b14dd5a3886a7/work"
      },
      "Name": "overlay2"
    },
    "Mounts": [
      {
        "Type": "bind",
        "Source": "/lib/modules",
        "Destination": "/lib/modules",
        "Mode": "ro",
        "RW": false,
        "Propagation": "rprivate"
      },
      {
        "Type": "volume",
        "Name": "minikube",
        "Source": "/var/lib/docker/volumes/minikube/_data",
        "Destination": "/var",
        "Driver": "local",
        "Mode": "z",
        "RW": true,
        "Propagation": ""
      }
    ],
    "Config": {
      "Hostname": "minikube",
      "Domainname": "",
      "User": "root",
      "AttachStdin": false,
      "AttachStdout": false,
      "AttachStderr": false,
      "ExposedPorts": {
        "22/tcp": {},
        "2376/tcp": {},
        "32443/tcp": {},
        "5000/tcp": {},
        "8443/tcp": {}
      },
      "Tty": true,
      "OpenStdin": false,
      "StdinOnce": false,
      "Env": [
        "container=docker",
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      ],
      "Cmd": null,
      "Image": "gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95",
      "Volumes": null,
      "WorkingDir": "",
      "Entrypoint": [
        "/usr/local/bin/entrypoint",
        "/sbin/init"
      ],
      "OnBuild": null,
      "Labels": {
        "created_by.minikube.sigs.k8s.io": "true",
        "mode.minikube.sigs.k8s.io": "minikube",
        "name.minikube.sigs.k8s.io": "minikube",
        "role.minikube.sigs.k8s.io": ""
      },
      "StopSignal": "SIGRTMIN+3"
    },
    "NetworkSettings": {
      "Bridge": "",
      "SandboxID": "51343e272dd712d5e628bfd91318e696eca8215b069f529460b005897347406c",
      "HairpinMode": false,
      "LinkLocalIPv6Address": "",
      "LinkLocalIPv6PrefixLen": 0,
      "Ports": {
        "22/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": "49182"
          }
        ],
        "2376/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": "49181"
          }
        ],
        "32443/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": "49178"
          }
        ],
        "5000/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": "49180"
          }
        ],
        "8443/tcp": [
          {
            "HostIp": "127.0.0.1",
            "HostPort": "49179"
          }
        ]
      },
      "SandboxKey": "/var/run/docker/netns/51343e272dd7",
      "SecondaryIPAddresses": null,
      "SecondaryIPv6Addresses": null,
      "EndpointID": "",
      "Gateway": "",
      "GlobalIPv6Address": "",
      "GlobalIPv6PrefixLen": 0,
      "IPAddress": "",
      "IPPrefixLen": 0,
      "IPv6Gateway": "",
      "MacAddress": "",
      "Networks": {
        "minikube": {
          "IPAMConfig": {
            "IPv4Address": "192.168.49.2"
          },
          "Links": null,
          "Aliases": [
            "98c644b3dc65",
            "minikube"
          ],
          "NetworkID": "eb826f2e0ca62958e927dbf413945af33ff07858b3e45f96d50b6cc11b743136",
          "EndpointID": "954f57baba886dc7820619deab0fa76ceb9b0096038721476c60cdce7defa87d",
          "Gateway": "192.168.49.1",
          "IPAddress": "192.168.49.2",
          "IPPrefixLen": 24,
          "IPv6Gateway": "",
          "GlobalIPv6Address": "",
          "GlobalIPv6PrefixLen": 0,
          "MacAddress": "02:42:c0:a8:31:02",
          "DriverOpts": null
        }
      }
    }
  }
]

[klaases]

Yes, I've just tested on clean Debian vm with 5.10 kernel, and it works fine.

Hi @gudn, is everything working now as expected?

I will go ahead and close the issue but feel free to re-open the issue by commenting with /reopen.

Thank you for sharing your experience!