FATAL: Unable to enable local routing for hairpin mode: open /proc/sys/net/ipv4/conf/docker0/route_localnet: read-only file system

[Coolfeather2]

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

Start Docker service

Actual behavior

Docker service fails to start (code=exited, status=1/FAILURE)

Steps to reproduce the behavior

Installed docker on new Ubuntu 18.04 LXC Container
Docker service would not start, reporting the error to journalctl
Tested by running just dockerd in console and same error occured

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Output of docker info:

Unable due to docker not running

Docker Log:

INFO[2021-02-07T12:34:27.763396538+08:00] Starting up                                  
INFO[2021-02-07T12:34:27.764246819+08:00] parsed scheme: "unix"                         module=grpc
INFO[2021-02-07T12:34:27.764259378+08:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2021-02-07T12:34:27.764270714+08:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2021-02-07T12:34:27.764276114+08:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2021-02-07T12:34:27.765587680+08:00] parsed scheme: "unix"                         module=grpc
INFO[2021-02-07T12:34:27.765613052+08:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2021-02-07T12:34:27.765638893+08:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2021-02-07T12:34:27.765649758+08:00] ClientConn switching balancer to "pick_first"  module=grpc
WARN[2021-02-07T12:34:27.926591817+08:00] Your kernel does not support cgroup rt period 
WARN[2021-02-07T12:34:27.926607702+08:00] Your kernel does not support cgroup rt runtime 
WARN[2021-02-07T12:34:27.926614012+08:00] Your kernel does not support cgroup blkio weight 
WARN[2021-02-07T12:34:27.926621374+08:00] Your kernel does not support cgroup blkio weight_device 
INFO[2021-02-07T12:34:27.926996744+08:00] Loading containers: start.                   
WARN[2021-02-07T12:34:28.325921682+08:00] could not create bridge network for id 4d0c79e91923c48756f23cbd35c86cbcc0f9b997998e1d9a3beef6824f2f26af bridge name br-4d0c79e91923 while booting up from persistent state: Unable to enable local routing for hairpin mode: open /proc/sys/net/ipv4/conf/br-4d0c79e91923/route_localnet: read-only file system 
WARN[2021-02-07T12:34:28.325971464+08:00] unable to disable IPv6 router advertisement   error="open /proc/sys/net/ipv6/conf/docker0/accept_ra: read-only file system"
WARN[2021-02-07T12:34:28.326047109+08:00] could not create bridge network for id c1c10381adb8e0853f81bfb42c5ca6d9f0cd306df445ba0aba05d8eed0112567 bridge name docker0 while booting up from persistent state: Unable to enable local routing for hairpin mode: open /proc/sys/net/ipv4/conf/docker0/route_localnet: read-only file system 
INFO[2021-02-07T12:34:28.327101002+08:00] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address 
WARN[2021-02-07T12:34:28.327172901+08:00] unable to disable IPv6 router advertisement   error="open /proc/sys/net/ipv6/conf/docker0/accept_ra: read-only file system"
INFO[2021-02-07T12:34:28.327692928+08:00] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
failed to start daemon: Error initializing network controller: Error creating default "bridge" network: Unable to enable local routing for hairpin mode: open /proc/sys/net/ipv4/conf/docker0/route_localnet: read-only file system

Additional environment details (AWS, VirtualBox, physical, etc.)
Much like #1033, I am having a similar issue but this also has /proc/sys/net/ipv4/conf/docker0/route_localnet being read-only
Initally the host rebooted and docker wouldn't start, I created a new LXC Container to test if it was an issue with the container but the same issue occured in the new one.

System information:

Linux 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) x86_64 x86_64 x86_64 GNU/Linux

Running inside LXC on Proxmox
Fuse, KeyCTL and nesting enabled.

[Lucavon]

Same issue. For me, this only happens if the file /etc/docker/daemon.json exists with the following content:

{
  "userland-proxy": false,
  "ipv6": true,
  "fixed-cidr-v6": "fd00::/80"
}

[Coolfeather2]

daemon.json contains:

{
    "storage-driver": "overlay2",
    "userland-proxy": false,
    "log-driver": "json-file",
    "init": true,
    "log-opts": {"max-size": "10m", "max-file": "3"}
}

[thaJeztah]

I suspect this would be a configuration issue for the LXC container; have you asked in the LXC forum? https://discuss.linuxcontainers.org/c/lxc/6

[Coolfeather2]

I'm able to run docker without applying the daemon, something in the daemon is messing with docker and removing the daemon after doesn't fix the issue

[Coolfeather2]

Isolated the cause to "userland-proxy": false in the daemon.json