18.09.1.beta1: failed to start after upgrade

[bjornmagnusson]

Upgraded from docker-ce 18.09.0 to 18.09.1.beta1.
After upgrade daemon is not starting.

• docker info
  Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

• systemctl status docker.service

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Mon 2018-12-03 10:12:44 CET; 27s ago
     Docs: https://docs.docker.com
  Process: 12371 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
 Main PID: 12371 (code=exited, status=1/FAILURE)

Dec 03 10:12:41 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
Dec 03 10:12:41 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:41 REDACTED systemd[1]: docker.service failed.
Dec 03 10:12:44 REDACTED systemd[1]: docker.service holdoff time over, scheduling restart.
Dec 03 10:12:44 REDACTED systemd[1]: start request repeated too quickly for docker.service
Dec 03 10:12:44 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
Dec 03 10:12:44 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:44 REDACTED systemd[1]: docker.service failed.

- journalctl -xe
Dec 03 10:12:37 REDACTED systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Dec 03 10:12:37 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Dec 03 10:12:37 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:37 REDACTED systemd[1]: docker.service failed.
Dec 03 10:12:39 REDACTED systemd[1]: docker.service holdoff time over, scheduling restart.
Dec 03 10:12:39 REDACTED systemd[1]: Starting Docker Application Container Engine...
-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has begun starting up.
Dec 03 10:12:39 REDACTED dockerd[12359]: time="2018-12-03T10:12:39+01:00" level=warning msg="Running experimental build"
Dec 03 10:12:39 REDACTED dockerd[12359]: Failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd
Dec 03 10:12:39 REDACTED systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Dec 03 10:12:39 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Dec 03 10:12:39 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:39 REDACTED systemd[1]: docker.service failed.
Dec 03 10:12:41 REDACTED systemd[1]: docker.service holdoff time over, scheduling restart.
Dec 03 10:12:41 REDACTED systemd[1]: Starting Docker Application Container Engine...
-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has begun starting up.
Dec 03 10:12:41 REDACTED dockerd[12371]: time="2018-12-03T10:12:41+01:00" level=warning msg="Running experimental build"
Dec 03 10:12:41 REDACTED dockerd[12371]: Failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd
Dec 03 10:12:41 REDACTED systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Dec 03 10:12:41 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Dec 03 10:12:41 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:41 REDACTED systemd[1]: docker.service failed.
Dec 03 10:12:44 REDACTED systemd[1]: docker.service holdoff time over, scheduling restart.
Dec 03 10:12:44 REDACTED systemd[1]: start request repeated too quickly for docker.service
Dec 03 10:12:44 REDACTED systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Dec 03 10:12:44 REDACTED systemd[1]: Unit docker.service entered failed state.
Dec 03 10:12:44 REDACTED systemd[1]: docker.service failed.

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

Docker daemon is started after upgrade

Actual behavior

Docker daemon is NOT starting after upgrade

Steps to reproduce the behavior

Install docker-ce 18.09.0
Upgrade to docker-ce 18.09.1.beta1

Output of docker version:

(paste your output here)

Output of docker info:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

(paste your output here)

Additional environment details (AWS, VirtualBox, physical, etc.)
On premise virtual machine
[REDACTED ~]# cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)
[REDACTED ~]# uname -a
Linux se-centos-bjma 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[poznachowski]

I had the same issue. My workaround was to enable nightly repo (test repo stores only single, latest version):
sudo dnf config-manager --set-enabled docker-ce-nightly

and then downgrade to previous version:
sudo dnf downgrade docker-ce

Docker version 18.09.1-beta1, build d3184b7 seems to work.

[andrewhsu]

Hmm..ok. Just chatted with @seemethere about this. Will look into it.

[seemethere]

Currently investigating this, will update later on what I find

[seemethere]

So socket activation was added in the latest beta for 18.09.1 for RPM based distributions, but the socket is not enabled by default, unlike DEB based distributions.

Quick workaround for this right now is to activate the socket manually using:

$ sudo systemctl start docker.socket

And then enable your docker.service using:

$ sudo systemctl reset-failed docker.service
$ sudo systemctl start docker.service

Will be submitting a patch later today to enable the docker.socket by default, will link that patch in this thread later on today.

Relates to:

• Add socket activation for RHEL based distributions docker-ce-packaging#273
• [18.09] Add socket activation for RHEL based distributions docker-ce-packaging#274

[bjornmagnusson]

Beta2 works fine :)