-
Notifications
You must be signed in to change notification settings - Fork 424
[18.09 backport] Delete stale containerd object on start failure #154
[18.09 backport] Delete stale containerd object on start failure #154
Conversation
containerd has two objects with regard to containers. There is a "container" object which is metadata and a "task" which is manging the actual runtime state. When docker starts a container, it creartes both the container metadata and the task at the same time. So when a container exits, docker deletes both of these objects as well. This ensures that if, on start, when we go to create the container metadata object in containerd, if there is an error due to a name conflict that we go ahead and clean that up and try again. Signed-off-by: Brian Goff <cpuguy83@gmail.com> (cherry picked from commit 5ba30cd) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ping @tonistiigi @cpuguy83 PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Interesting; both Power and S390x are failing with; https://jenkins.dockerproject.org/job/Docker-PRs-powerpc/13358/console https://jenkins.dockerproject.org/job/Docker-PRs-s390x/13246/console
|
From the daemon logs of that test;
|
I think the remaining failures are flaky tests |
https://github.com/docker/docker-ce/releases/tag/v18.09.3 The more important fixes in this version: * When copying existing folder, ignore xattr set errors when the target filesystem doesn't support xattr. docker-archive/engine#135 * Graphdriver: fix device mode not being detected if character-device bit is set. docker-archive/engine#160 * Fix nil pointer derefence on failure to connect to containerd. docker-archive/engine#162 * Delete stale containerd object on start failure. docker-archive/engine#154
https://github.com/docker/docker-ce/releases/tag/v18.09.3 The more important fixes in this version: * When copying existing folder, ignore xattr set errors when the target filesystem doesn't support xattr. docker-archive/engine#135 * Graphdriver: fix device mode not being detected if character-device bit is set. docker-archive/engine#160 * Fix nil pointer derefence on failure to connect to containerd. docker-archive/engine#162 * Delete stale containerd object on start failure. docker-archive/engine#154
…ll dockers are down except database It's an issue in docker engine, which has been resolved in PR#154 docker-archive/engine#154 And in this commit, we will update the docker to 19.03.0 for this Signed-off-by: Dante (Kuo-Jung) Su <dante.su@broadcom.com> Change-Id: Iecfb7b312abfbcc7741cdcd8b506f9d6c19c4eef
Should this bug be fixed in 19.03.6? |
Based on the error message it seems like there is still a task running, which this PR does not handle and sounds like a very different problem. |
Could you open a new issue with the details? Any clues on how the system got into that state? |
Done. Not really. I believe that it was caused by installing some updates (including |
backport of moby#38364 for 18.09
fixes moby#38346 for 18.09
containerd has two objects with regard to containers.
There is a "container" object which is metadata and a "task" which is
manging the actual runtime state.
When docker starts a container, it creartes both the container metadata
and the task at the same time. So when a container exits, docker deletes
both of these objects as well.
This ensures that if, on start, when we go to create the container metadata object
in containerd, if there is an error due to a name conflict that we go
ahead and clean that up and try again.
Signed-off-by: Brian Goff cpuguy83@gmail.com
(cherry picked from commit 5ba30cd)
Signed-off-by: Sebastiaan van Stijn github@gone.nl
- What I did
- How I did it
- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)