No space left on device even after system prune -a --force

[rawrgulmuffins]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID: 23B1CA2B-2812-4B75-B64C-4573FF32192D/20190215204442

Expected behavior

Running docker run -rm commands won't cause disk full errors for all commands.

That docker system prune -a --force will free up said disk space issues.

Actual behavior

executing a docker run command leads to this error

Error response from daemon: mkdir /var/lib/docker/tmp/docker-builder553623694: no space left on device
make: *** [env] Error 1

If I run docker ls I see a few orphaned images but not a huge amount.

alexl@MBP000413 ~/git/example :)% docker image ls
REPOSITORY                                              TAG                 IMAGE ID            CREATED             SIZE
<none>                                                  <none>              b68ba6447529        8 hours ago         929MB
artifacts.dev.versive.com/docker-users-virtual/python   3                   ac069ebfe1e1        2 days ago          927MB
alexl@MBP000413 ~/git/example :)% docker image ls -a
REPOSITORY                                              TAG                 IMAGE ID            CREATED             SIZE
<none>                                                  <none>              58244b31cc25        8 hours ago         929MB
<none>                                                  <none>              b68ba6447529        8 hours ago         929MB
<none>                                                  <none>              0f198fea228d        8 hours ago         929MB
<none>                                                  <none>              69868a8eab93        8 hours ago         927MB
artifacts.dev.versive.com/docker-users-virtual/python   3                   ac069ebfe1e1

Running docker system prune -a -m and re-running does not alleviate the issue.

Build Command

docker build -f Dockerfile \
            --build-arg SOURCE_IMAGE=python:3 \
            -t example_pytest .

Run Command

docker run -a stdin -a stdout -a stderr --rm --mount source=example,target=/home/pytest example_pytest py.test --cov

Dockerfile

# Original inspiration for this file from
# https://stackoverflow.com/questions/35322452/is-there-a-way-to-sandbox-test-execution-with-pytest-especially
ARG SOURCE_IMAGE
FROM ${SOURCE_IMAGE}

RUN adduser --disabled-password --gecos "" --uid 7357 pytest
COPY ./ /home/pytest
WORKDIR /home/pytest

# setup the python and pytest environments
RUN pip install --upgrade pip setuptools pytest
RUN pip install --upgrade -r requirements.txt
RUN pip install --upgrade -r requirements_dev.txt
RUN python setup.py develop

USER pytest

Deleting Docker.raw is a temporary fix

This ticket has a user delete the Docker.raw file as a work around for a similar issue. My Docker.raw file is 60Gb according to ls.

ls -lha ~/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw
-rw-r--r--  1 alexl  HQ\Domain Users    60G Feb 15 00:13 /Users/alexl/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw

Deleting the Docker.raw re-running the run` command does fix my issue. For 2-3 runs fo the run command and then I'm back to getting disk space errors.

Information

Please, help us understand the problem. For instance:

• Is it reproducible?
  I have not attempted this on another computer yet.

• Is the problem new?
  Yes

• Did the problem appear with an update?
  I did download the 2.0.0.3 update on the same day

• macOS Version:

[newtack]

I have the same error message on windows

[gswallow]

Increasing the amount of memory allocated to Docker Engine may alleviate this problem. I increased from 2 GB to 12 and the containers I was running were able to grow from 3.1GB to 5.4GB in size, where they were killed prior. My problem was probably due to some huge overlayfs layer that had to be cached in memory until it could be flushed to disk.

Obviously, allocating 12 GB of RAM to Docker Engine is a bit much, so you can try scaling down to something like 4 GB of RAM and up the swap to 4 GB. YMMV.

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[rawrgulmuffins]

/remove-lifecycle stale

[rawrgulmuffins]

Re-confirmed that this is still an open bug and it is not correct to close this.

[birgit-babbel]

I had the same issue today.
Running docker system df showed that for me the "Local Volumes" part took up the most space. docker volume prune solved it for me.

[rawrgulmuffins]

This issue still exists and should not be closed.

[ivanayov]

I have the same issue. What happens is that during a build a container is reaching full capacity of 174.9GB and docker goes out of memory. Not solved with docker container prune, as this is reproduced every time.
What's interesting is that I was building that image successfully before updating to docker 19.03.8, but I cannot say for sure that the problem has anything to do with the version.

[rawrgulmuffins]

I'm considering opening a new issue and pointing to this one since it seems like the project doesn't look at closed issues.

Opinions?

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked