Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker.qcow2 never shrinks - disk space usage leak in docker for mac #371

Closed
yakticus opened this issue Aug 19, 2016 · 309 comments
Closed

Docker.qcow2 never shrinks - disk space usage leak in docker for mac #371

yakticus opened this issue Aug 19, 2016 · 309 comments

Comments

@yakticus
Copy link

yakticus commented Aug 19, 2016

Expected behavior

Amount of disk space used under ~/Library/Containers/com.docker.docker/ should shrink when deleting images and/or containers

Actual behavior

Amount of disk space used under ~/Library/Containers/com.docker.docker/ doesn't shrink after deleting ~50 containers and ~141 images (out of a total of 194). My Docker.qcow2 was 41GB both before and after doing these deletions.

On the Diagnose & Feedback screen I see an error that looks relevant:

Docker for Mac: version: mac-v1.12.0.1
OS X: version 10.11.6 (build: 15G31)
logs: /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130.tar.gz
[OK]     docker-cli
[OK]     app
[OK]     moby-syslog
[ERROR]  disk
         disk check failed with: Failure("exec: /Applications/Docker.app/Contents/Resources/bin/../../../Contents/MacOS/qemu-img check /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 > /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130/qemu-img-check.stdout 2> /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130/qemu-img-check.stderr: exit 2")
[OK]     virtualization
[OK]     system
[OK]     menubar
[OK]     osxfs
[OK]     db
[OK]     slirp
[OK]     moby-console
[OK]     logs
[OK]     vmnetd
[OK]     env
[OK]     moby
[OK]     driver.amd64-linux

Information

For some history, see moby/moby#23437

  • Diagnostic ID from "Diagnose & Feedback" in the menu.
    I might not have this anymore since I had to do a factory reset in order to reclaim the disk space. After the factory reset, it is: 5F7A165B-83AC-4514-AFC4-DB53430D2E9D
  • a reproducible case if this is a bug, Dockerfiles FTW

Easy to reproduce by pulling some images, observing the size of Docker.qcow2, deleting the images and seeing that the size of Docker.qcow2 does not change. Let me know if more info is needed

  • page URL if this is a docs issue or the name of a man page
  • host distribution and version (OSX 10.10.x, OSX 10.11.x)

OSX 10.11.6

Steps to reproduce the behavior

  1. pull several images
  2. run a couple of containers
  3. observe size of ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
  4. stop and delete all containers
  5. delete all images
  6. repeat step 3. Size has not shrunk despite there being no images or containers

Transcript:

$ docker pull frolvlad/alpine-scala
Using default tag: latest
latest: Pulling from frolvlad/alpine-scala
e110a4a17941: Pull complete
052edde983dc: Pull complete
44b3390a7725: Pull complete
0f6b9d2879c1: Pull complete
Digest: sha256:1d9f6fd716526764a4ba6e389b635cbef8c5d413216bf5d00ca5175fc0d9bac8
Status: Downloaded newer image for frolvlad/alpine-scala:latest
$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
1290    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
$ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
2f0243478e1f: Pull complete
d8909ae88469: Pull complete
820f09abed29: Pull complete
01193a8f3d88: Pull complete
Digest: sha256:8e2324f2288c26e1393b63e680ee7844202391414dbd48497e9a4fd997cd3cbf
Status: Downloaded newer image for ubuntu:latest
$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
1456    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
$ docker pull python
Using default tag: latest
latest: Pulling from library/python
357ea8c3d80b: Pull complete
52befadefd24: Pull complete
3c0732d5313c: Pull complete
ceb711c7e301: Pull complete
4211bb537697: Pull complete
71f9074c0739: Pull complete
3e5349707036: Pull complete
Digest: sha256:a755ad5a30b2146f9a132bf74107aa5be9a26b7cc77956b379af678e0f311b3c
Status: Downloaded newer image for python:latest
$ docker run ubuntu
$ docker run -it ubuntu bash
root@a2ae24adfd12:/# exit
$ docker run -itd ubuntu bash
95f5bed47a790d2c8d60544cbc1fa870e1ef4b8c04653508bed34fb25e650536
$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
95f5bed47a79        ubuntu              "bash"              9 seconds ago       Up 9 seconds                                    hungry_lalande
a2ae24adfd12        ubuntu              "bash"              28 seconds ago      Exited (0) 21 seconds ago                       sharp_borg
a567a8a7aed3        ubuntu              "/bin/bash"         53 seconds ago      Exited (0) 52 seconds ago                       ecstatic_poitras
$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
2371    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
$ docker stop hungry_lalande
hungry_lalande
$ docker rm -v $(docker ps -a -q -f status=exited)
95f5bed47a79
a2ae24adfd12
a567a8a7aed3
$ docker rmi $(docker images -q)
Untagged: ubuntu:latest
Untagged: ubuntu@sha256:8e2324f2288c26e1393b63e680ee7844202391414dbd48497e9a4fd997cd3cbf
Deleted: sha256:f8d79ba03c00bbcd8079cf05b7526ac8f4f422744aad8c3747a29a38ed8c4a41
Deleted: sha256:b4119cffabb62bd6374f5b06adcb820344490252b621cb8f6a8fad7f6f61b3ae
Deleted: sha256:83a17331cb4259e2690395586b641aabadf30256f25a032019b816a53a06ae6e
Deleted: sha256:30ae5404a524e7a713f86ebe832b1e198d1270d62b27c860721055b890a3ae9c
Deleted: sha256:d8d865b23727fa63dc7d4e9f02efc3e2433c8f220972689c6b6ba09e8136c162
Untagged: python:latest
Untagged: python@sha256:a755ad5a30b2146f9a132bf74107aa5be9a26b7cc77956b379af678e0f311b3c
Deleted: sha256:a08871375facdc1619fc6d19608a4555db479219b2853e2e11de05eddf97e4cb
Deleted: sha256:0878d93537f4951aa5fa1ca6fd7ef085aa993e520b5af01148d84704ef49abcf
Deleted: sha256:2d5707b912a332c89665a078a2dabb088cbb4e6982b0d3db36217c89640074da
Deleted: sha256:5f6d1bd4cff3a349d2b5989bf04295987b6d679c411357612523341d4b6f2264
Deleted: sha256:0e54104f61d3ae37dc10fbc275796123a40b54859776fb6e937bb5c39655c3e6
Deleted: sha256:d3c72419ca593d81f207884b70670fd69c4fa3bdecf926ecbbbd9d3bbdd6c7bf
Deleted: sha256:016fc440b1e6b49d889b0f325d2439c82c92f554d3f015db67ffde823bd2ff3e
Deleted: sha256:2f71b45e4e254ddceb187b1467f5471f0e14d7124ac2dd7fdd7ddbc76e13f0e5
Untagged: frolvlad/alpine-scala:latest
Untagged: frolvlad/alpine-scala@sha256:1d9f6fd716526764a4ba6e389b635cbef8c5d413216bf5d00ca5175fc0d9bac8
Deleted: sha256:ac1474efe52e017303f24209fff310c734eed81cdd947604545f160d03f2caad
Deleted: sha256:a08f99075408eb40ce74942b01d08003c8d923b8df0b0e593b73b13540850171
Deleted: sha256:5c86a5cfb5b67143c7ec277ac533176abb9906072eac82de512f8e822b06e769
Deleted: sha256:2615ff2c2f8c27913239080269f5be78d541b2564592856548b435f5daa42514
Deleted: sha256:4fe15f8d0ae69e169824f25f1d4da3015a48feeeeebb265cd2e328e15c6a869f
$ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 0
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.15-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.954 GiB
Name: moby
ID: KQ7I:O3AX:CY77:DKU4:6LZR:Y6RF:ZM2P:U2YA:TZUV:3YDV:DQPU:OTJV
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 18
 Goroutines: 29
 System Time: 2016-08-19T21:24:33.294921776Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8
$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
2371    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
@kommen
Copy link

kommen commented Aug 19, 2016

Thanks for filing it over here @yakticus.

I also still have the problem and in the forum others have reported the same: https://forums.docker.com/t/docker-doesnt-release-disk-space-used-by-library-containers-com-docker-docker/15194

@lxhunter
Copy link

+1

@simond
Copy link

simond commented Aug 23, 2016

Yep, same issue here.

@yankcrime
Copy link

I've found the quickest way (as mentioned in other issues / forum threads) to fix this is to reset the client which forces the recreation of Docker.qcow2. If you don't want to do that however, there is another fix, once you've cleaned up any dangling images or exited containers.

NB: You'll need to install qemu via Homebrew as this process requires qemu-img to recompress the qcow2 disk image.

Connect to the VM with screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty and then login as root

Fill up the disk with zeroes:

dd if=/dev/zero of=/var/tempfile

This'll take at least couple of minutes on SSD. I wouldn't bother if you're on spinning rust...

Then you need to rm /var/tempfile, logout of the VM, and quit the Docker client entirely. Now we can recompress the disk:

$ pwd
/Users/nick/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux
$ mv Docker.qcow2 Docker.qcow2.original
$ du -hs Docker.qcow2.original
12G     Docker.qcow2.original
$ qemu-img convert -O qcow2 Docker.qcow2.original Docker.qcow2
$ rm Docker.qcow2.original
$ du -hs Docker.qcow2
772M    Docker.qcow2

@geerlingguy
Copy link

@yankcrime's workaround works... but it's a lot of steps to make Docker not continuously gobble up all the remaining space on my poor little MacBook's SSD. Between this and osxfs perf issues (#77) requiring similarly involved file sync workarounds, it's really hard to recommend Docker for Mac to anyone who's not an infra hacker at heart... I don't want to have to spend half my day helping other devs figure out why all their disk space is gone!

@kevinsimper
Copy link

Just deleting it seems to be the easiest solution, but it would be nice if the client could reclaim space itself!

@gumho
Copy link

gumho commented Sep 6, 2016

I've been running into this issue as well on the latest Stable Docker for Mac. @yankcrime's workaround does work. Would be nice to be able to provide an option to grow the disk size as well.

@creynders
Copy link

👍 💯

@chrisclark
Copy link

To add to this, I just deleted my qcow2 file and restart docker and disconnect completely from the internet, and docker ps shows no running containers, and docker images -a lists two containers, both of which are < 300mb...Docker will recreate the qcow2 file and I can just sit there, watching it grow in size, while Docker, and I, do absolutely nothing. There are no running containers, no nothing. What on earth is it even doing? I'm literally sitting here staring at the file in Finder (after re-enabling my network...) watching it grow from 200kb to...

...1.14GB
...1.15GB
...1.16GB

This is nuts. It'll happily just keep growing until it eats all of my available hard disk space.

@mochaslave
Copy link

@chrisclark I have the same problem. There is no image, no container, qcow2 just keeping grow up, but until to 1.16GB. Seem 1.16GB is the VM basic requested working size.

@marvin-w
Copy link

Same issue for me

@AzatKhalilov
Copy link

Yep! The same issue for me. If often rebuild my images it's trouble,headache! After 5-6 rebuild image has 1G size but disk space decrease on 20-30G!!!!!

@dmill-bz
Copy link

dmill-bz commented Sep 14, 2016

Can we have an update on this? It's a really big issue that needs solving.
I can appreciate that it might not be an easy fix but some kind of feedback would be awesome.

@alex-ross
Copy link

Using Docker to manage multiple development environments and this image often ends up to take 60GB for me. It's a lot on my 256GB SSD. All dev environments is using Docker Compose so I can reset docker once in a while but it's really annoying that I have to do so.

@anonymuse
Copy link

I've been using Docker for Mac and the Docker Machine images in order to experiment effectively with Docker Swarm and other orchestration features -- the large qcow file and not-insubstantial disks of my other Docker Machine hosts make it difficult to fit onto a MBP sized HDD.

Any update here?

@wbednarski
Copy link

wbednarski commented Sep 20, 2016

If you can remove all images/containers then:

Stop Docker.

docker rm $(docker ps -a -q)
docker rmi $(docker images -q)
docker volume rm $(docker volume ls |awk '{print $2}')
rm -rf ~/Library/Containers/com.docker.docker/Data/*

Start Docker, you have yours GB back.

@anonymuse
Copy link

@wbednarski -- that's helpful if keeping images/containers around wasn't a big deal (as it might not be on a server) but it tends to be important in development (on my laptop.) I've read also that using the "Reset to factory defaults" button can clean up your installation too.

Thanks for pointing out your solution.

@wbednarski
Copy link

wbednarski commented Sep 20, 2016

Didn't try reset. I'll check out it soon :]

@anonymuse you can look at @yankcrime solution #371 (comment) if you want to keep images/containers.

@lijingpeng
Copy link

@yankcrime I tried as you said, but nothing changed.

du -hs Docker.qcow2
17G Docker.qcow2

@anonymuse
Copy link

Frank, how sure are you that you followed all of the steps correctly? Do
you have the output of your actions that we could take a peek at?

I forgot to run the dd myself first time around!

On Sep 23, 2016 12:07 PM, "Frank" notifications@github.com wrote:

@yankcrime https://github.com/yankcrime I tried as you said, but
nothing changed.

du -hs Docker.qcow2
17G Docker.qcow2


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#371 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACnE0XEI6hkeOnvHaxI2_WR4CYBR1gCgks5qs_kigaJpZM4Jo3w2
.

@rafiki270
Copy link

any news on this?

@semerda
Copy link

semerda commented Sep 12, 2019

DockerEats @ 64 GB! https://www.flickr.com/photos/semerda/48720232588/
My ~/Library/Containers/com.docker.docker/ is a whopping 64.01 GB.

Running 'docker system prune' says:
Total reclaimed space: 1.986GB

But when I check ~/Library/Containers/com.docker.docker/ for size after prune it still says 64.01 GB.

Am I doing something wrong or is 'docker system prune' useless?

And why the heck is Docker.raw file so colossal?

@thaJeztah
Copy link
Member

@semerda read the discussion above; the Docker.raw file uses a sparse file format with (by default, but configurable) has a maximum size of 64GB ("logical size"). The actual ("physical") size is usually smaller, but you need to use slightly different commands to show that size; see https://docs.docker.com/docker-for-mac/faqs/#dockerraw-consumes-an-insane-amount-of-disk-space

Screenshot 2019-09-12 at 08 56 53

@danieldanielecki
Copy link

Super annoying bug, I was thinking what takes that much space on my Mac...

@adam-lee
Copy link

Whoa, this bug still persists after 3+ years? I just ran into this on the latest MacOS (Mojave / 10.14.6) with Docker Engine 19.03.2. After running docker volume rm my-volume, it is gone from the list, but my disk space is still occupied. Wonder what's taking so long to fix this? Isn't this as simply as removing the actual file once docker off loads it?

@akasandra
Copy link

akasandra commented Sep 28, 2019

@semerda read the discussion above; the Docker.raw file uses a sparse file format with (by default, but configurable) has a maximum size of 64GB ("logical size"). The actual ("physical") size is usually smaller, but you need to use slightly different commands to show that size; see https://docs.docker.com/docker-for-mac/faqs/#dockerraw-consumes-an-insane-amount-of-disk-space

Screenshot 2019-09-12 at 08 56 53

Mac thinks disk has no free memory (literally 0B for me frequently without disk reset). The image itself could be empty or contain only small images and containers.

UPD: ANYONE, PLEASE READ

this comment first for further discussion. I thought i did read the read, but there were so many comments, so it was hidden, but it is actually the explanation why it is very hard to fix.

#371 (comment)

An here is a solution, the only one available besides resetting qcow2 disk fully: #371 (comment)

I hope it helps to reduce further useless commenting :D

@djs55
Copy link
Contributor

djs55 commented Sep 30, 2019

@adam-lee to free space on the host the VM has to issue TRIM commands. It currently will execute an fstrim after images are deleted but not volumes.

Perhaps give it a kick with

 docker run --privileged --pid=host docker/desktop-reclaim-space

If you are using Docker.raw the results will be instantaneous according to:

ls -ksh ~/Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw 

If you are using Docker.qcow2 it will take several minutes to perform an online compaction.

@tcurdt
Copy link

tcurdt commented Sep 30, 2019

@djs55 at least for me this does nothing

$ ls -la  ~/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw 
-rw-r--r--@ 1 tcurdt  staff  15999172608 Sep 30 09:29 /Users/tcurdt/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw
$ docker run --privileged --pid=host docker/desktop-reclaim-space
$ ls -la  ~/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw 
-rw-r--r--@ 1 tcurdt  staff  15999172608 Sep 30 09:29 /Users/tcurdt/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw

@tcurdt
Copy link

tcurdt commented Sep 30, 2019

Ah - maybe this is a physical vs logical issue.

$ ls -ksh ~/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw 
1836084 /Users/tcurdt/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw

@mattolenik
Copy link

mattolenik commented Oct 11, 2019

@djs55 your fix worked for me, at least for now. Thank you!

Very frustrating that these bugs keep being closed/ignored. This is a fundamental resource leak, not something like a tiny UI issue that may get fixed by happenstance as someone does some refactoring. It's not something that you can close due to "inactivity." The fact that macOS is different and makes it difficult is irrelevant.

I mean, isn't it concerning that the "official fix" is to download a container and run it? Shouldn't that just be a part of Docker for Mac? Especially when that fix is very hard to find. I ran across several other GitHub issues and Stack Overflow questions, all with varying solutions, until I found this thread. I tried reinstalling Docker, all sorts of wasted time. Should this not run during the uninstall, at the very least? Because reinstalling didn't fix the issue, either. I uninstalled with the uninstall built into Docker for Mac and it still didn't fix the issue. This means the uninstaller does NOT actually uninstall Docker. This seems rather alarming to me, and would clearly be unacceptable for any other desktop software.

@MrUpsidown
Copy link

Total fail.

38846922752 15 oct 15:35 Docker.qcow2
docker run --privileged --pid=host docker/desktop-reclaim-space
34058207232 15 oct 15:38 Docker.qcow2

When I should have a maximum of 8 to 10 GB in there. So that does not work. Full stop.

@vocatan
Copy link

vocatan commented Oct 15, 2019

Note the different utilities report file sizes in different ways.

When I use ls it appears to be taking 60G

ls -Falh /Users/billm/Library/Containers/com.docker.docker//Data/vms/0/data/Docker.raw
-rw-r--r--  1 billm  staff    60G Oct 15 14:05 /Users/billm/Library/Containers/com.docker.docker//Data/vms/0/data/Docker.raw

but when I use du the actual size of just 37G is revealed.

$ du -ksh /Users/billm/Library/Containers/com.docker.docker//Data/vms/0/data/Docker.raw
 37G	/Users/billm/Library/Containers/com.docker.docker//Data/vms/0/data/Docker.raw

That's fair, given that I have a substantial # of images and containers:

$ docker images |wc -l
     127
$ docker ps -a |wc -l
      62

@KyeRussell
Copy link

There are countless posts here explaining how sparse images work, and why different tools will report different sizes. Yet every few days a new person jumps into the thread with a snarky comment incorrectly showing their huge image. Can y’all try to read the thread before posting please.

@jkahrs
Copy link

jkahrs commented Oct 16, 2019

There are countless posts here explaining how sparse images work, and why different tools will report different sizes. Yet every few days a new person jumps into the thread with a snarky comment incorrectly showing their huge image. Can y’all try to read the thread before posting please.

Maybe this should be extracted into the official docs to improve the overall discoverability of the issue? After all this thread contains a few hundred replys

@CamJN
Copy link

CamJN commented Oct 16, 2019

Quoting from https://docs.docker.com/docker-for-mac/space/

It might take a few minutes to reclaim space on the host depending on the format of the disk image file:

If the file is named Docker.raw: space on the host should be reclaimed within a few seconds.
If the file is named Docker.qcow2: space will be freed by a background process after a few minutes.

Space is only freed when images are deleted. Space is not freed automatically when files are deleted inside running containers. To trigger a space reclamation at any point, run the command:

$ docker run --privileged --pid=host docker/desktop-reclaim-space

Note that many tools report the maximum file size, not the actual file size. To query the actual size of the file on the host from a terminal, run:

$ cd ~/Library/Containers/com.docker.docker/Data
$ cd vms/0/data
$ ls -klsh Docker.raw
2333548 -rw-r--r--@ 1 username  staff    64G Dec 13 17:42 Docker.raw

In this example, the actual size of the disk is 2333548 KB, whereas the maximum size of the disk is 64 GB.

@charliereese
Copy link

This Stack Overflow thread fairly succinctly explains the high level cause and a quick solution (not perfect, but I was happy with it): https://stackoverflow.com/questions/49887747/what-is-docker-qcow2

Although I thought this behaviour was odd at first, after reading the SO thread, this doesn't really seem like a bug.

Maybe Docker for Mac should default to a smaller disk image size (16GB vs 64GB)?

@wanliqun
Copy link

I've tried this to trigger a manual TRIM from this blog:
Docker for Mac: reducing disk space (https://djs55.github.io/jekyll/update/2017/11/27/docker-for-mac-disk-space.html)

docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n fstrim /var/lib/docker

@spacelatte
Copy link

@wanliqun link has extra parenthesis at the end :)

Should be:

https://djs55.github.io/jekyll/update/2017/11/27/docker-for-mac-disk-space.html

Btw since el-capitan got EOL (1) and Sierra (10.12) has APFS support (2), .qcow2 does not matter anymore.

@dmasterstempus
Copy link

dmasterstempus commented Apr 13, 2021

@charliereese

This Stack Overflow thread fairly succinctly explains the high level cause and a quick solution (not perfect, but I was happy with it): https://stackoverflow.com/questions/49887747/what-is-docker-qcow2

Although I thought this behaviour was odd at first, after reading the SO thread, this doesn't really seem like a bug.

Maybe Docker for Mac should default to a smaller disk image size (16GB vs 64GB)?

From the linked stackoverflow answer (emphasis mine):

You can stop Docker and delete this file, however deleting it will also remove all your containers and images. And Docker will recreate this file on start.

Do you consider that a solution?!

@ds-clearago
Copy link

See https://github.com/wanliqun/macos_docker_toolkit – that seems to do the trick. On my end docker system prune helped a lot.

@joaocc
Copy link

joaocc commented Sep 18, 2021

Hi
When I try to run the reclaim disk space scripts on macos bigsur (11.5.2) and docker 4.0.1 (68347), in x64 mode, I get the following errors. it seems it is unable to set namespace.
Any ideas pls? Thx

$ docker run --rm --privileged --pid=host docker/desktop-reclaim-space
setns:mnt: Invalid argument

(base) bash-5.1$ docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n fstrim /var/lib/docker
nsenter: reassociate to namespace 'ns/mnt' failed: Invalid argument

@aaronkurz
Copy link

I just got the error mkdir: cannot create directory '/bitnami/postgresql/data': No space left on device

For me, docker system prune worked! 👍

@niccolomineo
Copy link

niccolomineo commented May 15, 2023

I have been struggling with this since forever.

Out of 58 gbs, pruning everything would only free a total of 11.

Only way to gain the whole space again is to manually delete ~/Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw

@lorenrh
Copy link
Member

lorenrh commented May 30, 2023

Closing this issue because a fix has been released in Docker Desktop 4.20. See the release notes for more details.

@lorenrh lorenrh closed this as completed May 30, 2023
@chamini2
Copy link

Hey @lorenrh, the release notes say

You can now reclaim disk space more quickly when files are deleted in containers. Related to docker/for-mac#371.

And that links here.

Can you please explain how can this be done?

@larssb
Copy link

larssb commented Jul 17, 2023

@lorenrh is this

Screenshot 2023-07-17 at 21 32 03

what you refer to?


Edit:

Hmm seems like it:

Screenshot 2023-07-17 at 21 36 31

From > https://docs.docker.com/desktop/faqs/macfaqs/

@tyanyaw
Copy link

tyanyaw commented Sep 2, 2023

Come 2023, already done all docker system prune & clean, but the Docker.raw size still big.
after searching, I found with method to :

  1. SAVE needed image to exclude
  2. DELETE Docker.raw
  3. LOAD chosen image back

Here the script to automate it :
clean-docker-for-mac.sh

EXAMPLE :
Chosen image : docker/getting-started alpine
Command : ./clean-docker-for-mac.sh docker/getting-started alpine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests