Poor performance with docker desktop on windows in WSLv2

[b-hayes]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: 885BBE16-B2D2-4A36-87D5-32DB7E8A0143/20211124221900

Actual behaviour

Running unit tests in Ubuntu in WSLv2 running Ubuntu involving many calls to a test database takes 10-20 minutes.

• SSD usage at 80% or higher.
• 8GB of ram approx consumed by vmmem in the process.
  (yes the project files are inside wsl)

Open a Virtual machine in VMware player with same version of Ubuntu, running the same tests completes 46 seconds.

• SSD usage at 1%
• The VM is only given 4gb ram is is only using 2gb of it. (while also running gnome desktop environment).

This is a new machine fresh copy of windows docker wsl etc. I see the same performance issues on all my computers.

This is also a reflection of docker performance on windows in general, most things are slower in some way but you don't really notice it too much until you do some long-running operation or some part of a web app that triggers a lot of database calls.

Expected behaviour

Efficient, fast responsive docker containers.

While I understand there is bound to be some performance overhead I'd not expect docker desktop to be so bad.
VMware Player also has overheads and yet it's thousand times more efficient.
Docker on windows has always been terrible and I was really hopeful for the new WSLv2 direction and at one point it was impressively fast. But after nearly 2 years it's becoming really bad again.

Information

• Windows Version: 10 v21H1 build 19043.1348
• Docker Desktop Version: 4.2.0 (70708)
• WSL2 or Hyper-V backend?
• Are you running inside a virtualized Windows e.g. on a cloud server or a VM:

Steps to reproduce the behavior

1. install windows, wslv2, docker desktop and a ubuntu 20 LTS distro
2. create a docker network
3. create an http container with apache2 and php
4. create a mysql container
5. create say a thousand unit tests in php unit with database calls (perhaps in a loop? I cant provide my project sorry)
6. watch your ram and SSD get eaten in task manager, record the time.
7. create a VM in VMplayer with the same version of ubuntu and give it access to a few CPU cores and 4gb of ram.
8. run the same tests and see that it is blazing fast in comparison and consumes hardly any resources at all.

IF this is not happening as bad for others then there must be something to do with how I install it on every system that I have because it has become like this on all my machines.

[TomasBarta123]

D5ED8F61-9B40-4EA3-9B21-B41E28C77488/20211127092238

[stephen-turner]

Is the database stored in a named volume, or in your Linux filesystem, or in your Windows filesystem? We advise against the last option because of known performance problems (see https://docs.docker.com/desktop/windows/wsl/#best-practices). Windows manages the sharing of files from the Windows filesystem into the WSL distro, so it's out of our control.

[b-hayes]

Database is inside a mysql container on another port as part of a network.

All files are inside WSLv2 file system. Nothing at all running on NTFS.

[b-hayes]

Looks like the data is inside a volume

    volumes:
      - ./mysql/data:/var/lib/mysql

The mysql/data folder is inside WSLv2 file system.

[stephen-turner]

Hmmm, in that case the only other thing I can think of is increasing the memory allowed to WSL in your .wslconfig. We have sometimes seen degraded performance if that is too low. Microsoft reduced the default at some point, which could explain why you say it was faster a couple of years ago.

[b-hayes]

It's not a memory limit issue. Mines set to 12GB. Can potentially go higher but I really shouldn't have to when I can run 6 projects like this with 40+ containers total and an entire OS + GUI inside a VM with half that ram limit and have it run 20 times faster.

To be sure I double checked wslconfig, my ram limit is 12GB.
Shut down all containers, shutdown docker and shutdown wsl entirely to clear out VMmems ram usage.
Then I spin up docker and only start the project set of containers to run the example test suit mentioned in the OP.

Results are identical, VMmem is using 5GB of the 12 for this project only, SSD usage rockets to 90% and the test still takes 20minutes to complete. (compared to 90 seconds inside the VMWare for the same project).

Excessive ram usage is a part the problem but its not hitting a limit and then slowing down.

Side note: I tried using tmpfs as the volume for MySQL on a smaller project instead. At the cost of even more ram used the SSD was no longer getting smashed, the tests were faster than the Linux VM. So potentially if I go buy even more ram and migrate all projects to use tmpfs in local dev then they'll all run super fast however, this is getting a bit silly at this point I shouldn't have to upgrade to 64GB of ram to have docker desktop run projects that only need 5GB.

[joeltoth]

This issue is occurring for us in our dev environments running docker v3.6.0 and 4.x.x and the symptoms are similar but we are running Windows containers using the WSL2 backend (mixed mode). We also have an Oracle DB with the database files stored locally in a folder in Windows and then mapped to a volume inside the container. What I have seen by doing an exec into the container is that the virtual memory used by Oracle keeps increasing until it hits WHATEVER ram limit is set. If this is set to 8 or 12GB I can watch the task manger process climb in memory usage anytime the website makes calls to the database. There is no memory being reallocated so it just keeps consuming the pagefile until it wither stops the container or completely slows my PC down. It feels like a memory leak but I am not sure. Oracle is pull/push from the container to the local files on the windows server but how could this be the cause?

This used to work in earlier docker versions before upgrading to using WSL2. I never thought this was an issue when we first upgraded so that doesn't seem like it would be the problem.

NOTE: I have seen here that Ben Dodds has had a similar issue recently and has reported it against docker-compose as an issue with the presence of custom port mappings: docker/compose#8978

Can anyone comment as this looks like we have multiple issues all pointing at poor performance in docker 3.6.0 which is not yet resolved in the latest version which also follows a new pricing model.

Can anyone provide any context and direction here? Where should I report this issue if this is not the right place?

[b-hayes]

@joeltoth in my isolating test the ram limit wasn't being reached (7GB unused) and the performance still tanked and hit 90% SSD usage 🤷‍♂️.

[b-hayes]

Now that we are having to pay for quite a lot of licences for all our devs would be nice to have at least some acknowledgment of the performance issues.
Perhaps we should cancel and invest time into finding the old free version that doesn't run slow.

[joeltoth]

@b-hayes I was writing to fast as I am trying to also work full time here and not get behind in my companies projects. My mistake is that I said the host memory when I meant that the virtual memory inside the container is being consumed (growing Priv working set for oracle only when my site is used and making calls to Oracle). This worked when the host was 20H1 and builds prior to 1903.

I was thinking maybe since my host PC went forward (PC upgraded to 20H2), the containers if not also upgraded, I understand server core will be running backward compatible and maybe that is where Oracle is being handled different than prior to the upgrade to 3.6.0?  Just trying to think outside the box here.
Oracle & Docker on windows:
Oracle Cloud Visualizations

The neat thing here is that when things start up I notice the heavy 2.9gbs but if I leave my containers running and come back after 30 min to 45min they will; drop down like this to under 400MB each.  Then when I start using one of those containers by hitting my 8.1.1 WebUI site I will see the Priv working set slowly grow over time.  Depending on the application, it may be faster or slower in growth depending on how much it is drawing from the database

Oracle consumes all the virtual memory in the container as is does the Oracle work (writing to the volume mapped to an NTFS local drive C:\envs\container\db<Oracle volume files> for enterprise production backups of client data. once it reaches the limit it consumes your page file which is SSD and borks performance. Something here is not being handles well. Can anyone hotfix something to fix this?

It looks like there is a 20H2 version available that now finally supports process isolation for windows containers. Windows Container Version Compatibility | Microsoft Docs. https://docs.microsoft.com/en-us/windows-server/get-started/whats-new-in-windows-server-2004. Would process isolation help here?

[b-hayes]

I meant that the virtual memory inside the container is being consumed

Do containers have their own lower ram limits that are below what's allocated to WSL?

[b-hayes]

The problem still persists with Docker Desktop 4.6.1 (76265)

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[b-hayes]

The issue still persists but I give up hope for this to get resolved.

I'm intermittently switching projects over to TMPFS nowadays as a workaround and will be ditching windows soon to avoid it entirely.
Guess it will never get enough attention to be resolved since most people on windows have no idea how much performance they are losing if they have not experienced anything else.
I'll miss windows but seems the only way forward 😢.

[florianmuellerCH]

/lifecycle frozen

I agree that this an absolute pain! We're professionally working on OSX systems and I'm doing my own projects privately on Windows. The difference in performance is staggering! I've tried everything from switching WSL 2 Distros to removing unused containers and volumes to enabling MariaDB's query cache - nothing worked. If I'm loading a page that takes less than a second on my productive Drupal Environment on a server, the same page load takes at least 20 seconds locally. I have simply no other options than to stop using docker entirely and switch to anything low-key that just works.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked