WSL 2 consumes massive amounts of RAM and doesn't return it

[LordMonoxide]

• Your Windows build number: 18917

• What's wrong / what should be happening instead: WSL 2 starts using huge amounts of RAM after a while, just using it like normal. At the moment I'm using phpstorm, and did a dump/load of a database. Vmmem is using 7 GB of my 16 GB of RAM and not returning any, even though Ubuntu is actually using much less. I have seen it grow until nearly 100% of my system memory is in use, and it will not release it until I shut down the WSL 2 VM.

This may or may not be related to #4159

corey@Corey-Laptop:/mnt/c/WINDOWS/system32$ vmstat -s
     15235516 K total memory
       920348 K used memory
      1886048 K active memory
      6434312 K inactive memory
      6606548 K free memory
        76280 K buffer memory
      7632340 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
       163729 non-nice user cpu ticks
          298 nice user cpu ticks
        13177 system cpu ticks
     68988300 idle cpu ticks
         8962 IO-wait cpu ticks
            0 IRQ cpu ticks
        10022 softirq cpu ticks
            0 stolen cpu ticks
      1481417 pages paged in
      6792976 pages paged out
            0 pages swapped in
            0 pages swapped out
      1079177 interrupts
      5131981 CPU context switches
   1560599814 boot time
         8772 forks

[Drakota]

Same here with Docker running on Ubuntu 18.04

[benhillis]

Thanks for opening the issue. We have a fix for this in the works.

[mithunshanbhag]

After moving to WSL2, VmMem seems to be constantly pegging my CPU. Please see attached screenshots.

Anything I can do to help your team troubleshoot?

Relevant details:

• Ubuntu 18.04 and Debian on WSL2, but none are running anything at the moment.
• Windows insider build 18917.1000 on Macbook Air (8 GB, 4 proc).

[crispinb]

After moving to WSL2, VmMem seems to be constantly pegging my CPU. Please see attached screenshots.

I'm experiencing this too. It happens every time I run wsl after some elapsed time (varying from minutes to hours). It continues regardless of what's running in wsl, and I need to issue a wsl --shutdown to stop it.

Windows insider build: 18922.1000
Linux distro: 18.04

I assume this really belongs in a new issue, but wanted to group my comment with that of @mithunshanbhag so I'll leave it here.

[md0x]

Hi,
Same problem here.
Lots of memory consumption with WSL2, nodejs app and VS Code with Remote - WSL while very low consumption in WSL1 with the same tasks.

[liyo]

Same issue on Win build 18950

[zachChilders]

@benhillis - Do we have a rough ETA on the fix? I'm actually hitting OOM issues on a machine with 32 gigs while just running a normal ninja build that should succeed.

Build 18955

[benhillis]

@zachChilders - ETA on this one is a bit hard unfortunately. Certainly before WSL2 ships to non-insiders.

[avafloww]

Can confirm this on 18963 as well. WSL 2 ate most (29GB+ of 32GB) of my RAM after less than 15 minutes of uptime for no good reason; all I've done is a rsync from the host NTFS volume to the ext4 volume, and an apt-get update.

[david-dumke]

I can confirm the same issue on Microsoft Windows [Version 10.0.18963.1000]

C:\Users\david_d>wsl -l -v
NAME STATE VERSION

• Ubuntu-18.04 Running 2
  Ubuntu Stopped 1

I at the time was running gdb

[rob-solana]

how about a workaround? this just started happening for me (with 18965) and it has effectively killed my WSL 2 environment

[mbc07]

Depending of your usage, nocache utility can be used as a workaround to greatly reduce WSL 2 memory consumption, especially if whatever you're running inside it does a lot of filesystem operations...

[rob-solana]

does nocache follow children? I could start my shell with "nocache"...

[mbc07]

Haven't tried, but if I understood how nocache works correctly, it should...

[rob-solana]

thanks! nocache has changed my env from "completely broken" to "merely sucks"

[Beretta1979]

thanks! nocache has changed my env from "completely broken" to "merely sucks"

Hi, could you explain how nocache should be used?

[willbuilds]

Hi @benhillis , any update on when a fix for this might be deployed to insiders? My 8gb of ram is really getting hammered by this issue... :(

[rob-solana]

I wrap compilations in nocache to keep limping along. wrapping nocache around a shell works, but breaks emacs. I've also ordered more RAM :-(

[benhillis]

Sorry no firm timeline, certainly before WSL2 is out of Insider builds though.

[fabienheureux]

Hi @benhillis , do you have a workaround you could recommend though ?

[fabienheureux]

Well...I need it all day long and the memory consumption goes crazy after only a few minutes (using docker and a quite heavy stack: elastic search, lots of workers, some databases etc). So...not an option unfortunately. Maybe go back to linux while it is not fixed 😎

[weberdominik]

I am facing the same issue. Is there any workaround (except shutting wsl down) for this @benhillis ?

[benhillis]

Not currently. For some additional context this change requires some changes to the Linux kernel that are in the process of being upstreamed. We will be integrating these changes into the WSL2 kernel as soon as we can.

[apostolos]

Workaround: Create a %UserProfile%\.wslconfig file in Windows and use it to limit memory assigned to WSL2 VM.

Example

[wsl2]
memory=6GB
swap=0
localhostForwarding=true

This will still consume the entire 6GBs regardless of Linux memory usage, but at least it'll stop growing more than that.

Supported settings are documented here.

[mslinn]

IMO, using Docker to run everything adds complexity and overhead without commensurate benefit. I wrote an article describing how I run Ollama on WSL2.

[sarim]

trying .wslconfig method but it still useless as you can see, i have 32 GB RAM installed and limiting wsl ram usage to 2GB but vmmem still consuming lot of RAM even though it didnt show in task manager (possibly more than 40%) and if i keep it runn for 2-3 days it will grow up to 80-90%

tried using wsl --shutdown to release the RAM but still useless, cause i see that in task manager it still consume 80-(90% RAM

525 + 1.1G = ~1.6G. Task manager shows 1701mb ~ 1.6G. Whats the problem?

[onomatopellan]

limiting wsl ram usage to 2GB but vmmem still consuming lot of RAM even though it didnt show in task manager.

That doesn't make sense. Open Resource Monitor and look at the Memory tab for the real culprit. Also make sure there is not running another Windows user session.

[fdc2005]

I am having a similar problem with wsl 2 and Docker. After a few days of running smoothly, vmmem takes up 90-100% of CPU and remains at this level, even after I shutdown docker desktop, and issue multiple "wsl --shutdown" commands. Note that this is a laptop - could it be related to putting the laptop to sleep occasionally?

Here is what I see:

I have tried to out-wait it to see what happens, but in each case I've become frustrated after 15-20 minutes and end up rebooting to fix the problem. Here is what I see after the reboot - I don't know why "docker-desktop-data" and "docker-desktop" appear now in the list, but didn't prior to the reboot:

[swebs]

There is a different issue regarding high CPU use: #8529

[jordan-huitema]

Just tried running my auto HandbreakCLI encoder on WSL2 and encountered this.
On WSL2 start my Mem jumps to 25G total
On handbrakeCLI start im at 40G
after 12hrs I come home to see 64G (100%) and my C:/ SSD appears to be getting used a lot (im encoding from and to a network drive)

Meanwhile my proxmox lxc which has been running for days is using 12G with the exact same OS and config and 2x the cores (id assume that effects encoding memory usage)

To be clear this is a fresh barebones WSL2 with only FFMPEG, core-utils and handbreakCLI installed. its just nuts.
Weirdly i have also noticed in windows 11 that memory never really gets returned either, I just closed everything and all I have running right now is firefox and im at 40G??? Im not sure this is a WSL2 issue and might be windows 11 or a combo from them both.

[mslinn]

I only use WSL for small processes. For bigger processes, I either use VirtualBox or native Linux.

[onomatopellan]

For users with AMD gpu, the community discovered a memory leak in the Adrenaline v24 video drivers. Disable the integrated GPU or downgrade the video drivers to Adrenaline v23. More info in this thread:

#8725 (comment)

[Gabrielcarvfer]

I don't have this problem on Win10, but have it on Win11.
htop reports 3GB used, Windows task manager reports 6GB. That is without doing anything special.
Trashing ssds instead of reclaiming memory or killing the process due to OOM is criminal.

[developer992]

same issue 5 years later

[Nikita-Kudrin]

Simply copying a few gigs of data in WSL is enough to consume a lot of RAM by WSL.
cp /mnt/d/DATA/20Gb.zip /home/user/

[prototact]

So WSL 2.3.11 update does not improve the situation? There is nothing in the notes about it

[gustavozantut]

So WSL 2.3.11 update does not improve the situation? There is nothing in the notes about it

How would they mine Bitcoin in our pc if they address this issue

[DT-BMA]

Still having the same issue even with the latest WSL2. Docker-Desktop having its own memory leaks isn't helping either :/

This is on a always running VMWare VM with Nested Virtualized Enabled on an Intel Xeon Gold 6430, Windows 10 and fully updated VM, WSL etc.

WSL2 Ubuntu 22.04.03 has been allocated 20GB RAM max (Out of 64GB).
It only really uses 4GB atm with VSCode remoted into it.
Kubernetes/Docker Desktop says it uses 4-10GB depending on its memory bloat

I had it saying 54GB used yesterday, so I ran RAMMAP64.exe. Amazingly, it flushed the RAM, and I went down to 22-24GB, which is about typical for my dev environment.
Now, after a day of not even using it, and just having things running, after re-logging in my RAM went from 24GB to 46GB usage in an instant. Not to say the huge VMEM CPU killing will always get worse over time until almost full resource utilization, even with less memory leaks

As you can see here, "Driver Locked" memory is what has stolen all of my RAM, and repeatedly terminating WSL2 / Docker to clear RAM shouldn't be the solution

Even after stopping WSL2 and Docker, we still have 16-20GB of unusable memory noted by Task Manager.
Which then, after 4 more minutes, clears up to this:

[kmalloy24]

same problem here

[msoltan2]

Same here... The more I use WSL, the more I want to return to native Linux. It has way too many issues.

[kyriakos]

Avoid using Docker Desktop, switch to Docker CE directly in your WSL2 distro - noticed memory usage has been a lot better and you don't miss anything other than the clunky electron UI of docker-desktop.

[alleje02]

How is this issue still open after ~ 5 years?

[luciorq]

How is this issue still open after ~ 5 years?

Many users face issues because they don’t properly configure the Docker VM WSL2 limits when using Docker Desktop.

Additionally, there’s a general misunderstanding of how RAM is used in Linux. For more information, check out this link: https://www.linuxatemyram.com/

[moracabanas]

How is this issue still open after ~ 5 years?

I am sorry about being toxic at this point, but I bet for WSL from V1 as my team's main development environment.

Now after waiting and fighting against the memory leak bug for 3 years or so, we ended up giving up for that.

We also bought 32Gb RAM laptops in case they are usable and they couldn't handle building docker images in some cases without hitting swap (throttling by rising disk I/O to 100%).

It feels like Docker Desktop, VSCode & WSL2 are doing this RAM hoarding on purpose.

I also tried out the new auto memory reclaim serting twice and in both cases it destroyed my WSL file system and affected my windows filesystem with file corruption.
I had to format and start again after that.

Sadly we are migrating to Linux desktop and pushing happy people on windows to a new OS, something we hate to do honestly.

I really hope a comeback on this issue

Thanks for trying 🙏

[alleje02]

To be clear, my comment was directed toward the developers / maintainers of WSL. I just don't understand how an issue could be reported and not solved in 5 years. You could rewrite the entire thing from scratch twice in that timeframe.

I too have continually revisited v2 and resorted to reverting to v1 for various reasons.

[mslinn]

The OP did not mention Docker. This issue is not restricted to Docker, and is not dependent on Docker.

[moracabanas]

The OP did not mention Docker. This issue is not restricted to Docker, and is not dependent on Docker.

You are right, Docker and VSCode makes the issue worse taking large amounts of ram.

Docker had issues itself taking more than 3Gb just for running 0 containers.

VSCode WSL extension adds an extra layer of memory leak as reported in other thread, where a faulty AMD driver from x version takes pieces of RAM over the time on top of this thread issue.

[runthis]

The longer WSL runs and I do things in it, even after configuring WSL to only use a maximum amount of memory, it will use more and overage usage will become "hidden" in task manager until the usage is hitting 90+ percent of 64GB of memory with task manager stating WSL is only using around the max I set. People online will say "Unused ram is wasted ram" but as this reaches over 90 percent used ram, many other processes begin slowing down -- this ram can't be freed for some odd reason. Even shutting down WSL will not free the majority of ram this process secretly consumes.

[Gabrielcarvfer]

People online will say "Unused ram is wasted ram" but as this reaches over 90 percent used ram, many other processes begin slowing down -- this ram can't be freed for some odd reason.

While is true that unused RAM is wasted RAM, not the case when you have two OSes competing for it.
If you want to free it, people in the comments above said

sync; echo 3 > /proc/sys/vm/drop_caches

And it actually works, you will see the memory being reclaimed. I guess its a trick to make Plan9 share not suck, which ended up not flushing any cache.

[moracabanas]

People online will say "Unused ram is wasted ram" but as this reaches over 90 percent used ram, many other processes begin slowing down -- this ram can't be freed for some odd reason.

While is true that unused RAM is wasted RAM, not the case when you have two OSes competing for it.
If you want to free it, people in the comments above said

sync; echo 3 > /proc/sys/vm/drop_caches

And it actually works, you will see the memory being reclaimed. I guess its a trick to make Plan9 share not suck, which ended up not flushing any cache.

Any chance to run this with cron so every X seconds the unused memory is released?
Is it harmful?
I can test it building huge docker images which ingest every single MB of ram until swap hit and permanet peeformance drop for that session

[validatedev]

People online will say "Unused ram is wasted ram" but as this reaches over 90 percent used ram, many other processes begin slowing down -- this ram can't be freed for some odd reason.

While is true that unused RAM is wasted RAM, not the case when you have two OSes competing for it.
If you want to free it, people in the comments above said

sync; echo 3 > /proc/sys/vm/drop_caches

And it actually works, you will see the memory being reclaimed. I guess its a trick to make Plan9 share not suck, which ended up not flushing any cache.

Any chance to run this with cron so every X seconds the unused memory is released?
Is it harmful?
I can test it building huge docker images which ingest every single MB of ram until swap hit and permanet peeformance drop for that session

Let me introduce my script which does exactly the same thing you mentioned, based on CPU load so you won't lose required cached data: https://github.com/validatedev/drop-cache-if-idle

[moracabanas]

People online will say "Unused ram is wasted ram" but as this reaches over 90 percent used ram, many other processes begin slowing down -- this ram can't be freed for some odd reason.

While is true that unused RAM is wasted RAM, not the case when you have two OSes competing for it.
If you want to free it, people in the comments above said

sync; echo 3 > /proc/sys/vm/drop_caches

And it actually works, you will see the memory being reclaimed. I guess its a trick to make Plan9 share not suck, which ended up not flushing any cache.

Any chance to run this with cron so every X seconds the unused memory is released?
Is it harmful?
I can test it building huge docker images which ingest every single MB of ram until swap hit and permanet peeformance drop for that session

Let me introduce my script which does exactly the same thing you mentioned, based on CPU load so you won't lose required cached data: https://github.com/validatedev/drop-cache-if-idle

You can't imagine how much appreciated hommie.
Ty!

[Gabrielcarvfer]

Any chance to run this with cron so every X seconds the unused memory is released? Is it harmful? I can test it building huge docker images which ingest every single MB of ram until swap hit and permanet peeformance drop for that session

Yes, its in one of the hidden comments above.

I had the same problem with docker. I just don't have enough memory, but instead of releasing the cache, the WSL VM simply tries to get more RAM instead. and ends up swapping as a result. Not sure if validatedev's solution will work, since you are supposed to be busy while building these images.

You get a performance hit for dropping the caches, but at least things continue working and it won't kill the SSD, which is what I'm concerned. Lucky me I can totally use WSLv1 for most of my stuff, and it just works. Until you get a bsod that is.