Windows .exe execution is very slow when .exe is in WSL filesystem

[kallja]

Version

Microsoft Windows [Version 10.0.22621.521]

WSL Version

• WSL 2
• WSL 1

Kernel Version

5.15.57.1

Distro Version

Ubuntu 20.04

Other Software

No response

Repro Steps

# Copy a Windows executable file to WSL file system
cp /mnt/c/WINDOWS/system32/whoami.exe /tmp
# Execute copied Windows .exe from WSL's filesystem
/tmp/whoami.exe

Expected Behavior

Windows executables run quickly in WSL.

Actual Behavior

Executables from /mnt/c run quickly. Same executables copied into WSL's Linux's filesystem run horribly slow.

WSL hangs for a long time when starting execution of a Windows .exe file. During the hang it doesn't respond to interrupts such as ctrl + c.

$ /mnt/c/WINDOWS/system32/whoami.exe exec time 0s
$ /tmp/whoami.exe exec time 90 s

Diagnostic Logs

WslLogs-2022-10-04_14-00-34.zip

[kallja]

I'm encountering the described behavior with all .exe files I've tried so far including 3rd party apps and stock Windows programs like in the provided example.

[NotTheDr01ds]

Based on the kernel version, I'm assuming you are using one of the pre-release Systemd WSL versions? Is this a new problem that you are experiencing only under that release or did the problem occur on "stock" Windows 11 WSL?

[lucidfrontier45]

I encountered the same problem. As @NotTheDr01ds said, I removed wsl preview (now the kernel version is back to 5.10.102.1-microsoft-standard-WSL2 ) and the problem was fixed.

[NotTheDr01ds]

@lucidfrontier45 In the hopes of narrowing this down a bit, you might want to try:

• If you were using an earlier pre-release, try the latest one (currently 0.68.4) and see if the problem persists.

• If the problem persists there, then try 0.66.2 since that was the last release before the Systemd changes. I'm expecting you won't have the problem in that release.

I'm not able to reproduce this myself using (currently) 0.68.2 or 0.68.4 on two different systems, so there's likely something system-dependent in the issue as well, it seems to me.

[lucidfrontier45]

@NotTheDr01ds Could you share me any document for installing specific version of WSL? I'll happy to test those version.

[NotTheDr01ds]

No problem:

• wsl --shutdown

• On the releases page here, expand the "Assets" for the release you want to try

• Download the .msixbundle

• Open an Admin PowerShell and run:

  cd <download_folder>
  Add-AppxPackage <package_filename>
  

  Sometimes you need to reboot before it will release the files from the old version, but that shouldn't be a problem the first time since you are using the "stock" WSL installation at the moment.

[lucidfrontier45]

@NotTheDr01ds thank you so much. Can you also tell me how to revert to the stock WSL?

[lucidfrontier45]

Here is a simple benchmark program of vector dot product.
https://gist.github.com/lucidfrontier45/c83a58ff423f26dc0feb9ce84ea8f836

I compiled it in Ubuntu 22.04 on WSL2 with mingw.

/usr/bin/x86_64-w64-mingw32-g++ --std=c++17 -O3 --static main.cpp

1 Running the result a.exe inside WSL

$ ./a.exe 
s = 2.49998e+07
elapsed = 2.179[sec]

2 Running it from Windows

PS D:\share> .\a.exe
s = 2.49998e+07
elapsed = 1.631[sec]

3 Running Linux binary built by g++ --std=c++17 -O3 --static main.cpp

$ ./a.out                              
s = 2.49998e+07
elapsed = 1.351[sec]

The above result is obtained from preview version WSL. (Installed by wsl --update in Windows Powershell)

Windows: 22H2
CPU: Ryzen 7 5800
RAM: 16GB
WSL: 0.66.2.0
Kernel: 5.15.57.1
WSL Ubuntu: 22.04

With stock WSL, 1 and 2 result the same duration about 1.6 sec.

[lucidfrontier45]

another benchmark, just memory allocation and free.
https://gist.github.com/lucidfrontier45/c83a58ff423f26dc0feb9ce84ea8f836#file-main2-cpp

1 a.exe inside preview WSL

$ /mnt/d/share/a.exe
elapsed = 11.487[sec]

2 a.exe in Windows

PS D:\share> .\a.exe
elapsed = 5.282[sec]

3 a.exe inside stock WSL

$ /mnt/d/share/a.exe
elapsed = 5.585[sec]

4 native Linux binary inside WSL

$ ./a.out
elapsed = 0.612[sec]

Compared it with the previous benchmark that runs vector dot product, I think the main cause of performance degradation is memory allocation.

[lucidfrontier45]

Whether the .exe is in WSL filesystem or not is irrelevant. Whether we call it from Windows or WSL matters. Perhaps I should submit a separate issue.

[NotTheDr01ds]

@lucidfrontier45 Looks like you already figured this out, but (for anyone else), you can uninstall the Preview just like you do with any Store application. After uninstalling (typically no reboot required), you'll be back to the built-in WSL release.

So, for example:

• wsl --shutdown
• Hit the 🪟 (Windows) key to bring up the Start menu
• Start typing "Subsystem"
• Right-click on the "Windows Subsystem for Linux Preview" result in the Start menu
• Select "Uninstall"

Of course, you can also do the same through the "Add or Remove Programs" settings.

[lucidfrontier45]

@NotTheDr01ds

Thank you for sharing how to install/uninstall.

I tested the benchmark programs which I posted above. In both 0.66.2 and 0.68.4 the performance degraded. Only in the stock WSL the .exe called from Windows and one called from WSL ended in the same duration.

[boborjan2]

Hi I am experiencing the same on:
C:\work>wsl --version
WSL version: 1.0.3.0
Kernel version: 5.15.79.1
WSLg version: 1.0.47
MSRDC version: 1.2.3575
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.1105

What version shall we revert to? I believe I am using the stock install..?

[boborjan2]

I made benchmarks using the main2.cpp provided above:
win11 native cmd: elapsed = 6.8[sec]
wsl2 1.03: elapsed = 12.692[sec]
wsl2 0.66.2: elapsed = 12.621[sec]
wsl2 1.1.2: elapsed = 10.262[sec]

Could you please elaborate what your "stock" version is?

[lucidfrontier45]

@boborjan2

Stock WSL is what is bundled in the Windows. If you can use wsl --version command, then it is installed from the Microsoft Store app and thus is not stock WSL.

If you want to revert to stock WSL, you need to go to Windows system settings and remove WSL just like other apps.

Unfortunately, Microsoft is going to remove the stock WSL in the future.
https://www.xda-developers.com/windows-11-dev-channel-25272/

[boborjan2]

I made a test with wsl kernel 5.10.102: using wsl version 1.1.2, kernel version does not make any difference.

[boborjan2]

Executing the .exe encapsulated in cmd.exe /c does the trick for me.

[benburrill]

For me, it's not only that .exes within WSL filesystem are slow, but also even exes outside WSL are slow if launched with a $PWD that is inside WSL. But if you launch them with a PWD of /mnt/c and then use some means (eg powershell's Start-Process -WorkingDirectory) to set the correct working directory from the windows side, then it's nice and fast.

[boborjan2]

Yes I also experienced that. I wonder if encapsulating in cmd/c only makes it faster because of pwd of cmd must be set to non-wsl dir anyhow (it does not support unc paths).

[boborjan2]

I confirm: even without cmd.exe, the execution is fast (i.e comparable to native windows) if the cwd of the .exe is non-wsl AND the .exe itself is located on a non-wsl fs. Otherwise it get can terribly slow.

[maxisam]

For me, it's not only that .exes within WSL filesystem are slow, but also even exes outside WSL are slow if launched with a $PWD that is inside WSL. But if you launch them with a PWD of /mnt/c and then use some means (eg powershell's Start-Process -WorkingDirectory) to set the correct working directory from the windows side, then it's nice and fast.

I observed the same thing too. It is kind of odd, it only happened to certain computers.

It becomes very slow whenever I try to access /mnt/c. But some computers work perfectly fine. I only see this happened to Windows 11.

And I can use wsl --version

WSL version: 1.0.3.0
Kernel version: 5.15.79.1
WSLg version: 1.0.47
MSRDC version: 1.2.3575
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.1265

This one works fine. The other one with 10.0.22621.1194 doesn't

actually I have another one with

WSL version: 1.0.3.0
Kernel version: 5.15.79.1
WSLg version: 1.0.47
MSRDC version: 1.2.3575
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.1194

and it works fine. So i am not sure what is the root cause.

[benburrill]

It becomes very slow whenever I try to access /mnt/c. But some computers work perfectly fine. I only see this happened to Windows 11.

I am on Windows 10. I think I occasionally observed even just ls being super slow to list files in /mnt/c, but it's been a while since I've seen that. Launching exes is consistently slow though.

It always seems to be just over 10 seconds for me which makes me wonder if there could be some kind of timeout going on.

[maxisam]

It becomes very slow whenever I try to access /mnt/c. But some computers work perfectly fine. I only see this happened to Windows 11.

I am on Windows 10. I think I occasionally observed even just ls being super slow to list files in /mnt/c, but it's been a while since I've seen that. Launching exes is consistently slow though.

It always seems to be just over 10 seconds for me which makes me wonder if there could be some kind of timeout going on.

What is your Windows version? It seems working better with Windows version: 10.0.22621.1265

[benburrill]

What is your Windows version? It seems working better with Windows version: 10.0.22621.1265

Mine is 10.0.19045.2604

[maxisam]

@benburrill so my colleague has the exactly same issue. Here is his path

Windows 10, update doesn't work.

upgrade to windows 11, with Windows version: 10.0.22621.1265 still doesn't work

add a new wsl distro with ubuntu 22 and everything works smoothly.

[0anton]

I have this too: starting Windows executables from WSL2 filesystems first hangs the program for ~90 seconds, then it runs.
Looks like there is a wait for some timeout to pass...

Executable started from Windows share takes nothing to start:

$ time /mnt/c/Windows/system32/whoami.exe
global\xxx

real    0m0.089s
user    0m0.003s
sys     0m0.000s

Same executable started from WSL2 filesystems takes 1m45sec to run (sic!)

$ cp /mnt/c/Windows/system32/whoami.exe .
$ time ./whoami.exe
global\xxxxxx

real    1m45.172s
user    0m0.003s
sys     0m0.000s

$ uname -a
Linux w-B0ZXM03 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[benburrill]

My issue might have been different, but since upgrading to WSL 1.1.3.0, my seemingly related problem seems to have been resolved. whoami.exe in the Windows filesystem is now fast. When copied to the Linux filesystem, while it's noticeably slower, it only takes ~0.2 seconds for me.

[keai4le]

Redirection to cat for every command help for me, but seems MS don't want to revolve this issue.
e.g.
wsl.exe --update | cat

[0anton]

It does not seems to be a pure kernel problem. I've got an instance with Linux kernel 5.15.90.1, which does not hang starting windows executables from WSL2 filesystem:

$ cp /mnt/c/Windows/system32/whoami.exe .
$ time ./whoami.exe
xxx\yyy

real    0m0.054s
user    0m0.000s
sys     0m0.003s

$ uname -a
Linux xxx 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[mblais]

Fixed for me by updating WSL to latest kernel: 5.10.102.1-microsoft-standard-WSL2

WSL2 Linux kernel updates: https://www.catalog.update.microsoft.com/Search.aspx?q=wsl

[0anton]

Thanks for sharing your findings!
I've tried to install 5.10.102.2 version following your link. Starting wsl_update_x64.msi throws the error:

Starting This update only applied to machines with the Windows Subsystem for Linux.

Updating with wsl.exe --update brings kernel 5.15.90.1, which is still affected by the issue:

$ time ./whoami.exe
xxx\yyyg

real    1m20.500s
user    0m0.004s
sys     0m0.000s

[keai4le]

Thanks for sharing your findings! I've tried to install 5.10.102.2 version following your link. Starting wsl_update_x64.msi throws the error:

Starting This update only applied to machines with the Windows Subsystem for Linux.

Updating with wsl.exe --update brings kernel 5.15.90.1, which is still affected by the issue:

$ time ./whoami.exe
xxx\yyyg

real    1m20.500s
user    0m0.004s
sys     0m0.000s

Same issue for me with kernel 5.15.90.1

❯ time whoami.exe
________________________________________________________
Executed in   18.13 secs      fish           external
   usr time    2.62 millis   58.00 micros    2.57 millis
   sys time    0.23 millis  231.00 micros    0.00 millis

[thewingit]

Hi,
This is happening for me regardless of which WSL kernel I am using.

It happens to me if I use the bundled WSL that comes with Windows when you turn the feature on (The wsl.exe is located in C:\Windows\System32), but also if I use the WSL that you can install from the Microsoft store (App name: "Windows Subsystem for Linux"). I tried to run both "Debian" and "Ubuntu" distributions, and it happens on both.

$ time /mnt/c/WINDOWS/system32/whoami.exe
xx\yyyy

real    0m0.363s
user    0m0.002s
sys     0m0.000s

$ time /tmp/whoami.exe
xx\yyy

real    2m0.210s
user    0m0.001s
sys     0m0.000s

Also, this is happening since some time on the old WSL1, even if I did not do any WSL changes (before the exes were running fine). Then I tried to install "Windows Subsystem for Linux" from Microsoft store, install Ubuntu distribution, but the very same issue is happening.

My current WSL version:

wsl --version
WSL version: 1.2.5.0
Kernel version: 5.15.90.1
WSLg version: 1.0.51
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.3208

The "bundled" wsl.exe that is in the Windows System32 folder has a File version: 10.0.10941.2311.

Interestingly, when running WSL, the "new" WSL still runs as a child process of the "bundled" WSL (but I guess that's expected right?):

[microsoft-github-policy-service]

This issue has been automatically closed since it has not had any activity for the past year. If you're still experiencing this issue please re-file this as a new issue or feature request.

Thank you!

[GentleGhostCoder]

I think it's still an issue. Is there an update?