Discussion of Known issues with Valet on WSL2

[tnaffh]

Does this support WSL 2 latest release? if it does, would it work the same way?

[valeryan]

I don't have WSL 2 setup anywhere to test with currently. Confirmation from users that have WSL 2 installed would be helpful.

[victorlap]

I have WSL 2 installed and everything seems to work perfectly. Apart maybe from the fact that I can't get valet secure to work. Although I doubt this has something to do with WSL 2.

[maarten-v]

Nice! What error did you get?
And you had to install the Windows Insider to get WSL2 right?

[victorlap]

Yes, you have to be an insider to get the Windows 10 20H1 build (which includes WSL 2)

I don't get an error, but Firefox complains that the certificate is not secure (although this might have to do with the fact that I'm using Firefox, I have not tested this in another browser atm).

[valeryan]

Yeah, I should add a note somewhere about Firefox. Firefox uses it's own trusted certificate store. The script to install the wsl CA cert only adds it to windows cert store which only covers chrome and ie. You can add the cert in C/tools/wsl/certs to Firefox manually and it will then work for valet secure.

[victorlap]

Whoops. Turns out I was not using WSL2, sorry guys! I thought that by running in the Fast track I would automatically have the latest WSL as well, but I needed to activate it manually.

After doing so I discovered that WSL2 and Acrylic both want to use port 53 and therefore cannot be ran together atm. See also microsoft/WSL#4364 for more information.

Another things that you would need to change is the Acrylic settings, as localhost is not forwarded to the linux vm anymore (although they are working on supporting this https://docs.microsoft.com/en-us/windows/wsl/wsl2-ux-changes#accessing-network-applications).

[valeryan]

When wsl 2 comes out I may be able to use dnsmasq as normal. It may also render this project unneeded as well. The wsl for Linux may be all that is needed when it is released.

[martinwolf]

I hope it's okay to chime in. Has Microsoft said anything about a release date of WSL2?

[valeryan]

@martinwolf I don't think it has been announced when WSL 2 will be publicly available. It's currently available as part of the Windows Insider program if you sign up for the fast ring.

When WSL2 does official release I will test a few things and then may deprecate this project. There are 3 main changes that this package provides that make it work with WSL

1. Changes to configuration to make WSL use ip:port for accessing fpm instead of socks
2. Changes to how startup and services are handled to be compatible with WSLs lightweight handling of run time services.
3. Disabling the DNSMasq stuff that is incompatible with WSL

WSL 2 has a full Linux kernel and a more robust service handling along with possible support for socks and better networking. It may be able to run everything in the valet Linux package natively.

The reason that Acrylic and WSL2 conflict is that dnsmasq in WSL2 grabs port 53 and in theory should be able to work as a DNS proxy for windows like Acrylic is doing, thus removing any need for Acrylic as well.

All that would be pretty nice to have but I don't feel like setting up an environment to test WSL2 right now because I would have to move to the Insider fast ring on a windows pc somewhere. I will wait for it to release and then figure out if this project is still needed.

[valeryan]

As far as I can tell WSL2 does not cause any issue with valet-wsl. I still can't get dnsmasq to work for windows though so that is disappointing.

[valeryan]

I changed the title of this to make it a de facto place to discuss any issues that are found with WSL2.

[valeryan]

If microsoft/WSL#4592 gets resolved I will hopefully be able to switch this over to using dnsmasq at some point. Currently, the only issue I know of with running valet-wsl under WSL2 is the conflict between Acrylic and dnsmasq both trying to bind port 53.

[tnaffh]

I changed the title of this to make it a de facto place to discuss any issues that are found with WSL2.

Great

[irakan]

WSL 2 has a full Linux kernel and a more robust service handling along with possible support for socks and better networking. It may be able to run everything in the valet Linux package natively.
I tried to install valet Linux package natively using wsl2, but it seems it doesnt work.. and I dont know why!?
https://stackoverflow.com/questions/60518957/valet-status-is-showing-is-stopped

I reinstalled using your fork, and now its working great.

[valeryan]

valet-wsl has a patch that helps it better determine the services state and valet-linux is probably not going to work out of the box. However, if the changes become minor tweaks instead of large changes then they can hopefully be upstreamed into valet-linux.

I am not running fast ring on my windows system so I don't have ways of testing WSL 2 currently. When it is released I will evaluate how and if this fork is still needed.

Thanks for the info.

[thomasaull]

@irakan could you outline the steps you have taken to mak valet-wsl work with WSL2? I can't get it running

[valeryan]

@thomasaull the only thing that is known not to work right now is the port conflict between dnsmasq and acrylic proxy in windows. You should be able to follow the install guide. https://github.com/valeryan/valet-wsl/wiki/Installation-Guide

[nickescobedo]

I followed the guide but still haven't been able to get it working with WSL2.

Domains aren't resolving and I can't figure out how to get it working. I've disabled dnsmasq.

The domains don't appear to route through Acrylic at all.

[valeryan]

Does your domain route to 127.0.0.1 when you ping it from windows cmd line? If acrylic is running and configured correctly your domain will point to your internal up.

I don't official support WSL2 at the moment because it is very much in flux and has some network issues. Some of which may be fixed on the fast ring. See microsoft/WSL#4769 and microsoft/WSL#4636 as examples.

[thomasaull]

I got it running with a workaround. Basically installing Acrylic made the WSL2 don't start anymore (don't know why, didn't investigate).
For routing the hosts from windows to WSL2 I'm using https://github.com/shayne/go-wsl2-host/. After installing, I created the ~/.wsl2hosts config file and put my hosts in. Now wsl2host will add entries to the windows hosts file to point all these hosts to the current wsl2 ip. Works pretty well for me! Ah and one thing, you might want to deactivate the automated generation of the hosts file inside of wsl2, since it ended up overwriting the hosts from wsl2host with old IP adresse. To deactivate:

# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry to /etc/wsl.conf:
# [network]
# generateHosts = false

[irakan]

@thomasaull
I followed the guide in documentation but I didn't install Acrylic, I manually edits my hosts file and add the sites. nothing else.

[thomasaull]

This way you need to update your host file whenever you restart the wsl, don‘t you?

[valeryan]

It's one of the know issues in WSL2. Acrylic Proxy causes issues microsoft/WSL#4364. Also, none of the host file hacks you can find out there are going to allow wildcard routing like Acrylic or dnsmasq can. Manually adding the sites to your host file sucks but wsl2 is binding port 53 and causing Acrylic to crash.

[nickescobedo]

Yea when Acrylic was running and I didn't have WSL2 running and tried to start WSL2 it wouldn't start because something to do with creating the network adapter.

I gave up trying to get it to work unfortunately and just manually add sites to my hosts file until something works out. I appreciate your quick responses! 😄

[irakan]

This way you need to update your host file whenever you restart the wsl, don‘t you?

No, only once but every-time your create a site inside your parked folder, you will need to add it manually to hosts file.

Or you can use this, I have created it to append sites automatically when I do valet park, valet link etc..

---

1 - add this to your .bash_aliases file to intercept valet command

SETTUP_HOSTS_PATH=~/setup-hosts.sh

valet() {
    if [[ $@ == "park" || $@ == "link" ]]; then
        command valet "$@" && bash $SETTUP_HOSTS_PATH
    else
        command valet "$@"
    fi
}

2 - run source .bashrc to apply the change

3 - create setup-hosts.sh file in the same place as SETTUP_HOSTS_PATH and put:

#!/bin/sh
WINDOWS_HOSTS_PATH=/mnt/c/Windows/System32/drivers/etc/hosts

function add() {
   grep -qxF "127.0.0.1 $1.test" $WINDOWS_HOSTS_PATH || echo "127.0.0.1 $1.test" >> $WINDOWS_HOSTS_PATH
}

valet paths | grep -oP '"(.*?)"' | tr -d '"' | while read -r line ; do
    cd $line

    for f in * ; do
        add $f
    done

done

4 - go to C:\Windows\System32\drivers\etc and right click on hosts => properties => security and give your user write permission.

[thomasaull]

Ah of course, I did setup my host file to point at the WSL IP, but I can just use localhost, since it will also resolved by WSL. That makes a whole lot more sense. And many thanks for your script @irakan

[artistro08]

Sorry for the confusion, but it appears I am confused as well. I've tried so many things that I no longer remember everything. So sorry for planting false hope: if you want a properly behaving Linux, you'll need to install a VM.

The short version of the story:

• There's no loopback connection between WSL and the Windows host
• A setup called systemd-genie further added to the confusion by separating the Windows and Linux hosts into two separate hostnames and communicating through the loopback. I'm not really clear on what ended up happening there, but I ended up with an address of 127.0.1.1 for a host with the same name as my Windows host. It wasn't.
• The not-Windows host was running dnsmasq and so appeared to respond like Acrylic would
• I have no idea where the not-Windows host is... systemd-genie installs some kind of weird inner environment to start up systemd with a PID of 1. I think the loopback connects the genie "bottle" with the WSL host.

Conclusion: WSL has too much weirdness to work like a proper Linux host. Systemd support is absolutely necessary but that opens a huge support can of worms since many programs will break (like valet-linux) without it being clear why. The fact is, Systemd can only work if it starts at boot time. And that means putting it in a VM.

So I was actually able to get this working with systemd-genie. My steps were as follows:

• Install Ubuntu 20.04 WSL
• Install valet-linux
• install systemd-genie from source
• adjust the hosts file for windows (e.g. 127.0.0.1 test.test)
• profit.

I've been running this commercially so no drawbacks that I can see.

[ghost]

Sorry for the confusion, but it appears I am confused as well. I've tried so many things that I no longer remember everything. So sorry for planting false hope: if you want a properly behaving Linux, you'll need to install a VM.
The short version of the story:

• There's no loopback connection between WSL and the Windows host
• A setup called systemd-genie further added to the confusion by separating the Windows and Linux hosts into two separate hostnames and communicating through the loopback. I'm not really clear on what ended up happening there, but I ended up with an address of 127.0.1.1 for a host with the same name as my Windows host. It wasn't.
• The not-Windows host was running dnsmasq and so appeared to respond like Acrylic would
• I have no idea where the not-Windows host is... systemd-genie installs some kind of weird inner environment to start up systemd with a PID of 1. I think the loopback connects the genie "bottle" with the WSL host.

Conclusion: WSL has too much weirdness to work like a proper Linux host. Systemd support is absolutely necessary but that opens a huge support can of worms since many programs will break (like valet-linux) without it being clear why. The fact is, Systemd can only work if it starts at boot time. And that means putting it in a VM.

So I was actually able to get this working with systemd-genie. My steps were as follows:

• Install Ubuntu 20.04 WSL
• Install valet-linux
• install systemd-genie from source
• adjust the hosts file for windows (e.g. 127.0.0.1 test.test)
• profit.

I've been running this commercially so no drawbacks that I can see.

Just to be sure: you still need to add every host to hosts file on Windows, right?

[devcircus]

Correct. I'm not sure what the above is claiming to solve. This already works out-of-the-box if you add each site to your hosts file. The whole issue is about not having to add each site manually. Currently, due to wsl issues, this doesn't work, so we just add every site to hosts.

[insign]

I think maybe we can use DNSCrypt-proxy to redirect using cloaking rules. It worked on linux. I'll try then edit this comment.

[plakhin]

The solution is to bind dnsmasq to different port, smth like 5353. Then use smth like YogaDNS and configure it to resolve *.test using 127.0.0.1:5353 (I guess dnsmasq should be accessible by 127.0.0.1 ip address as other WSL2 services likes nginx, but I don't have WSL2 installed at the moment so can't check). Currently I'm running Ubuntu inside VirtualBox VM for local development and using dnsmasq+YogaDNS to resolve *.test, works great.

The biggest problem with WSL2 is stil... filesystem access performance.

1. If you store your project in windows filesystem and use /mnt/c/... paths you're getting native performance in Windows IDE which is great, but inside WSL2 it performs extremely slow.
2. If you store your project inside WSL2 on its filesystem and access files from windows by \\wsl$\Ubuntu\... you have really great performance inside WSL2, faster than inside VirtualBox VM, but accessing files from Windows IDE becomes paing because it is really slow and doesn't support filesystem notifications, so you have to manually refresh file list each time new file or folder is created and so on.
   I know there is VS Code Remote WSL extension which is actually great, but I'm not using VS Code.

As soon as filesystem access performance issue will be resolved, I'll move to WSL2 + Valet + Docker (for MySQL, Postgre, Redis etc.) for local development without any doubt.

[valeryan]

My system finally upgraded to windows 2004. I will try to look into getting this working but Microsoft has not really given any indications that they find it worthwhile to fix localhost forwarding or using dnsmasq. I am not super optimistic at this point but I think the answer will be something similar to making sure that dnsmasq does not bind port 53 so that acrylic and work or figuring out how to provide some other kind of interoperable DNS solution.

[artistro08]

My system finally upgraded to windows 2004. I will try to look into getting this working but Microsoft has not really given any indications that they find it worthwhile to fix localhost forwarding or using dnsmasq. I am not super optimistic at this point but I think the answer will be something similar to making sure that dnsmasq does not bind port 53 so that acrylic and work or figuring out how to provide some other kind of interoperable DNS solution.

You might look into editing the hosts file manually. I have a script that watches the parked directory and creates an entry whenever there is a new folder in there.

[valeryan]

Host file hacks do not support wildcard domains and will never be capable of providing the coverage of a dns proxy. If your project only uses a simple single domain layer then something like domain.test in your host file is enough but many projects need support for *.domain.test which is not possible with host file entries.

[plakhin]

I've already posted the working solution for wildcard domains.

Another option is just to set up your windows to use 127.0.0.1 as DNS server, but then you need WSL and dnsmasq always running.

[valeryan]

You are using virtual box, wsl has a different network stack and communication between windows and dnsmasq is broken as I have pointed out before: microsoft/WSL#4592

[plakhin]

You should have access to services running inside WSL on 127.0.0.1:port by TCP and UDP. Like you have an access to nginx. The trick is to bind dnsmasq to different port, not 53.
I'm not sure if Acrylic allows to add nameservers with different ports, but YogaDNS does.
At the moment I don't have WSL, but when I did have I was able to access any of it's services, like nginx, mysql, ssh on 127.0.0.1:port via TCP.
And binding dnsmasq to port 53 will not work.

[valeryan]

Should is the important word there. But currently it does not work for dns like it does for other services. Which is the point of the issue I linked.

[plakhin]

Did you try different port, not 53?
Do you access WSL by 127.0.0.1?
Did you uninstall acrylic and/or other software which can impact?
Did you try to disable windows firewall?

[valeryan]

Did you look at at the issue I posted and everything everyone has tried... There is a possible work around near the end but it is unreliable and requires some annoying changes to some deep windows settings. It's not super viable for this project.

[plakhin]

I've looked at microsoft/WSL#4592.
Information there is either quite outdated (there was changes to WSL2 before official release of 2004 update) or irrelevant to proposed solution.

[Chakratos]

Are there some news regarding WSL2 and Valet?
I would love to use it for work but im not sure how to approach it, im already using WSL2 Ubuntu and have valet installed but the whole Acrylic problem is nerve wracking. I will try to change the dnsmasq port so acrylic can still run but i don't think thats all i need in order for it to run well.

[artistro08]

Are there some news regarding WSL2 and Valet?
I would love to use it for work but im not sure how to approach it, im already using WSL2 Ubuntu and have valet installed but the whole Acrylic problem is nerve wracking. I will try to change the dnsmasq port so acrylic can still run but i don't think thats all i need in order for it to run well.

You don't need acrylic at all. Just add the .test domain to C:\Windows\System32\drivers\etc\hosts

example

127.0.0.1 test.test

[valeryan]

That's not very doable when you have hundreds of projects or a need for wildcard domains. The host file is well know to most developers but does not fulfill their needs. I have tried to find a working solution, the problem is that wsl binds port 53 even if you don't enable dnsmasq. Also, windows can't connect to dnsmasq in wsl even with yogadns. The network stack in wsl2 is not as easy to work with as wsl1. I tried configuring acrylic to use a different port like 5353 and then have yogadns handle resolving .test from acrylic. Acrylic does not respond as a dns server would.

[artistro08]

@valeryan you're 100% correct. It's not ideal, but it works for me since I don't have too much going on right now. One thing I did do to get around that was create a bash script that watched my parked valet folder and added the folder name plus .test and the localhost setup. This obviously isn't ideal either since you have to take ownership of the file in Windows. This was the only thing I could think of that made sense for my needs. Heres' the script if anyone wants it

#!/bin/bash
inotifywait -m ~/.sites -e create -e moved_to |
    while read dir action file; do
        echo "The file '$file' appeared in directory '$dir' via '$action'"
        echo -e "\n127.0.0.1 $file.test" >> /mnt/c/Windows/System32/drivers/etc/hosts
    done

Then I added it to my bashrc while directing the output to /dev/null

[decalages]

I came out with a workaround solution that I'd like to share.
Since Acrylic is not working with wsl2, I'm using http://xip.io as a wildcard dns alternative.
Configure valet with:
valet domain 127.0.0.1.xip.io
then access your projet using myproject.127.0.0.1.xip.io

You can also use vcap.me
valet domain vcap.me
then access your projet using myproject.vcap.me

Not perfect, but it does the job until we can use an internal solution !

[plakhin]

As a workaround https://github.com/TechnitiumSoftware/DnsServer/ can be used,
You can install on windows host and create .test (or whatever) zone there, then create * record of A type and IPv4 Address 127.0.0.1

[thatguyerik]

As a workaround https://github.com/TechnitiumSoftware/DnsServer/ can be used,
You can install on windows host and create .test (or whatever) zone there, then create * record of A type and IPv4 Address 127.0.0.1

I tried this method, but it still creates the same conflict on port 53 that was causing WSL2 to fail on startup if Acrylic is installed. Also, the installation set off multiple alerts from my antivirus software, which makes me nervous about using it.

[plakhin]

I was using it with WSL2 without any problems. I wasn't using dnsmasq inside WSL2 and my antivirus is default windows defender.
Later I switched back to manual host addition to hosts files cause found this solution quite overhead.

[thatguyerik]

At this point it's looking like it's easier to just add your dev domains (including potentially hundreds of subdomains each, depending on the project) to the Windows hosts file. It's frustrating because on a Mac, everything just works. Now I've got this beast of a PC that could eat the Mac for breakfast that's doing nothing more than Slack, Chrome, and Spotify.

[plakhin]

YogaDNS authors promised to add a rule to resolve any wildcard domain to specific IP (or just localhost) in the next release.
But hosts is not the main problem.
The main problem is filesystem access. If you don't use VS Code with WSL extension and access files inside WSL via network share - it is too slow and the main problem is the lack of fs notifications, so when you create new file from terminal it takes some time to appear in your IDE file list, sometimes even hitting refresh doesn't help. This is the most annoying part for me. Also, working with large git repos taking ages...

[thatguyerik]

If you're not using the WSL2 home directory for your user to house your project repos, it will be incredibly slow. There's a tool called Mutagen that may help with that somewhat. I've only used it on a Mac with Docker for Desktop, but I've heard it works well on Windows also.

That's something I'm definitely looking into because I've run in to the same problem you have with both Visual Studio Code and pretty much every JetBrains IDE in existence. It should be able to sync files pretty fast between your local Windows file system and your home directory in WSL via the \wsl$ network path (mine would be something like \wsl$\Ubuntu-18.04\home\erik\code for example).

[seklyza]

I decided not to use PHPStorm eventually but you can install it on WSL2 (using VcxSrv on Windows) and it works. It's not perfect but it works. Regarding the DNS issue, I decided to not put much time into it and just valet domain localhost.

[artistro08]

Hate to bring up an old thread, but I wrote a tutorial on how to setup valet linux in wsl2 with all the new features

https://gist.github.com/artistro08/5ab9ad7e43c000bb15c23a4f779e5449

[jason-guru]

Hate to bring up an old thread, but I wrote a tutorial on how to setup valet linux in wsl2 with all the new features

https://gist.github.com/artistro08/5ab9ad7e43c000bb15c23a4f779e5449

This works thanks. 🥇