Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wsl 2 ubuntu 18.04 unable to connect to IP resources with Cisco Annyconnect active #4277

Open
eclay11 opened this issue Jul 6, 2019 · 250 comments
Labels
network wsl2 Issue/feature applies to WSL 2

Comments

@eclay11
Copy link

eclay11 commented Jul 6, 2019

Please fill out the below information:

  • Your Windows build number: (Type ver at a Windows Command Prompt)
    Microsoft Windows [Version 10.0.18932.1000]

  • What you're doing and what's happening: (Copy&paste the full set of specific command-line steps necessary to reproduce the behavior, and their output. Include screen shots if that helps demonstrate the problem.)
    I've installed ubuntu 18.04 and converted it to wsl v2 using the following command in windows powershell running as administrator.
    PS C:\WINDOWS\system32> wsl --set-version Ubuntu-18.04 2

When I start the Ubuntuu 18.04 instance I'm able to communicate with resources using IPs and hostnames.

eclay@wh-lpt-eclay:$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=49 time=115 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=49 time=95.4 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=49 time=77.3 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=49 time=92.6 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=49 time=90.6 ms
64 bytes from 8.8.8.8: icmp_seq=6 ttl=49 time=78.9 ms
64 bytes from 8.8.8.8: icmp_seq=7 ttl=49 time=84.1 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=49 time=90.9 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=49 time=80.7 ms
64 bytes from 8.8.8.8: icmp_seq=10 ttl=49 time=80.0 ms
^C
--- 8.8.8.8 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9081ms
rtt min/avg/max/mdev = 77.316/88.602/115.084/10.734 ms
eclay@wh-lpt-eclay:~$ trace
tracepath traceroute6 traceroute6.iputils tracerpt.exe
eclay@wh-lpt-eclay:$ tracepath 8.8.8.8
1?: [LOCALHOST] pmtu 1500
1: wh-lpt-eclay.mshome.net 0.497ms
1: wh-lpt-eclay.mshome.net 0.379ms
2: 10.179.10.1 2.890ms
3: 192.168.1.1 8.474ms
4: 192.168.1.1 10.395ms pmtu 1430
4: 172.26.96.161 71.223ms
5: 172.18.84.36 73.898ms asymm 6
6: 12.249.2.25 73.083ms asymm 7
7: 12.122.146.97 96.016ms asymm 14
8: 12.122.111.33 95.857ms asymm 13
9: sffca22crs.ip.att.net 95.209ms asymm 12
10: 12.122.163.61 90.288ms asymm 11
11: 12.255.10.236 103.555ms asymm 15
12: no reply
13: no reply
14: no reply
15: no reply
16: no reply
17: no reply
18: no reply
19: no reply
20: no reply
21: no reply
22: no reply
23: no reply
24: no reply
25: no reply
26: no reply
27: no reply
28: no reply
29: no reply
30: no reply
Too many hops: pmtu 1430 Resume: pmtu 1430

Once I connect to my work VPN using cisco annyconnect client v 4.7.00136 I'm not longer able to connect to IP resources.

eclay@wh-lpt-eclay:$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 7277ms

eclay@wh-lpt-eclay:$ tracepath 8.8.8.8
1?: [LOCALHOST] pmtu 1430
1: ??? 0.362ms
1: ??? 0.364ms
2: ??? 0.392ms pmtu 1200
2: no reply
3: no reply
4: no reply
5: no reply
6: no reply
7: no reply
8: no reply
9: no reply
10: no reply
11: no reply
12: no reply
13: no reply
14: no reply
15: no reply
16: no reply
17: no reply
18: no reply
19: no reply
20: no reply
21: no reply
22: no reply
23: no reply
24: no reply
25: no reply
26: no reply
27: no reply
28: no reply
29: no reply
30: no reply
Too many hops: pmtu 1200
Resume: pmtu 1200

  • What's wrong / what should be happening instead:
    I should be able to ping 8.8.8.8 or other outside IPs once the annyconnect vpn client is running.
@ghost
Copy link

ghost commented Jul 7, 2019

Not 100% sure, but it can be that annyconnecy blocks any dns server except provided by connection. I was observed something similar and was unable to disable this behaviour.

@eclay11
Copy link
Author

eclay11 commented Jul 7, 2019

I've tried the method of creating a manual named.conf and it didn't fix anything. I can ping the 1st dns server from windows but not from ubuntu.
Windows DNS servers after vpn connection established.

DNS Servers . . . . . . . . . . . : 172.24.0.25
8.8.8.8
C:\Users\eclay>ping 172.24.0.25

Pinging 172.24.0.25 with 32 bytes of data:
Reply from 172.24.0.25: bytes=32 time=131ms TTL=126
Reply from 172.24.0.25: bytes=32 time=103ms TTL=126
Reply from 172.24.0.25: bytes=32 time=109ms TTL=126

Ubuntu

$ ping 172.24.0.25
PING 172.24.0.25 (172.24.0.25) 56(84) bytes of data.

^C
--- 172.24.0.25 ping statistics ---
190 packets transmitted, 0 received, 100% packet loss, time 196530ms

It seems to be more of a routing problem than a name resolution. For instance I can ping www.microsoft.com from a cmd prompt but not from ubunutu.

windows

ping www.microsoft.com

Pinging e13678.dspb.akamaiedge.net [23.202.232.226] with 32 bytes of data:
Reply from 23.202.232.226: bytes=32 time=103ms TTL=51
Reply from 23.202.232.226: bytes=32 time=105ms TTL=51
Reply from 23.202.232.226: bytes=32 time=121ms TTL=51
Reply from 23.202.232.226: bytes=32 time=109ms TTL=51

Ping statistics for 23.202.232.226:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 103ms, Maximum = 121ms, Average = 109ms

Ubuntu

$ ping 23.202.232.226
PING 23.202.232.226 (23.202.232.226) 56(84) bytes of data.
^C
--- 23.202.232.226 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4176ms

@WsK
Copy link

WsK commented Aug 16, 2019

Having exactly same problem...Errrrr

@kobenauf
Copy link

kobenauf commented Aug 16, 2019

I have the same issue, that from within the WSL 2 shell, I'm unable to use AnyConnect running in Windows. However, in WSL I installed openconnect, and I run:
sudo openconnect --no-dtls -u myUsername vpn.myhost.com
Then I can access my company ip addresses from within WSL 2 sessions, but not from Windows, but this lets me at least ssh and run git, so that's an improvement.

Also, this only works if I disconnect the VPN in Windows first, which is annoying.

@chunkaichang
Copy link

Same issue here. I tried to add search list in /etc/resolv.conf followed the guide here but still failed.

@craigloewen-msft craigloewen-msft added network wsl2 Issue/feature applies to WSL 2 labels Aug 27, 2019
@craigloewen-msft
Copy link
Member

If you're still experiencing this issue could you please take some networking logs? It will help us diagnose what's going on.

You can find instructions on how to do so here!

@mcurole
Copy link

mcurole commented Sep 5, 2019

I'm definitely experiencing this issue and I'm running 18970. I can capture logs if needed. The problem is easy to reproduce.

@mcurole
Copy link

mcurole commented Sep 5, 2019

Well, cancel that. I don't have the Feedback Hub installed and I have another issue where I can't download app from the Microsoft Store because of some weird Microsoft Account issue that I'm working with support on.

@MichaelTong
Copy link

Also, if WSL 2 starts first, Cisco AnyConnect will fail to connect.

@MichaelTong
Copy link

I filed a feedback in FH. https://aka.ms/AA60j0u

@cmeiklejohn
Copy link

Same issue, I filed feedback using Feedback Hub.

@paulguerrero
Copy link

paulguerrero commented Oct 14, 2019

I also have the same issue with AnyConnect, also gave feedback in the Hub (although the option to post a recording was grayed out for me).

@neileadobe
Copy link

neileadobe commented Oct 31, 2019

I'm also having this problem and opened an issue for it in Feedback Hub with logs: https://aka.ms/AA6fthe

@uc-cjdavis
Copy link

I have the same issue, that from within the WSL 2 shell, I'm unable to use AnyConnect running in Windows. However, in WSL I installed openconnect, and I run:
sudo openconnect --no-dtls -u myUsername vpn.myhost.com
Then I can access my company ip addresses from within WSL 2 sessions, but not from Windows, but this lets me at least ssh and run git, so that's an improvement.

Also, this only works if I disconnect the VPN in Windows first, which is annoying.

I tried this route, but our AnyConnect uses 2FA which makes using openconnect impossible

@raisin-loaf
Copy link

I'm in the same boat. WSL 2 is a godsend while I'm in the office; it's so much faster and better integrated than using a VirtualBox (or similar) VM. Unfortunately I still have to use said VM when connecting remotely via VPN because, as others here have described, all network connectivity from WSL 2 stops the moment the VPN is up.

I use Cisco AnyConnect too although I imagine the problem is common to most VPN clients.

Like @Haselton I'm unable to use OpenConnect as the company I work for enforces 2FA.

@craigloewen-msft - I'd dearly love to send you some logs but our workstation diagnostic data settings are locked down by Group Policy. If there's anything else I can do to help please let me know.

WSL 2 is fabulous. 🙂

@craigloewen-msft
Copy link
Member

@raisin-loaf thank you for the offer! I'll let you know if you can help out in any other way. We are investigating issues related to the VPN.

@AdonisLL
Copy link

AdonisLL commented Dec 4, 2019

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

@raisin-loaf
Copy link

I was able to resolve this by installing anyconnect pwa from the windows store

It works! Thanks so much for posting this.

@dalgibbard
Copy link

dalgibbard commented Dec 10, 2019

I was having the same issue; i found that Anyconnect was setup to do full-tunnel, and therefore a route existed to throw all traffic through the VPN connection (likely including packets destined for the WSL2 vm).

WSL2 routes are configured with a metric of 5256, and anyconnect routes have a metric of 2, and therefore take priority. I amended the routes for anyconnect (where metric == 2) to be 5257 instead, which seems to largely work. I used "NetRouteView" run as Admin to do the changes because I was too lazy to find the powershell equivilent commands :)

This restored connectivity, but I found DNS to still be broken with the WSL default resolvconf (nameserver 172.xx.xx.x) so i dropped in nameserver 1.1.1.1 instead which seems OK. Will likely replace this with the work DNS for work resources to resolve nicely :)
and now full connectivity is working; no additional changes needed; but this likely will need applying each time I start anyconnect.

@chrisjones-brack3t
Copy link

Any movement on a fix for this? In my attempt to move away from macOS to Windows for dev, this has effectively stopped me completely as the company I work for uses AnyConnect.

@craigloewen-msft
Copy link
Member

Yes we are tracking this and are looking into solutions with the networking team! I'll post any updates here as soon as they become available.

@caal-15
Copy link

caal-15 commented Jan 10, 2020

Thanks a lot, this would be a life saver!

@kzeitz
Copy link

kzeitz commented Jan 18, 2020

For what it's worth I was having all the same issues described here.

I saw AdonisLL post and tried it. I installed the AnyConnect UWP version available on the Windows store. Now, it seems to work perfectly! No jacking around with resolv.conf, or turning off generateResolvCon in wsl.conf.

In fact the VPN seems much faster than the AnyConnect client 4.5 version I was using. Considering all the grief this was causing, this turned out to be a fast and easy fix for this!

@chrisjones-brack3t
Copy link

@kzeitz Company I work for requires 2FA authentication which it seems the Windows store version of AnyConnect doesn’t support.

@kzeitz
Copy link

kzeitz commented Jan 20, 2020 via email

@qwertycody
Copy link

I'm having this issue in December 2022 on an up to date version of Windows 10 with WSL 2 and the latest Ubuntu from Microsoft Store.

I agree with the others here that setting the metric high is not the answer - additionally these solutions do not consider developers that do not have administrative rights on their machine due to corporate policy.

Part of the issue I'm experiencing is the DNS issue where I have to manually set the DNS IP address in the /etc/resolv.conf

However, even after doing this sporadically I'll be unable to even ping an IP address directly leading me to believe that there still are some issues to be resolved with how the network adapters are handled by Windows.

Just wanted to share my experience.

P.S. - I wrote up an automatic powershell script to handle the DNS issue - but still have the other issue sporadically I described.

I guess I'll take 50% working over 0% working?

@th0ger
Copy link

th0ger commented Nov 30, 2022

We have activated the BypassVirtualSubnetsOnlyV4 fix of Cisco AnyConnect 4.10, but it did not appear to help. Anyone had success with that fix?

Probably one for Cisco support I guess, but based on this: https://www.cisco.com/c/en/us/td/docs/security/vpn_client/anyconnect/anyconnect410/administration/guide/b-anyconnect-admin-guide-4-10/troubleshoot-anyconnect.html#Cisco_Task_in_List_GUI.dita_3a9a8101-f034-4e9b-b24a-486ee47b5e9f You have to create the custom attribute names, and assign them to a group policy; and then the version must be 4.10.01075 or newer.

Finally the BypassVirtualSubnetsOnlyV4 fix from Cisco solved it for us! No more messing with interface metric workarounds.

@JorisNinja
Copy link

I had the same issue and resolved it with:

Get-NetIPInterface -InterfaceAlias "vEthernet (WSL)" | Set-NetIPInterface -InterfaceMetric 1
Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 6000

after connecting my VPN and then restarting WSL. I also had to downgrade my Cisco AnyConnect to the version found here: https://olemiss.edu/helpdesk/vpn/windows.html

It's work for me. Thank you you saved my time.

This was actually my problem, Cisco VPN had connected status (even when disconnected). By chaging the priority of the Cisco one, the WSL2 variant started working again. Also changed my WiFi (primary interface) to priority of 1.

Thanks and you rock!!

@alanlivio
Copy link

Thank you @prasadrajesh @JorisNinja, your commands worked for me.

@Pit-Storm
Copy link

You shouldn't change the Interface-Metric due to the than different routing. See the following blogpost for explanation: https://janovesk.com/wsl/2022/01/21/wsl2-and-vpn-routing.html
TLDR: If it works, it doesn't mean that it doesn't have side effects. And it's not only solving the thing that you was intended to fix.

The problem of not using the correct DNS-Server is properly explained and the suggested solution should be used from networking point of view.

For the IP-Range problem you have only the following two options:

  1. Changing the routing table (see blogpost above)
  2. Changing the subnet-range that WSL is using

How second could work, is shown in this Microsoft Q&A: https://learn.microsoft.com/en-us/answers/questions/1123820/set-wsl2-subnet

TLDR: Change SubNet of WSL NAT-Router to a different one which does not collide with your Company-VPN subnet. To do so got to regedit and edit the following entries:
Path: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Lxss
Entries: NatGatewayIpAddress and NatNetwork
Values e.g.: 19.16.0.1 and 19.16.0.0/16

Again: Please don't change the InterfaceMetric!

@dalgibbard
Copy link

dalgibbard commented Jun 15, 2023

@Pit-Storm Unless I'm missing something, I think this is irrelevant when dealing with full-tunnel VPN; the route added is for 0.0.0.0/0 via the VPN, and this encompasses the WSL subnet. Deleting this route would obviously result in no traffic traversing the VPN.

Amending the MetricID just ensures that traffic for the WSL interface is prioritised above the VPN adapter, so that traffic can return.

Yes it does add the ability for WSL to route to the local network whilst on VPN when you normally shouldn't be able to (frowned upon in a full tunnel setup), but that's a small price to pay (and arguably convenient lol)

@Domain
Copy link

Domain commented Aug 26, 2023

We have activated the BypassVirtualSubnetsOnlyV4 fix of Cisco AnyConnect 4.10, but it did not appear to help. Anyone had success with that fix?

Probably one for Cisco support I guess, but based on this: https://www.cisco.com/c/en/us/td/docs/security/vpn_client/anyconnect/anyconnect410/administration/guide/b-anyconnect-admin-guide-4-10/troubleshoot-anyconnect.html#Cisco_Task_in_List_GUI.dita_3a9a8101-f034-4e9b-b24a-486ee47b5e9f You have to create the custom attribute names, and assign them to a group policy; and then the version must be 4.10.01075 or newer.

Finally the BypassVirtualSubnetsOnlyV4 fix from Cisco solved it for us! No more messing with interface metric workarounds.

I don't have the ASDM, how can I change the setting directly?

@craigloewen-msft
Copy link
Member

Hi folks, we have put out a new update that aims to address networking issues in WSL. In your .wslconfig file you can set experimental.networkingMode=mirrored, as well as some other key settings that should improve your network compatibility! Please try them out and let us know what you think.

More info on this release and the changes can be found here in the blog post.

@balmeida-nokia
Copy link

@craigloewen-msft What about users with older windows versions, such as 21H2?

@NiklasBr
Copy link

NiklasBr commented Oct 3, 2023

@craigloewen-msft I updated to WSL 2.0.3 on 22H2 and added below section to .wslconfig, rebooted the PC, but launching Cisco Anyconnect still blocks it:

[experimental]
networkingMode = "mirrored"
dnsTunneling = true

@sortelyn
Copy link

sortelyn commented Oct 7, 2023

If possible utilize the configuraiton of AnyConnect, then it works without any hacks. If you have access though:

image

@craigloewen-msft
Copy link
Member

craigloewen-msft commented Oct 10, 2023

@craigloewen-msft I updated to WSL 2.0.3 on 22H2 and added below section to .wslconfig, rebooted the PC, but launching Cisco Anyconnect still blocks it:

[experimental]
networkingMode = "mirrored"
dnsTunneling = true

Please change your .wslconfig to this content instead. Does that resolve you? (You need to remove the quotes!)

[experimental]
networkingMode=mirrored
dnsTunneling=true

@NiklasBr
Copy link

@craigloewen-msft I can't tell what is different between those two code blocks.

@craigloewen-msft
Copy link
Member

@NiklasBr my mistake I failed on copy and pasting. I've adjusted it!

@NiklasBr
Copy link

This appeared to work 🙌 but it broke Docker Desktop.

@craigloewen-msft
Copy link
Member

@NiklasBr glad to hear it! That is likely a known issue being tracked here: #10494

@craigloewen-msft
Copy link
Member

These new networking features are now available on the latest version of Win11 22H2!

Please make sure you're on the latest build to get these features, you can do that by clicking "Check for Updates" in Windows settings. You can check you have the right build by either ensuring you have KB5031354 installed, or run cmd.exe /c ver and ensure that your build number is 22621.2428 or higher (Including the minor build number which is after the . as this was a backport!)

@kjeremy
Copy link

kjeremy commented Mar 21, 2024

Is there any way to enable these options on Windows 10 22H2?

@testuser7
Copy link

@craigloewen-msft I'm trying to get access to VPN network from WSL2. Is it possible?
After add this .wlsconfig:

[wsl2]
networkingMode=mirrored
firewall=false
dnsTunneling=true

I have no communication. This is what my wsl network looks like:

~$ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever

@donkkis
Copy link

donkkis commented Aug 27, 2024

Hi folks, we have put out a new update that aims to address networking issues in WSL. In your .wslconfig file you can set experimental.networkingMode=mirrored, as well as some other key settings that should improve your network compatibility! Please try them out and let us know what you think.

After years of struggling with various hacks and workarounds, I was eager to try this one out. Unfortunately, someone at my company's IT dept has decided to set the DisabledComponents to 0xFF in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters , which effectively torpedoes any attempts to use the mirrored mode with WSL, as explained thorougly in #11002.

I'm willing to go on a mission in corporate to get that policy changed, but before this I'd like some reassurance that the effort actually stands a chance to yield some progress.

@craigloewen-msft are there confirmed cases that networkingMode=mirrored has helped people specifically suffering from loss of connectivity while connected to VPN via Cisco AnyConnect?

@dalgibbard
Copy link

@donkkis as a Cisco Anyconnect user with enforced full-tunnel connection, I can confirm that mirrored networking does work as you would hope.
I have been using it since the feature was first released.

@timoooo
Copy link

timoooo commented Sep 5, 2024

@dalgibbard So no further workaround are needed? Can you tell me what AnyConnect version you are using?

With AnyConnect we can resolve our internal IPs and connect them, but anything that goes outside (e.g. archive.ubuntu.com fails) fails to connect to. According to our IT Department its because the Proxy authentication fails.

@balmeida-nokia
Copy link

@timoooo With the new versions of AnyConnect,
To deal with proxies, set the HTTP_PROXY collection of environment variables so everything connects using your company's proxy

@timoooo
Copy link

timoooo commented Sep 5, 2024

I set the values HTTP_PROXY, HTTPS_PROXY, http_proxy and https_proxy to my Proxy address but the issues persist

@balmeida-nokia
Copy link

balmeida-nokia commented Sep 5, 2024

Are you passing them to apt when you run sudo (sudo clears most env vars by default)?

@timoooo
Copy link

timoooo commented Sep 5, 2024

@balmeida-nokia no i just run sudo apt ... straight after setting the env variables

@timoooo
Copy link

timoooo commented Sep 5, 2024

@balmeida-nokia thanks that worked for the Ubuntu repos however i appaerently need to authenticate to get to Err:1 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease
Invalid response from proxy: HTTP/1.1 407 authenticationrequired Date: Thu, 05 Sep 2024 13:10:31 GMT Content-Type: text/html Cache-Control: no-cache Content-Length: 4734 X-Frame-Options: deny Proxy-Connection: Keep-Alive Proxy-Authenticate: Negotiate any ideas how i could authenticate to the proxy via wsl?

@balmeida-nokia
Copy link

Did you try searching for HTTP_PROXY authentication?

@madscientist42
Copy link

madscientist42 commented Oct 30, 2024

This is still busted out of the box. C'mon, Microsoft...I don't care if it's FOSS, it's a PRODUCT you offer- fix this properly.

I shouldn't have to use Admin Mode Powershell scripting hacks to "fix" this. Even though it's slower, VirtualBox manages to actually WORK in NAT mode and YOU DO NOT here.

Five years, guys. This isn't acceptable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
network wsl2 Issue/feature applies to WSL 2
Projects
None yet
Development

No branches or pull requests