yggdrasil's IP seems unroutable when existing network interfaces contain v6 address

[dilinger]

On my LineageOS phone with Yggdrasil 0.1-019 from f-droid, I've run into the following problem.

Phone yggdrasil IP: 202:[...]:5131
Debian host 'hm90' (wired, on ethernet) yggdrasil IP: 200:[...]:dbd7
Phone has Yggdrasil enabled showing up as the only VPN, "Always-on VPN" enabled, and "Block connections without VPN" disabled.

There are multiple yggdrasil peers seen by the phone, and I can ping them from an android terminal (and in the other direction, ping the phone's yggdrasil IP from the other peers), but trying to contact them over TCP fails. In particular, I have an HTTP service running on a Debian host (the dbd7 IP above) called 'hm90' that is unreachable with standard routing from the phone. Again, I can use ping6 to reach it, but none of the browsers or other apps work. Here's what happens from an adb console:

130|OP594DL1:/ $ curl -v -6 http://hm90.[...]:4533/                                 
* Host hm90.[...]:4533 was resolved.
* IPv6: 200:[...]:dbd7
* IPv4: (none)
*   Trying [200:[...]:dbd7]:4533...
* connect to 200:[...]:dbd7 port 4533 from 2607:fb91:[...]:be14 port 43122 failed: Connection timed out
* Failed to connect to hm90.[...] port 4533 after 31459 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to hm90.[...] port 4533 after 31459 ms: Couldn't connect to server

Note that the IPv6 address 2607:fb91:[...]:be14 that it's attempting to use is bound to wlan0, and is a real (non-yggdrasil) routable address issued by my ISP. I'm not sure how android handles routing tables, but 'ip -6 r' is empty. So this appears to be a routing issue. If I manually specify the interface to use, it will actually work:

130|OP594DL1:/ $ curl -v -6 --interface tun0 http://hm90.[...]:4533/                
* Host hm90.[...]:4533 was resolved.
* IPv6: 200:[...]:dbd7
* IPv4: (none)
*   Trying [200:[...]:dbd7]:4533...
* socket successfully bound to interface 'tun0'
* Connected to hm90.[...] (200:[...]:dbd7) port 4533
> GET / HTTP/1.1
> Host: hm90.[...]:4533
> User-Agent: curl/8.6.0-DEV
> Accept: */*
> 
< HTTP/1.1 302 Found
< Content-Type: text/html; charset=utf-8
< Location: /music/app/
< Permissions-Policy: autoplay=(), camera=(), microphone=(), usb=()
< Referrer-Policy: same-origin
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< Date: Tue, 31 Dec 2024 14:57:12 GMT
< Content-Length: 34
< 
<a href="/music/app/">Found</a>.

* Connection #0 to host hm90.[...] left intact

Unfortunately with other apps, I can't manually specify the path for a TCP connection to take. However, I can make it work by enabling "Block connections without VPN", so that all packets are routing through the tun0 interface. That is not what I want, though, as this is a private yggdrasil network and I don't have any kind of routing/tunneling through it; in other words, enabling "Block connections without VPN" allows me to make TCP connections to the yggdrasil 'hm90' host, but then I can't reach any normal internet hosts (like www.google.com). So that's not really a solution.

[dilinger]

Alright, after messing around a bunch with android's wacky routing tables; it turns out that it makes heavy use of policy routing. The output from 'ip rule' changes between the "Block connections without VPN" being enabled and disabled, while the individual routing tables (shown by 'ip -6 route show table tun0' and 'ip -6 route show table wlan0' stays the same.

The tun0 table looks reasonable:

OnePlus6:/ # ip -6 route show table tun0                                                     
200::/7 dev tun0 proto static metric 1024 pref medium
2000:: dev tun0 proto static metric 1024 pref medium

The diff between "Block connections without VPN" being disabled and enabled:

@@ -5,13 +5,18 @@
 11000:	from all iif lo oif wlan0 uidrange 0-0 lookup wlan0 
 11000:	from all iif lo oif r_rmnet_data0 uidrange 0-0 lookup r_rmnet_data0 
 12000:	from all iif tun0 lookup local_network 
+13000:	from all fwmark 0x0/0x20000 iif lo uidrange 0-99999 lookup tun0 
+13000:	from all fwmark 0xc0066/0xcffff lookup tun0 
+14000:	from all fwmark 0x0/0x20000 iif lo uidrange 1-10190 prohibit
+14000:	from all fwmark 0x0/0x20000 iif lo uidrange 10192-20190 prohibit
+14000:	from all fwmark 0x0/0x20000 iif lo uidrange 20192-99999 prohibit
 15998:	from all fwmark 0x10064/0x1ffff iif lo uidrange 0-2147483647 lookup wlan0 
 16000:	from all fwmark 0x10063/0x1ffff iif lo lookup local_network 
 16000:	from all fwmark 0xd0001/0xdffff iif lo lookup rmnet_data0 
 16000:	from all fwmark 0x10064/0x1ffff iif lo lookup wlan0 
 16000:	from all fwmark 0xd0002/0xdffff iif lo lookup r_rmnet_data0 
-16000:	from all fwmark 0x10067/0x1ffff iif lo uidrange 0-99999 lookup tun0 
-16000:	from all fwmark 0x10067/0x1ffff iif lo uidrange 0-0 lookup tun0 
+16000:	from all fwmark 0x10066/0x1ffff iif lo uidrange 0-99999 lookup tun0 
+16000:	from all fwmark 0x10066/0x1ffff iif lo uidrange 0-0 lookup tun0 
 17000:	from all iif lo oif dummy0 lookup dummy0 
 17000:	from all fwmark 0xc0000/0xc0000 iif lo oif rmnet_data0 lookup rmnet_data0 
 17000:	from all fwmark 0xc0000/0xc0000 iif lo oif wlan0 lookup wlan0 
@@ -24,10 +29,8 @@
 21000:	from all iif r_rmnet_data0 lookup dummy0 
 21000:	from all iif r_rmnet_data0 lookup wlan0 
 22998:	from all fwmark 0x64/0x1ffff iif lo uidrange 0-2147483647 lookup wlan0 
-24000:	from all fwmark 0x0/0x30000 iif lo uidrange 0-99999 lookup tun0 
-24000:	from all fwmark 0xc0067/0xcffff lookup tun0 
 25000:	from all fwmark 0x0/0x10000 iif lo uidrange 0-2147483647 lookup wlan0_local 
-28000:	from all fwmark 0xc0067/0xdffff lookup wlan0 
+28000:	from all fwmark 0xc0066/0xdffff lookup wlan0 
 29998:	from all fwmark 0x0/0xffff iif lo uidrange 0-2147483647 lookup wlan0 
 31000:	from all fwmark 0xc0000/0xcffff iif lo lookup wlan0 
 32000:	from all unreachable

I have no idea what the firewall marking stuff is (and android's iptables is.. wild). However, it does appear that adding a rule after 12000 or so to check the tun0 table is the way to go. So I tried the following (with "Block connections without VPN" disabled): 'ip -6 rule add from all lookup tun0 pref 12999'

Now it appears to work. I can connect to random sites through wlan0 as expected, and also connections to 200::/7 or whatever go through tun0 as desired. I'm not sure how proper this is, though.