IPSec / L2TP connections broken

[rmenessec]

I'm using OmniROM (4.4.2) nightlies. Either the OmniROM kernel does not support UDP connection tracking, AFWAll is failing to use connection tracking, or the default ruleset is breaking UDP connection tracking.

Enabling the "inbound" option in Preferences has no effect. AFWall has to be completely disabled to use the native VPN support.

In watching packet captures, I see that my device is returning ICMP unreachable when return packets arrive. The native Android client is using tunneled IPSec (4500/udp) as is common practice.

[cernekee]

Normally AFWall only restricts outbound traffic. Are there any INPUT rules in your setup that could account for the errors you see?

What do you see in the firewall log?

[rmenessec]

There are no custom INPUT rules. I'm running AFWall+ defaults.

The firewall log shows only that it's blocking packets from something running as root. No UID is shown. The box for recording UIDs is ticked. Is it possible that an application name has changed in 4.4.2 or in OmniROM? AFWall+ is supposedly configured to permit packets for "VPN networking" (and almost every other system app).

Apart from disabling the firewall entirely, the only other way this works is if I tick the boxes for "all" applications running as UID 0.

It's definitely AFWall+ blocking the VPN service.

[cernekee]

Could you please post the output from:

su -c dmesg | grep AFL

Thanks

[rmenessec]

Nothing. grep returned with exitcode 1.

[cernekee]

Did you try immediately after seeing the blocked traffic problem?

Does your afwall-reject chain use the LOG target, the NFLOG target, or neither?

[rmenessec]

Ah. Wait, here we go. This is just after attempting to connect to the VPN provider.

<4>[71331.985637] c1 {AFL}IN= OUT=wlan0 SRC=10.0.0.210 DST=66.55.144.180 LEN=472 TOS=0x00 PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=500 DPT=500 LEN=452 UID=0 GID=1016

[rmenessec]

To answer your other question... I think ...'iptables -n -L | grep -i log' returns this.

LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 1000/min burst 5 LOG flags 8 level 4 prefix "{AFL}"

[cernekee]

PROTO=UDP SPT=500 DPT=500

This looks like IPsec IKE traffic - do you see "racoon" in ps?

UID=0 GID=1016

Our "VPN networking" option tries to match UID 1016 (vpn). I see that UID 1016 is used by ipsec-tools in AOSP 4.4.2:

external/ipsec-tools/main.c:    setuid(AID_VPN);

But maybe we need to match GID 1016 too.

[rmenessec]

This is the last time I could catch it running:

vpn       1694  1     2448   1140  c0116cd8 4015a8e8 S /system/bin/racoon

After a few quick searches, I'm not seeing an easy way to map user and group names to UIDs and GIDs.

[cernekee]

After a few quick searches, I'm not seeing an easy way to map user and group names to UIDs and GIDs.

https://android.googlesource.com/platform/system/core/+/master/include/private/android_filesystem_config.h

vpn 1694 1 2448 1140 c0116cd8 4015a8e8 S /system/bin/racoon

Hmm, this looks like UID vpn, but the log showed UID root.

Can you look it up in /proc, e.g.

su -c 'cat /proc/1694/status'
su -c 'strings /proc/1694/cmdline'

[rmenessec]

No cmdline contents, but I've got this. Notice that there are two processes running as the 'vpn' user:

root@t0ltevzw:/ # ps | grep vpn                                                
vpn       12462 1     2468   1400  c0116cd8 401428e8 S /system/bin/racoon
vpn       12767 1     1880   676   c0116cd8 401198e8 S /system/bin/mtpd
root@t0ltevzw:/ # cat /proc/12462/status                                       
Name:   racoon
State:  S (sleeping)
Tgid:   12462
Pid:    12462
PPid:   1
TracerPid:      0
Uid:    1016    1016    1016    1016
Gid:    1016    1016    1016    1016
FDSize: 32
Groups: 3003 3005 
VmPeak:     2468 kB
VmSize:     2468 kB
VmLck:         0 kB
VmHWM:      1400 kB
VmRSS:      1400 kB
VmData:      188 kB
VmStk:       136 kB
VmExe:       148 kB
VmLib:      1652 kB
VmPTE:        10 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/14330
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 000000000000c4fb
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed:   f
Cpus_allowed_list:      0-3
voluntary_ctxt_switches:        165
nonvoluntary_ctxt_switches:     39
root@t0ltevzw:/ # cat /proc/12767/status                                       
Name:   mtpd
State:  S (sleeping)
Tgid:   12767
Pid:    12767
PPid:   1
TracerPid:      0
Uid:    1016    1016    1016    1016
Gid:    1016    1016    1016    1016
FDSize: 32
Groups: 3003 3004 3005 
VmPeak:     1880 kB
VmSize:     1880 kB
VmLck:         0 kB
VmHWM:       676 kB
VmRSS:       676 kB
VmData:      104 kB
VmStk:       136 kB
VmExe:        16 kB
VmLib:      1356 kB
VmPTE:        10 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/14330
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 000000000001c4fb
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed:   f
Cpus_allowed_list:      0-3
voluntary_ctxt_switches:        65
nonvoluntary_ctxt_switches:     3

[rmenessec]

...the connection failed last time, even with all root procs allowed. After deleting the rules again, I got this, after the connection succeeded:

root@t0ltevzw:/ # ps | grep vpn                                                
vpn       13964 1     2468   1400  c0116cd8 401638e8 S /system/bin/racoon
vpn       14261 1     1880   676   c0116cd8 401a98e8 S /system/bin/mtpd
vpn       14263 14261 2296   932   c0116cd8 4013c7fc S pppd

[cernekee]

Uid: 1016 1016 1016 1016
Gid: 1016 1016 1016 1016

The ordering is: (real, effective, saved, FS) uid/gid. I'm not sure why these processes would show UID=0 unless that is happening before it drops privilege.

...the connection failed last time, even with all root procs allowed

Do you see any new denials logged in dmesg?

[rmenessec]

Every time, unless AFWall+ is disabled. That hasn't changed.

[rmenessec]

By the way, the privilege drop theory is interesting. One of these daemons (not sure which) must either be root or else using Android capabilities. After looking again at the characteristics of the dropped packets, they all have a source port of 500 as well as destination port of 500.

[rmenessec]

So, is there anything else I can do to help get this fixed?

[cernekee]

If you're allowing root apps, I would not expect to see the UID=0 denials showing up in dmesg:

<4>[71331.985637] c1 {AFL}IN= OUT=wlan0 SRC=10.0.0.210 DST=66.55.144.180 LEN=472 TOS=0x00 PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=500 DPT=500 LEN=452 UID=0 GID=1016

Can you figure out which rule(s) are causing the denials? The relevant counters in "iptables -nxvL" should increment for each packet.

[rmenessec]

Well, again, the VPN processes are running as the 'vpn' user by the time I can get ps output. See my comment from 3 days ago. I'm not sure why it worked only the once when I allowed all UID 0 apps. I would treat that as a red herring for now.

The only rule matches I'm certain of are these:

Chain afwall-reject (5 references)
    pkts      bytes target     prot opt in     out     source               destination         
       1      472 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 1000/min burst 5 LOG flags 8 level 4 prefix "{AFL}"
       1      472 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

...

Chain afwall-wifi-wan (1 references)
       1      472 afwall-reject  all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner UID match 0-999999999

That matches up with the ICMP unreach packets I see going back to the VPN provider when I run tcpdump against my firewall's internal port. Now that I think about it, is there anything useful that I can provide in the way of firewall / router logs? I have a real router.

[cernekee]

   1      472 afwall-reject  all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner UID match 0-999999999

That rejected 1 packet. Does that chain have RETURN entries for UID 0 and UID 1016?

[rmenessec]

It does for 1016, but the closest rule for UID 0 is only:

udp dpt:53 owner UID match 0

[rmenessec]

Ah. Yep, there we go. Allowing all for UID 0 once again serves as a temporary workaround. One or more processes are changing UID.

[rmenessec]

So... any updates? Plans to accommodate apps that drop UID 0? That seems to be the only way to resolve this permanently, short of an AOSP 4.4.2+ code change to ensure that UID 0 is dropped before any packets are sent on 500/udp.

[cernekee]

The netfilter "owner" module appears to match the fsuid that was in effect when the process created the socket. So this test case will succeed even if I add a REJECT rule for UID 1234:

iptables -I OUTPUT -m owner --uid-owner 1234 -j REJECT

but moving setfsuid() above the call to socket() changes the result.

If your VPN client creates the IKE sockets as root (which may be done in order to use a source port under 1024) then the easiest way to whitelist it is to allow traffic from root apps.

[rmenessec]

Yes, I know. However, I prefer fine-grained access control. Or are you saying that it's the only fix?

[cernekee]

Couple options here:

• If you prefer to stick with per-UID access controls through the standard AFWall GUI, you can whitelist all root apps, or you can patch the VPN daemons so that sockets are created under e.g. UID 1016 (possibly using something like CAP_NET_BIND_SERVICE to allow binding to low ports).
• If you're willing to write custom firewall rules, you could allow root-owned traffic only on the ports that are used by VPN, and deny it to other apps
• Or, just switch to a non-IPsec VPN, such as OpenVPN or AnyConnect. The latter two have non-root VpnService implementations which can be easily whitelisted without side effects.

In general, if you are hesitant to whitelist root because you are running other root apps that you do not completely trust, that is a bigger problem that should be addressed first.

[rmenessec]

• I don't have an Android development environment.
• I guess this is the best option until/unless AFWall+ special-cases AOSP 4.2.2.
• Unfortunately, because it's not system-level, it comes with a few other problems, such as not being able to take advantage of the Android "VPN leak" prevention, requiring an extra Xposed module to bypass the hideous "VPN warning" dialog, and running a higher risk of being OoM-killed.

I'm not exactly worried that I have malicious root-permissions apps installed on my device. Rather, I'm concerned about running a risk of leaking information, esp. from usage-tracking and crash-reporting code.

Is this more or less what the custom AFWall script should look like in this case?

#!/system/bin/sh

$IPTABLES -A "afwall" -p UDP --dport 500 -m owner --uid-owner 0 -j RETURN

[rmenessec]

...answered my own question. 4500/udp is also required. I added '|| exit 1' to both lines to be on the safe side.