mount nfs/cifs is blocked despite no deny rules

[carpii]

Describe the bug
opensnitch is configured to deny connections when 'GUI is disconnected'
Also configured to prompt for new rules, if GUI is running and enabled
I have no deny rules at all in my opensnitch UI
If I try to mount remote NFS or CIFS it blocks indefinitely (no GUI prompt, even though GUI is running and enabled).

If I keep the GUI running, but choose to disable it, the issue is resolved but Im confused why it's being blocked to begin with?

Versions

• OpenSnitch version: 1.5.2
• OS: ArchLinux
• Version: 5.19.12-arch1-1
• Window Manager: xfce desktop with awesomewm
• Kernel version: 5.19.12-arch1-1 x86_64 GNU/Linux

Steps to reproduce the behavior

• Remove all deny rules from opensnitch
• Make sure gui is connected to daemon
• Try to mount a remote NFS or CIFS filesystem from commandline, will block indefinitely until GUI is 'disabled'

Post error logs:
No relevant logs as far as I can see

Expected behavior (optional)
If GUI is disconnected I woudl expect it to deny
If GUI is running and enabled, then I would expect it to prompt me (since there are no deny rules in the UI)

[gustavo-iniguez-goya]

Hi @carpii ,

Probably opensnitch is not discovering the process of those connections at kernel level. There has been a change in kernel 5.19 that breaks our interception of VPNs (#454 ), but last time I tried nfs worked fine.

Could you post the output of the following commands?

$ objdump -h /etc/opensnitchd/opensnitch.o | grep xmit
$ sudo grep xmit /sys/kernel/debug/tracing/kprobe_events

As a workaround you can enable [x] Debug invalid connections under Preferences -> Nodes, you should see a pop-up to allow an "Outgoing connection".

[carpii]

Hi @gustavo-iniguez-goya

Thanks for the quick reply.

[Edit: This reply was edited from earlier, as I noticed I did not have opensnitch-ebpf-module package installed]

[carpii@beep:~]$ objdump -h /etc/opensnitchd/opensnitch.o | grep xmit
  7 kprobe/iptunnel_xmit 00000390  0000000000000000  0000000000000000  00000c98  2**3

[carpii@beep:~]$ sudo grep xmit /sys/kernel/debug/tracing/kprobe_events
(grep produces no output)

Still unable to mount or unmount while opensnitch UI is running and enabled
There are no UI prompts either, even after enabling the 'Debug Connections' option

[gustavo-iniguez-goya]

[carpii@beep:~]$ sudo grep xmit /sys/kernel/debug/tracing/kprobe_events
(grep produces no output)

If you didn't restart the daemon do it now (and try again): $ sudo systemctl stop opensnitchd; sudo systemctl start opensnitchd

If you already did it, could you review the log /var/log/opensnitchd.log for errors? I think there should be a cryptic error related to ebpf, just after starting the daemon.

[carpii]

I hadnt restarted the daemon, but have now (a few times)

Unfortunately there's very little in the logs....

[2022-09-30 23:58:17]  IMP  Got signal: terminated
[2022-09-30 23:58:17]  IMP  Start writing logs to /var/log/opensnitchd.log

This covers stopping and restarting daemon, aswell as trying to mount then disabling UI, mounting and then re-enabling UI

I think my problem now is that opensnitchd seems to be exiting shortly after starting...

● opensnitchd.service - OpenSnitch is a GNU/Linux port of the Little Snitch application firewall.
     Loaded: loaded (/usr/lib/systemd/system/opensnitchd.service; enabled; preset: disabled)
     Active: activating (auto-restart) (Result: exit-code) since Sat 2022-10-01 01:02:57 BST; 2s ago
       Docs: https://github.com/gustavo-iniguez-goya/opensnitch/wiki
    Process: 302348 ExecStartPre=/bin/mkdir -p /etc/opensnitchd/rules (code=exited, status=0/SUCCESS)
    Process: 302349 ExecStart=/usr/bin/opensnitchd -rules-path /etc/opensnitchd/rules (code=exited, status=2)
   Main PID: 302349 (code=exited, status=2)
        CPU: 150ms

I'll test again after a reboot tomorrow, and report back. Thanks for the help so far

[gustavo-iniguez-goya]

Yes, it seems that it's not starting.There should be errors in the journalctl, but try launching it manually as root, it'll output more errors:
# /usr/bin/opensnitchd -rules-path /etc/opensnitchd/rules/

be sure that it's not running: $ pgrep -a opensnitchd

[gustavo-iniguez-goya]

oh by the way, 1) once it's started and running, and 2) $ sudo grep xmit /sys/kernel/debug/tracing/kprobe_events output is not empty , if it still doesn't prompt you to allow/deny the connection:

1. Set log level to DEBUG , under Preferences -> Nodes
2. Empty the log file: $ sudo truncate -s 0 /var/log/opensnitchd.log
3. try mounting the nfs share again (post the command you are using, like -> mount -t nfs 1.2.3.4:/xxx /yyy

If enabling [x] Debug invalid connections doesn't intercept it, there's something strange.

[carpii]

I've been troubleshooting this a bit more...

The opensnitchd service was terminating shortly after a start, due to a segfault. I noticed I was still using 1.52 so upgraded to the git version from AUR (1.6x). I also enabled the DEBUG connections option.

At this point, it did start prompting me for kernel level connections, although they were listed as 'unknown process' and I still could not use mount. I'm a little confused on whether the process monitor is working properly (still ebpf).

When opensnitchd is started, I now see an error in the logs....

[2022-10-03 21:42:47]  IMP  Start writing logs to /var/log/opensnitchd.log
[2022-10-03 21:42:47]  ERR  Error parsing firewall configuration /etc/opensnitchd/system-fw.json: json: cannot unmarshal string into Go struct field FwRule.SystemRules.Rule.Position of type uint64

This is odd, because the system rules display ok in the UI (I dont think my previous version had a section to show system rules).

But it appears to have a knock-on effect, that none of my user rules appear in the UI anymore, even though they still exist in /etc/opensnitchd/rules/
It's not clear whether the user rules are still being applied.. I think they are, so maybe just a GUI issue caused by the earlier error?

I've had to disable opensnitchd for the time being, as it now seems to prevent me logging into xfce (it was just shows a black screen with mouse pointer after logging in - maybe a system connection being blocked, I'm not sure).

Is there a way to just reset /etc/opensnitchd/system-fw.json to initial state, so that I don't get a parser error anymore?

[gustavo-iniguez-goya]

Is there a way to just reset /etc/opensnitchd/system-fw.json to initial state, so that I don't get a parser error anymore?

The system-fw.json error message shouldn't be there really. That field is not being used right now, so you can ignore it.

It's not clear whether the user rules are still being applied.. I think they are, so maybe just a GUI issue caused by the earlier error?

hmm, applications rules should appear in the GUI. If there're any errors loading them there should be a warning message in the log /var/log/opensnitchd.log:

WAR Error parsing rule from /etc/opensnitchd/rules/test.json: invalid character ':' after object key:value pair

If you create a new rule, is it added to the list of rules? is it loaded after reboot?

I'm a little confused on whether the process monitor is working properly (still ebpf).

If you've been prompted to allow kernel connections, the monitor method is ebpf for sure, but you can grep the logs for this trace:
INF Process monitor method ebpf

I've had to disable opensnitchd for the time being, as it now seems to prevent me logging into xfce (it was just shows a black screen with mouse pointer after logging in - maybe a system connection being blocked, I'm not sure).

You can try to set the DefaultAction to allow in /etc/opensnitchd/default-config.json, that way when the daemon starts the connections will be allowed and a temporary rule will be created. When the daemon connects to the GUI, there should be a temporary rule in the panel Rules -> App Rules -> Temporary.

Probably it'll be caused by dirmngr or xbrlapi

[gustavo-iniguez-goya]

At this point, it did start prompting me for kernel level connections, although they were listed as 'unknown process'

Please, check (as root) if the file /sys/kernel/debug/tracing/kprobe_events has entries like these ones when the daemon is running:

p:kprobes/pudpv6_sendmsg udpv6_sendmsg
p:kprobes/piptunnel_xmit iptunnel_xmit     <----------------
p:kprobes/ptcp_v4_connect tcp_v4_connect
r16:kprobes/rtcp_v4_connect tcp_v4_connect
p:kprobes/ptcp_v6_connect tcp_v6_connect
r16:kprobes/rtcp_v6_connect tcp_v6_connect
p:kprobes/pudp_sendmsg udp_sendmsg

If iptunnel_xmit doesn't appear there, then we're not going to be able to intercept kernel connections.
If it's not listed, please paste the output of this command: # grep iptunnel /sys/kernel/debug/tracing/available_events

On the other hand, if it's an intercepted kernel connection, the pop-up should say it (instead of "unknown"):

[carpii]

Appreciate the help, thanks. I'll do some more testing over the next few days, so I've something useful to report.

But yes, after installing the git version, I was using the command you suggested earlier..

sudo grep xmit /sys/kernel/debug/tracing/kprobe_events

and this was returning an xmit line (whereas previously it wasnt).

[gustavo-iniguez-goya]

@carpii sorry :( usually using opensnitch is very straightforward, but in some cases...

and this was returning an xmit line (whereas previously it wasnt).

That's good. At least your kernel supports it. Please, set log level to DEBUG, try to reproduce it again and post the logs.

[seonwoolee]

I'm having this issue as well on Arch Linux, opensnitch version 1.6.0rc2.r48.a3538a7. I know NFS only uses port 2049 so I have an allow from any process to 10.42.42.4:2049 (10.42.42.4 being the IP of my NFS server) rule. I know this used to work before Sept 12th or so.

But for me, when I run objdump -h /etc/opensnitchd/opensnitch.o | grep xmit, it says there is no such file /etc/opensnitchd/opensnitch.o (which is true, I don't see that file in there).
And sudo grep xmit /sys/kernel/debug/tracing/kprobe_events also says /sys/kernel/debug/tracing/kprobe_events doesn't exist. Heck, /sys/kernel/debug itself doesn't exist at all

I do have debug invalid connections checked and ebpf set as the process monitor method.

What's interesting is that when I try the NFS connection, I see this in the logs

[2022-11-01 14:30:00]  DBG  new connection tcp => 858:10.42.42.5 -> 10.42.42.4:2049 uid: 0
[2022-11-01 14:30:00]  DBG  [0/1] outgoing connection uid: 0, 858:10.42.42.5 -> 10.42.42.4:2049 || netlink response: 858:10.42.42.5 -> 10.42.42.4:2049 inode: 222683 - loopback: false multicast: false unspecified: false linklocalunicast: false ifaceLocalMulticast: false GlobalUni: true 
[2022-11-01 14:30:00]  DBG  new pid lookup took (-1): 55.550515ms
[2022-11-01 14:30:00]  DBG  (1) Could not find process by its pid -1 for: 858:10.42.42.5 (uid:-1) ->(tcp)-> 10.42.42.4:2049

[seonwoolee]

Update: I thought I had the ebpf module installed, but it turns out I did not. After installing it and restarting opensnitchd, objdump -h /etc/opensnitchd/opensnitch.o | grep xmit returns

7 kprobe/iptunnel_xmit 00000390  0000000000000000  0000000000000000  00000c98  2**3

/sys/kernel/debug/ is still not a valid path though.

I still get the same couldn't find process by its pid error

[bingobr]

works for me when I edit /etc/opensnitchd/system-fw.json to say

{ "SystemRules": [ { "Rule": { "Description": "Allow icmp", "Table": "mangle", "Chain": "OUTPUT", "Parameters": "-p icmp", "Target": "ACCEPT", "TargetParameters": "" } }, { "Rule": { "Description": "Allow nfs", "Table": "mangle", "Chain": "OUTPUT", "Parameters": "-p tcp --dport 2049", "Target": "ACCEPT", "TargetParameters": "" } } ] }

[gustavo-iniguez-goya]

hey @bingobr @seonwoolee @carpii could any of you try out this module? udp_tunnel_xmit_skb.zip
md5sums: opensnitch.o -> bd398d7f0fdc719ce043510ca06bd6d5, opensnitch.c -> 8d19ec5ccd2e21e385715b25fbb02a36

Now instead of hooking ip_tunnel_xmit now udp_tunnel_xmit_skb is hooked.

iptunnel_xmit fails with kernels >= 5.19 if we compile the module against kernel 5.8 because the sk_buff struct changed. But with udp_tunnel_xmit_skb we can access directly the source and dest port via parameters.

The zipped module has been compiled against kernel 5.8, tested with kernels 5.15 (wireguard + nfs were already intercepted) and 5.19 (wireguard and nfs were not intercepted, needed to be compiled against kernel 5.19), and with both kernels nfs and vpns are intercepted (ubuntu 22).

Just backup your module (sudo cp /etc/opensnitchd/opensnitch.o /etc/opensnitchd/opensnitch.o.bak), and replace it with the new one.

After stop and start opensnitch, there should be an entry like this both in the module and kprobe_events file:

$ objdump -h opensnitch.o | grep udp_tunnel
7 kprobe/udp_tunnel_xmit_skb 00000400  0000000000000000  0000000000000000  00000c98  2**3
$ sudo grep udp_tunnel /sys/kernel/debug/tracing/kprobe_events
p:kprobes/pudp_tunnel_xmit_skb udp_tunnel_xmit_skb

[gustavo-iniguez-goya]

hmm, so yesterday I was able to insert a kprobe for udp_tunnel_xmit_skb on ubuntu 22 for kernels 5.15 and 5.19, but today that function is not exported by the kernel on the same system, without changes?(!)

If the kprobe is not added to kprobe_events, please verify if the symbol is exported by your kernel:

$ sudo grep udp_tunnel /sys/kernel/debug/tracing/available_filter_functions

On a 5.19 Debian kernel there're 27 functions related to udp_tunnel. On ubuntu (today...?) 0. I need to investigate this behaviour.

[gustavo-iniguez-goya]

Ok, found the problem: you need to load the module udp_tunnel. It's loaded whenever a vpn, ip2ip tunnel, etc is configured.

So, before start the daemon and after copy the new module, load it: sudo modprobe udp_tunnel

all the steps:

$ sudo service opensnitch stop
$ sudo mount udp_tunnel
$ unzip udp_tunnel_xmit_skb.zip
$ sudo cp  /etc/opensnitchd/opensnitch.o /etc/opensnitchd/opensnitch.o.bak
$ sudo cp opensnitch.o /etc/opensnitchd/opensnitch.o
$ sudo grep udp_tunnel_xmit /sys/kernel/debug/tracing/available_filter_functions
udp_tunnel_xmit_skb [udp_tunnel]
$ sudo service opensnitch start
$ sudo grep udp_tunnel_xmit /sys/kernel/debug/tracing/kprobe_events
p:kprobes/pudp_tunnel_xmit_skb udp_tunnel_xmit_skb

both greps must output the above lines.

[gustavo-iniguez-goya]

Regarding this issue, latest v1.6.0rc5 is compiled against 5.19, so nfs should work again.
Also, now if [x] Debug invalid connections is selected, you should be prompted to allow these connections.

On the other hand, for < 5.19 kernels there're precompiled binaries that you can use: https://github.com/evilsocket/opensnitch/actions/runs/4276068016

[gustavo-iniguez-goya]

hi!

I've changed the way we compile the ebpf modules: #942

Could you download and test the precompiled binaries to see if there's any errors before the next release? (links at the bottom of that discussion)

Thank you!