Iptables module unable to parse some logs from UDM-Pro device

[JAndritsch]

I'm using the Iptables module of Filebeat (master branch on Github, commit ddcf8f1aa) to receive and parse logs from a Unifi Dream Machine Pro over UDP. My module configuration looks like this:

- module: iptables
  log:
    enabled: true

The data is sent from the beat to a Logstash pipeline. That pipeline does no processing/parsing of the message itself. It just sends the incoming data into the appropriate Elasticsearch index. All of the message processing happens using the ingest pipeline provided by the Iptables module.

Most of the sample log data I've tested parses fine, however there are a handful of logs (about 70 out of 3800) that fail to parse properly by the module.

Here's a few logs that fail to parse:

May  5 20:46:45 My-Office-Gateway user.info kernel: TTL=126 ID=15317 DF PROTO=TCP SPT=59344 DPT=443 WINDOW=8212 RES=0x00 ACK PSH URGP=0
May  5 20:46:46 My-Office-Gateway user.info kernel:  TTL=126 ID=51392 DF PROTO=TCP SPT=51653 DPT=7914 WINDOW=1024 RES=0x00 ACK PSH URGP=0
May  5 20:46:46 My-Office-Gateway user.info kernel: L=126 ID=8698 DF PROTO=TCP SPT=88 DPT=51179 WINDOW=2053 RES=0x00 ACK URGP=0
May  5 20:47:09 My-Office-Gateway user.info kernel: 0 TTL=126 ID=15461 DF PROTO=TCP SPT=59289 DPT=443 WINDOW=8208 RES=0x00 ACK PSH URGP=0
May  5 20:46:56 My-Office-Gateway user.info kernel: L=126 ID=8702 DF PROTO=TCP SPT=88 DPT=51182 WINDOW=2053 RES=0x00 ACK URGP=0
May  5 20:45:44 My-Office-Gateway user.info kernel: TL=126 ID=4622 DF PROTO=TCP SPT=389 DPT=49209 WINDOW=8192 RES=0x00 ECE ACK SYN URGP=0

[legoguy1000]

Should be easy to update the grok pattern

[legoguy1000]

are some of those supposed to be L and TL and not TTL??

[JAndritsch]

are some of those supposed to be L and TL and not TTL??

That's exactly how they appear in my log file. It almost looks as if part of the message is missing, but then the date and other information is still there.

[legoguy1000]

are some of those supposed to be L and TL and not TTL??

That's exactly how they appear in my log file. It almost looks as if part of the message is missing, but then the date and other information is still there.

Ya, I'm definitely confused about that. Could it be a UDM bug? Did this just start happening recently? after an update??

[JAndritsch]

Ya, I'm definitely confused about that. Could it be a UDM bug? Did this just start happening recently? after an update??

Same here. This is the first time I've tried collecting these logs so I can't say that it was working anytime before. I know very little about UDM, but I also wondered if it could be a bug with the device. I'll see if I can get in touch with someone who might be able to help figure this out.

Thanks for looking at this so quickly. I'll let you know when I have more information.

[legoguy1000]

Ya, I'm definitely confused about that. Could it be a UDM bug? Did this just start happening recently? after an update??

Same here. This is the first time I've tried collecting these logs so I can't say that it was working anytime before. I know very little about UDM, but I also wondered if it could be a bug with the device. I'll see if I can get in touch with someone who might be able to help figure this out.

Thanks for looking at this so quickly. I'll let you know when I have more information.

I would hit up the Unifi forums. You'd probably get teh best answer there.

[JAndritsch]

I would hit up the Unifi forums. You'd probably get teh best answer there.

That's a great idea. Thank you!

[legoguy1000]

I opened a draft PR to parse these logs. Let me know what u think.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)