[Filebeat] Palo Alto integration with GlobalProtect

[willemdh]

Please provide support for the panw panos globalprotect log type..

Example logs:

1,2021/03/24 11:30:00,022201001305,GLOBALPROTECT,0,2305,2021/03/24 11:30:00,vsys1,portal-prelogin,before-login,,,,BE,,11.134.5.168,0.0.0.0,0.0.0.0,0.0.0.0,09300bcc-23-4900-8de9-32695452fa,,5.2.4,Windows,"Microsoft Windows 10 Pro , 64-bit",1,,,"",success,,0,,0,GlobalProtect Portal,69200719497738,0x0

1,2021/03/24 11:29:49,022201001308,GLOBALPROTECT,0,2305,2021/03/24 11:29:49,vsys1,gateway-config-release,configuration,,,domain\user,BE,CP935,83.14.113.11,0.0.0.0,10.20.13.217,0.0.0.0,e0957c11-93-437a-9e23-9f0c24059898,5J9VN53,5.2.4,Windows,"Microsoft Windows 10 Pro , 64-bit",1,,,"",success,,0,,0,GlobalProtect_GW,6919501582016786,0x0

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[legoguy1000]

I can try to update the PanOS fileset to parse these logs.

[legoguy1000]

I've started a draft for this

[willemdh]

@legoguy1000 Feel free to ask me questions or let me know if you want some data or let me test what you have.

[legoguy1000]

@legoguy1000 Feel free to ask me questions or let me know if you want some data or let me test what you have.

The more sample data with variations you can provide, the better and more accurate the ingest pipelines can be.

[willemdh]

@legoguy1000 had a look at your PR and I must say it looks good.

I would like to make a suggestion though. Currently user.name is never populated. Imho, after intensively using panw.panos for over a year, I have though multiple times that if would be a good idea to copy client.user.name to user.name. Or at least the part without the domain.

client.user.name looks like this currently => DOMAIN\USER

It's the only log source we have that is using this kind of username notation. Maybe client.user.name DOMAIN could be copied to user.domain and client.user.name USER to user.name ?

This actually comes down to the same problem of what convention Elastic is planning to use for user.name.

See elastic/ecs#1239

Maybe @webmat or @ebeahan know what's the state of the multiple user RFC and can confirm if my propsal to partially copy client.user.name to user.name is a good idea.

[willemdh]

Also, as per your request I sanitized a few more globalprotect logs:

1,2021/04/07 17:41:30,013101305,GLOBALPROTECT,0,2305,2021/04/07 17:41:30,vsys1,gateway-hip-check,host-info,,,domain\user1,,HOST82878,7.2.2.193,0.0.0.0,12.30.0.210,0.0.0.0,523e8b-7efa-4397-a4d5-824dfa4d8a,F1SM2,5.2.4,,"",1,,,"HIP report is not needed",success,,0,,0,GlobalProtect_GW,6920071768563516860,0x0

1,2021/04/07 17:41:29,013101308,GLOBALPROTECT,0,2305,2021/04/07 17:41:29,vsys1,gateway-getconfig,configuration,,IPSec,pre-logon,BE,HOST73486,7.2.2.171,0.0.0.0,1.40.2.67,0.0.0.0,7d01b5-f538-4fa3-a2a2-83980d1325,5C261FNR,5.2.4,Windows,"Microsoft Windows 10 Pro , 64-bit",1,,,"Config name: , Client region: BE.",success,,0,,0,GlobalProtect_GW,6944137135219737,0x0

1,2021/04/07 17:41:28,0131001309,GLOBALPROTECT,0,2305,2021/04/07 17:41:28,vsys1,gateway-tunnel-latency,tunnel,,,userlterso,HOSTP92413,7.2.17.120,0.0.0.0,0.0.0.0,0.0.0.0,2ba9f01-b83b-4902-a1fb-1748c0365,GJG98Y2,5.2.4,,"",1,,,"Pre-tunnel latency: 67ms, Post-tunnel latency: 47ms",success,,0,,0,GlobalProtect_GW,6920071768563516847,0x0

[legoguy1000]

@legoguy1000 had a look at your PR and I must say it looks good.

I would like to make a suggestion though. Currently user.name is never populated. Imho, after intensively using panw.panos for over a year, I have though multiple times that if would be a good idea to copy client.user.name to user.name. Or at least the part without the domain.

client.user.name looks like this currently => DOMAIN\USER

It's the only log source we have that is using this kind of username notation. Maybe client.user.name DOMAIN could be copied to user.domain and client.user.name USER to user.name ?

This actually comes down to the same problem of what convention Elastic is planning to use for user.name.

See elastic/ecs#1239

Maybe @webmat or @ebeahan know what's the state of the multiple user RFC and can confirm if my propsal to partially copy client.user.name to user.name is a good idea.

Ya i was reading the spec and focused on teh first line saying user.* should be nested but then also says it may be at the root. I can easily do that. Do you think its worth keeping at source.user or client.user if its at the root of the document??

[legoguy1000]

Should be updated with the new logs u provided and the User ID logs as well.

[ebeahan]

This actually comes down to the same problem of what convention Elastic is planning to use for user.name.

For an event containing a source.user ( or client.user) and destination.user (or server.user), the guidance is to copy the source.user fields to user at the root. A full example of this type of use case is discussed here (this section was recently added to the ECS docs based on the multiple users RFC 😃 ).

However, for GlobalProtect and User-ID logs, would the user values go into top-level user.* fields instead of source.user.*? The source.user.* fields are intended to be paired with destination.user.*, and while some Palo events have both source/destination user (like traffic), I only see a single user in either User-ID or GlobalProtect log fields.

[legoguy1000]

I think that as long as its minimally at the root user.*, having it in the source.user.* doesn't hurt anything for now.

[legoguy1000]

I updated the pipeline to copy source.user to user

[legoguy1000]

Took PR out of draft