Support firewall logs from Cortex Data Lake #162
Comments
|
🎉 Thanks for opening your first issue here! Welcome to the community! |
btorresgil
changed the title
|
Logs from Cortex Data Lake have been supported for a long time using Log Forwarding in Cortex. However, a recent change to Log Forwarding made it so you can't use Splunk with Cortex if you have customized the filters or create new filters in your Log Forwarding Profile. Important facts about this issue:
You can tell if you have a migrated filter in the Forwarding Profile. Any filters that work with Splunk will say
|
|
I've pinned this issue. |
|
what is different about the migrated filter? My team started recreating PA transforms and other KOs, but if a small tweak results in normal interpretation, I'll put a halt on that. |
|
Migrated filter is any filter from before November that hasn't been changed. As soon as it's changed, it will no longer be migrated and will use the format you designate rather than the standard PANOS format. If you would like to make a change but maintain the migrated filter for the standard PANOS format, you'll need to open a TAC case to make the change and restore the migrated status. This will be resolved in a couple weeks when we release HEC log support for CDL and the Splunk Addon, which is a key value format so it won't have the issue. |
|
I think that you have a specific understanding that I am not quite catching. I think that you are saying that
Based on the Addon, I think that when we receive the migrated log format, we should set the sourcetype to pan:log for the CDL syslog data. Is that correct? Sorry if this all seems obvious to you, it may not be clear across the board. We've been piloting PrismaAccess since the beginning of this year and our PaloAlto contacts did not tell us that there was a special format that we needed to request when forwarding logs to Splunk. In fact a coworker asked support for more info and the response he got suggested to him that it was just supposed to work as is with the Addon. (something that I already learned was not so, resulting in my team creating new sourcetypes prismaaccess:* and trying to map fields ourselves) |
|
@MonkeyKa That's all correct, though for clarity I would rephrase point 2 as "In November, the format changed, but all existing users remained on the PANOS format unless they made a config change. Any new users or old users who made a config change got the new format. Any user who does not want the new format can submit a TAC case to revert to the PANOS format. Yes, you should use Thankfully this will all be resolved by HEC support which is coming very soon. We're wrapping up the Splunk Add-on with HEC support now so all the CIM mappings and our Splunk App dashboards will work regardless if you use PANOS syslogs or CDL with HEC. |
## [6.6.0](v6.5.2...v6.6.0) (2021-04-01) ### Features * **addon:** Cortex Data Lake HEC log support - #162 #176 * **app/addon:** Add IoT Security - #158 ### Bug Fixes * **addon:** Add fields for GlobalProtect logs * **addon:** Add modinputs as tasks in app.manifest - #153 * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix nav bar background color * **addon:** Parse GP and Decryption logs w/ pan:firewall - #168 * **app:** Incident counters flash in Splunk 8.1 - #163 * **app/addon:** correct user-id tag_user / untag_user
|
🎉 This issue has been resolved in version 6.6.0 🎉 This release is available on SplunkBase: App - Add-on
|
|
🎉 This issue has been resolved in version 6.6.0 🎉 This release is available on SplunkBase: App - Add-on
|
## [5.0.0-beta.1](v4.2.2...v5.0.0-beta.1) (2022-03-07) ### Features * **addon:** Add Decryption Log Support for PANOS 10 - PaloAltoNetworks#126 * **addon:** Cortex Data Lake HEC log support - PaloAltoNetworks#162 PaloAltoNetworks#176 * **addon:** PAN Quality Validation and Improvement * **addon:** Significantly improve and modernize CIM compliance * **app/addon:** Add Cortex XDR incident support to App and Add-on including new XDR Incidents dashboard - PaloAltoNetworks#166 * **app/addon:** Add IoT Security - PaloAltoNetworks#158 * **app/addon:** Feature/dynamic user groups - PaloAltoNetworks#150 * **app/addon:** Python 3 Support - PaloAltoNetworks#124 * **app/addon:** Support GlobalProtect log type in PANOS 9.1 - PaloAltoNetworks#118 * **app/addon:** Update pandevice to 0.14.0 - PaloAltoNetworks#145 ### Bug Fixes * **addon:** Add fields for GlobalProtect logs * **addon:** Add fields for GlobalProtect logs * **addon:** Add GlobalProtect SourceUserName - PaloAltoNetworks#209 PaloAltoNetworks#202 * **addon:** Add modinputs as tasks in app.manifest - PaloAltoNetworks#153 * **addon:** Add virus eventtype to malware CIM - PaloAltoNetworks#114 PaloAltoNetworks#138 * **addon:** Fix appserver/static files * **addon:** Fix CDL logs contained string 'null' in 'user' field - PaloAltoNetworks#187 * **addon:** Fix error from Minemeld automatic lookup * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix nav bar background color * **addon:** Fix src_user field contained destination user - PaloAltoNetworks#186 * **addon:** Fix typo in transform.conf ([PaloAltoNetworks#227](https://github.com/btorresgil/SplunkforPaloAltoNetworks/issues/227)) * **addon:** Fix user showing as unknown from GlobalProtect logs. - PaloAltoNetworks#217 * **addon:** Parse GP and Decryption logs w/ pan:firewall - PaloAltoNetworks#168 * **addon:** Parse GP and Decryption logs w/ pan:firewall - PaloAltoNetworks#168 * **addon:** Remove endpoint tags and eventtypes - PaloAltoNetworks#196 * **addon:** Remove port from `dest_name` field - PaloAltoNetworks#129 PaloAltoNetworks#128 * **addon:** Remove white space from GlobalProtect sourcetype - PaloAltoNetworks#131 * **addon:** Restore "unknown" string for empty 'user' field * **app:** Fix error after upgrade to 7.0.0: "Unknown search command 'panwildfirereport'" - PaloAltoNetworks#189 * **app:** Fix IoT Security dashboard filter - PaloAltoNetworks#181 * **app:** Fix panContentPack error. Fixes bug [PaloAltoNetworks#222](https://github.com/btorresgil/SplunkforPaloAltoNetworks/issues/222) - PaloAltoNetworks#225 * **app:** Incident counters flash in Splunk 8.1 - PaloAltoNetworks#163 * **app:** Incident counters flash in Splunk 8.1 - PaloAltoNetworks#163 * **app:** Remove endpoint from Data Model Audit dashboard - PaloAltoNetworks#218 * **app/addon:** correct user-id tag_user / untag_user * **app/addon:** Fix background color of logo - PaloAltoNetworks#141 ### Performance Improvements * **app:** Change simple XML to use JQuery 3.5 - PaloAltoNetworks#207 * **app:** Remove high cardinality fields from datamodel ### ⚠ MAJOR RELEASE CHANGES This is a major release Splunk dashboards and searches you have created might be affected by these changes. Please be prepared to test and adjust any dashboards not included with the App after upgrade. * **addon:** pan_traffic_start logs no longer included in CIM * **addon:** pan_traffic_end logs moved from Network Session to Network Traffic datamodel * **addon:** pan_threat event type now includes wildfire and data logs * **addon:** pan_file logs moved from Web to IDS datamodel * **addon:** pan_virus logs moved from Malware to IDS datamodel * **addon:** pan_wildfire logs moved from Malware to IDS datamodel * **addon:** pan_email removed from Email datamodel * **app:** Removes datamodel for GlobalProtect logs before PAN-OS 9.1 * **app/addon:** Removes Traps 4 support * **app/addon:** Deprecates Traps 5 and Traps 6 support * **app:** Removes support for legacy WildFire Report API * **app/addon:** Requires Splunk 8.0 or higher * **app/addon:** Replaces Adversary Scoreboard and Incident Feed dashboards with new XDR Incidents dashboard
Is your feature request related to a problem?
Customers using Splunk/Splunk Connect for Syslog have asked for support for cortex as a source in Splunk.
Describe the solution you'd like
On closer inspection we have realized the csv order of fields from cortext are not identical to pan sources preventing use of the existing addon source types. Update the addon to provide cortex:* version of pan:* with the correct order
Describe alternatives you've considered
Considered have the customer reorder the fields in cortext via the UI but this seems impracticle at scale
Additional context
Blocked use of cortext and Splunk
The text was updated successfully, but these errors were encountered: