[Filebeat] Add CustomString mapping to CEF for Forcepoint NGFW (elast…

…ic#15910)

* Add CustomString mapping to CEF for Forcepoint NGFW

Closes elastic#14663

CHANGELOG.next.asciidoc

@@ -107,6 +107,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Add dashboard for AWS ELB fileset. {pull}15804[15804]
 - Add dashboard for AWS vpcflow fileset. {pull}16007[16007]
 - Add ECS tls fields to zeek:smtp,rdp,ssl and aws:s3access,elb {issue}15757[15757] {pull}15935[15936]
+- Add custom string mapping to CEF module to support Forcepoint NGFW {issue}14663[14663] {pull}15910[15910]
 
 *Heartbeat*
 

filebeat/docs/fields.asciidoc

@@ -4655,9 +4655,27 @@ type: keyword
 [[exported-fields-cef-module]]
 == CEF fields
 
-Module for receiving CEF logs over Syslog. The module does not add fields beyond what the decode_cef processor provides.
+Module for receiving CEF logs over Syslog. The module adds vendor specific fields in addition to the fields the decode_cef processor provides.
 
 
+
+[float]
+=== forcepoint
+
+Fields for Forcepoint Custom String mappings
+
+
+
+*`forcepoint.virus_id`*::
++
+--
+Virus ID
+
+
+type: keyword
+
+--
+
 [[exported-fields-cisco]]
 == Cisco fields
 

filebeat/docs/modules/cef.asciidoc

@@ -40,13 +40,27 @@ The UDP port to listen for syslog traffic. Defaults to `9003`
 
 NOTE: Ports below 1024 require Filebeat to run as root.
 
+[float]
+==== Forcepoint NGFW Security Management Center
+
+This module will process CEF data from Forcepoint NGFW Security
+Management Center (SMC).  In the SMC configure the logs to be
+forwarded to the address set in `var.syslog_host` in format CEF and
+service UDP on `var.syslog_port`.  Instructions can be found in
+https://support.forcepoint.com/KBArticle?id=000015002[KB 15002] for
+configuring the SMC.  Testing was done with CEF logs from SMC version
+6.6.1 and custom string mappings were taken from 'CEF Connector
+Configuration Guide' dated December 5, 2011.
+
+
 :has-dashboards!:
 
 :fileset_ex!:
 
 :modulename!:
 
 
+
 [float]
 === Fields
 

x-pack/filebeat/module/cef/_meta/docs.asciidoc

@@ -35,8 +35,22 @@ The UDP port to listen for syslog traffic. Defaults to `9003`
 
 NOTE: Ports below 1024 require Filebeat to run as root.
 
+[float]
+==== Forcepoint NGFW Security Management Center
+
+This module will process CEF data from Forcepoint NGFW Security
+Management Center (SMC).  In the SMC configure the logs to be
+forwarded to the address set in `var.syslog_host` in format CEF and
+service UDP on `var.syslog_port`.  Instructions can be found in
+https://support.forcepoint.com/KBArticle?id=000015002[KB 15002] for
+configuring the SMC.  Testing was done with CEF logs from SMC version
+6.6.1 and custom string mappings were taken from 'CEF Connector
+Configuration Guide' dated December 5, 2011.
+
+
 :has-dashboards!:
 
 :fileset_ex!:
 
 :modulename!:
+

x-pack/filebeat/module/cef/_meta/fields.yml

@@ -1,6 +1,7 @@
 - key: cef-module
   title: CEF
   description: >
-    Module for receiving CEF logs over Syslog. The module does not add fields
-    beyond what the decode_cef processor provides.
+    Module for receiving CEF logs over Syslog. The module adds vendor
+    specific fields in addition to the fields the decode_cef processor
+    provides.
   fields:

x-pack/filebeat/module/cef/log/_meta/fields.yml

@@ -0,0 +1,10 @@
+- name: forcepoint
+  type: group
+  default_field: false
+  description: >
+    Fields for Forcepoint Custom String mappings
+  fields:
+    - name: virus_id
+      type: keyword
+      description: >
+        Virus ID

x-pack/filebeat/module/cef/log/ingest/fp-pipeline.yml

@@ -0,0 +1,27 @@
+---
+description: Pipeline for Forcepoint CEF
+
+processors:
+  # cs1 is ruleID
+  - set:
+      field: rule.id
+      value: "{{cef.extensions.deviceCustomString1}}"
+      if: "ctx.cef?.extensions?.deviceCustomString1 != null"
+
+  # cs2 is natRuleID
+  - set:
+      field: rule.id
+      value: "{{cef.extensions.deviceCustomString2}}"
+      if: "ctx.cef?.extensions?.deviceCustomString2 != null"
+
+  # cs3 is VulnerabilityReference
+  - set:
+      field: vulnerability.reference
+      value: "{{cef.extensions.deviceCustomString3}}"
+      if: "ctx.cef?.extensions?.deviceCustomString3 != null"
+
+  # cs4 is virusID
+  - set:
+      field: cef.forcepoint.virus_id
+      value: "{{cef.extensions.deviceCustomString4}}"
+      if: "ctx.cef?.extensions?.deviceCustomString4 != null"            

x-pack/filebeat/module/cef/log/ingest/pipeline.yml

@@ -45,7 +45,9 @@ processors:
         field: destination.as.organization_name
         target_field: destination.as.organization.name
         ignore_missing: true
-
+  - pipeline:
+        name: '{< IngestPipeline "fp-pipeline" >}'
+        if: "ctx.cef?.device?.vendor == 'FORCEPOINT'"
 on_failure:
   - set:
         field: error.message

x-pack/filebeat/module/cef/log/manifest.yml

@@ -13,7 +13,10 @@ var:
   - name: input
     default: syslog
 
-ingest_pipeline: ingest/pipeline.yml
+ingest_pipeline:
+  - ingest/pipeline.yml
+  - ingest/fp-pipeline.yml
+
 input: config/input.yml
 
 requires.processors:

x-pack/filebeat/module/cef/log/test/fp-ngfw-smc.log

@@ -0,0 +1,13 @@
+CEF:0|FORCEPOINT|Firewall|6.6.1|0|Generic|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=log server connection established deviceFacility=Logging System rt=Jan 17 2020 08:52:10
+CEF:0|FORCEPOINT|Firewall|6.6.1|9005|FW_Communication-Communication-Error|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=Communication error: No route to host (-3, 5, 0) deviceFacility=Management rt=Jan 17 2020 08:52:09
+CEF:0|FORCEPOINT|Firewall|6.6.1|70018|Connection_Allowed|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 src=10.37.205.252 dst=10.1.1.40 proto=1 deviceOutboundInterface=255 act=Allow msg=Referred connection: 10.1.1.40 -> 10.37.133.35 frag\=0x4000 TCP 47413->3020 deviceFacility=Packet Filtering rt=Jan 17 2020 08:52:09 app=Dest. Unreachable (Host Unreachable) cs1Label=RuleID cs1=2097157.1
+CEF:0|FORCEPOINT|Firewall|unknown|70019|Connection_Discarded|0|deviceExternalId=Firewall-10 node 1 dvc=10.1.1.10 dvchost=10.1.1.10 src=172.16.1.1 dst=255.255.255.255 spt=68 dpt=67 proto=17 deviceOutboundInterface=255 deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:21 app=BOOTPS (UDP) cs1Label=RuleID cs1=605.0
+CEF:0|FORCEPOINT|Firewall|unknown|70020|Connection_Refused|0|deviceExternalId=Firewall-1 node 1 dvc=10.1.1.1 dvchost=10.1.1.1 src=172.16.1.1 dst=192.168.1.1 proto=1 deviceOutboundInterface=255 act=Refuse deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:23 app=Echo Request (No Code) cs1Label=RuleID cs1=601.0
+CEF:0|FORCEPOINT|Firewall|unknown|70021|Connection_Closed|0|deviceExternalId=Firewall-6 node 1 dvc=10.1.1.6 dvchost=10.1.1.6 proto=6 deviceOutboundInterface=255 destinationServiceName=YouTube suser=alice deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:20 app=TCP in=32526 out=27366
+CEF:0|FORCEPOINT|Firewall|unknown|72714|ECA_Metadata_login|0|deviceExternalId=Firewall-3 node 1 dvc=10.1.1.3 dvchost=10.1.1.3 src=192.168.1.1 suser=bob deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:33
+CEF:0|FORCEPOINT|Firewall|unknown|72715|ECA_Metadata_logout|0|deviceExternalId=Firewall-10 node 1 dvc=10.1.1.10 dvchost=10.1.1.10 src=192.168.1.1 suser=bob deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:31
+CEF:0|FORCEPOINT|Firewall|unknown|72716|ECA_Metadata_system_metadata_received|0|deviceExternalId=Firewall-8 node 1 dvc=10.1.1.8 dvchost=10.1.1.8 src=172.16.2.1 suser=alice deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:26
+CEF:0|FORCEPOINT|Firewall|6.6.1|78002|TLS connection state|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=TLS: Couldn't establish TLS connection (11, N/A) deviceFacility=Management rt=Jan 17 2020 08:52:09
+
+
+