Cisco asa/ftd: Remove _temp_ fields on failure (elastic#18392) (elast…

…ic#18476)

The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 9c3faed)

CHANGELOG.next.asciidoc

@@ -127,6 +127,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Fixing `ingress_controller.` fields to be of type keyword instead of text. {issue}17834[17834]
 - Fixed typo in log message. {pull}17897[17897]
 - Unescape file name from SQS message. {pull}18370[18370]
+- Improve cisco asa and ftd pipelines' failure handler to avoid mapping temporary fields. {issue}18391[18391] {pull}18392[18392]
 
 *Heartbeat*
 

x-pack/filebeat/module/cisco/shared/ingest/asa-ftd-pipeline.yml

@@ -1275,6 +1275,16 @@ processors:
       ignore_missing: true
 
 on_failure:
+  # Copy any fields under _temp_.cisco to its final destination. Those can help
+  # with diagnosing the failure.
+  - rename:
+      field: _temp_.cisco
+      target_field: "cisco.{< .internal_prefix >}"
+      ignore_failure: true
+  # Remove _temp_ to avoid adding a lot of unnecessary fields to the index.
+  - remove:
+      field: _temp_
+      ignore_missing: true
   - append:
       field: "error.message"
       value: "{{ _ingest.on_failure_message }}"