Migrate obs-infraobs-integrations to package-spec v3 #2 (#8171)

* [hadoop]: migration with yq * [haproxy]: migration with yq * [ibmmq]: migration with yq * [iis]: migration with yq * [influxdb]: migration with yq * [kafka]: migration with yq * [kafka_log]: migration with yq * [memcached]: migration with yq * [microsoft_sqlserver]: migration with yq * [mongodb]: migration with yq * [mysql]: migration with yq * [nagios_xi]: migration with yq * [nginx]: migration with yq * [golang] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 2.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/golang * [hadoop] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/hadoop * [jolokia] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 2.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/jolokia_input * [kafka_log] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/kafka_log * [memcached] - change to format_version 3.0.0 The format_version in the package manifest changed from 1.0.0 to 3.0.0. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/memcached * [iis] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/iis * [influxdb] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/influxdb * [kafka] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/kafka * [microsoft_sqlserver]: migration with yq * [microsoft_sqlserver] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/microsoft_sqlserver * [mongodb] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/mongodb * [mysql] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/mysql * [nginx] - removed dotted YAML keys from manifest The format_version in the package manifest changed from 1.0.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest. [git-generate] go run github.com/andrewkroh/go-examples/ecs-update@latest -v -format-version=3.0.0 -skip-format -fix-dotted-yaml-keys -add-owner-type packages/nginx * Add validation.yml * Format packages and quote fields where necessary * revert changes for packages for which v3 migration failed * [hadoop,haproxy,iis,mongodb]: remove duplicates from agent.yml instead of ecs.yml * Fix PR number in changelog * Fix inconsistency w/ elastic-build * [mongodb]: Add dimension field which was removed during dedup of fields * quote fields where necessary * Add release field to data stream's manifest if removed from fields * Quote index.mapping.dimension_fields.limit to avoid getting changed by ep format * Revert "Quote index.mapping.dimension_fields.limit to avoid getting changed by ep format" This reverts commit 3ad6d2f. * Add comment * [mongodb]: revert changes package-spec v3 migration * [influxdb]: revert changes package-spec v3 migration * remove validation.yml for package which are not migrated to v3 * remove comment * [hadoop]: update definition for tags * remove release: beta
elastic · Oct 19, 2023 · 6445a75 · 6445a75
1 parent 0ac9fd2
commit 6445a75
Show file tree

Hide file tree

Showing 122 changed files with 1,669 additions and 1,080 deletions.
diff --git a/packages/golang/changelog.yml b/packages/golang/changelog.yml
@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: 1.1.0
+  changes:
+    - description: Update the package format_version to 3.0.0.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/8171
 - version: "1.0.0"
   changes:
     - description: Make Golang GA.

diff --git a/packages/golang/manifest.yml b/packages/golang/manifest.yml
@@ -1,14 +1,16 @@
-format_version: 2.0.0
+format_version: "3.0.0"
 name: golang
 title: Golang
-version: "1.0.0"
+version: "1.1.0"
 description: This Elastic integration collects metrics from Golang applications.
 type: integration
 categories:
   - observability
 conditions:
-  kibana.version: ^8.7.1
-  elastic.subscription: basic
+  kibana:
+    version: ^8.7.1
+  elastic:
+    subscription: basic
 screenshots:
   - src: /img/golang-heap-dashboard.png
     title: Golang Heap dashboard
@@ -96,3 +98,4 @@ policy_templates:
               #    -----END CERTIFICATE-----
 owner:
   github: elastic/obs-infraobs-integrations
+  type: elastic
diff --git a/packages/golang/validation.yml b/packages/golang/validation.yml
@@ -0,0 +1,3 @@
+errors:
+  exclude_checks:
+    - SVR00002
diff --git a/packages/hadoop/changelog.yml b/packages/hadoop/changelog.yml
@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: 1.3.0
+  changes:
+    - description: Update the package format_version to 3.0.0.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/8171
 - version: "1.2.1"
   changes:
     - description: Remove forwarded tag from metrics data streams.

diff --git a/...oop/data_stream/application/_dev/test/pipeline/test-application-metrics.log-expected.json b/...oop/data_stream/application/_dev/test/pipeline/test-application-metrics.log-expected.json
@@ -5,11 +5,15 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
                 "module": "httpjson",
                 "original": "{\"allocatedMB\":-1,\"allocatedVCores\":-1,\"amContainerLogs\":\"http://localhost:8042/node/containerlogs/container_1646029193591_0003_01_000001/bhagyaraj\",\"amHostHttpAddress\":\"localhost:8042\",\"amNodeLabelExpression\":\"\",\"amRPCAddress\":\"localhost:38279\",\"applicationTags\":\"\",\"applicationType\":\"MAPREDUCE\",\"clusterId\":1646029193591,\"clusterUsagePercentage\":0,\"diagnostics\":\"\",\"elapsedTime\":35114,\"finalStatus\":\"SUCCEEDED\",\"finishedTime\":1646030395258,\"id\":\"application_1646029193591_0003\",\"launchTime\":1646030360297,\"logAggregationStatus\":\"DISABLED\",\"masterNodeId\":\"localhost:45960\",\"memorySeconds\":103467,\"name\":\"wordcount\",\"numAMContainerPreempted\":0,\"numNonAMContainerPreempted\":0,\"preemptedMemorySeconds\":0,\"preemptedResourceMB\":0,\"preemptedResourceSecondsMap\":{},\"preemptedResourceVCores\":0,\"preemptedVcoreSeconds\":0,\"priority\":0,\"progress\":100,\"queue\":\"default\",\"queueUsagePercentage\":0,\"reservedMB\":-1,\"reservedVCores\":-1,\"resourceSecondsMap\":{\"entry\":{\"key\":\"vcores\",\"value\":\"57\"}},\"runningContainers\":-1,\"startedTime\":1646030360144,\"state\":\"FINISHED\",\"timeouts\":{\"timeout\":[{\"expiryTime\":\"UNLIMITED\",\"remainingTimeInSeconds\":-1,\"type\":\"LIFETIME\"}]},\"trackingUI\":\"History\",\"trackingUrl\":\"http://localhost:8088/proxy/application_1646029193591_0003/\",\"unmanagedApplication\":false,\"user\":\"bhagyaraj\",\"vcoreSeconds\":57}",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "application": {
@@ -38,11 +42,15 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
                 "module": "httpjson",
                 "original": "{\"allocatedMB\":-1,\"allocatedVCores\":-1,\"amContainerLogs\":\"http://localhost:8042/node/containerlogs/container_1646029193591_0001_02_000001/bhagyaraj\",\"amHostHttpAddress\":\"localhost:8042\",\"amNodeLabelExpression\":\"\",\"applicationTags\":\"\",\"applicationType\":\"MAPREDUCE\",\"clusterId\":1646029193591,\"clusterUsagePercentage\":0,\"diagnostics\":\"Application application_1646029193591_0001 failed 2 times due to AM Container for appattempt_1646029193591_0001_000002 exited with  exitCode: 1\\\\nFailing this attempt.Diagnostics: [2022-02-28 12:03:43.814]Exception from container-launch.\\\\nContainer id: container_1646029193591_0001_02_000001\\\\nExit code: 1\\\\n\\\\n[2022-02-28 12:03:43.822]Container exited with a non-zero exit code 1. Error file: prelaunch.err.\\\\nLast 4096 bytes of prelaunch.err :\\\\nLast 4096 bytes of stderr :\\\\nError: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster\\\\nCaused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster\\\\n\\\\nPlease check whether your \\\\u003cHADOOP_HOME\\\\u003e/etc/hadoop/mapred-site.xml contains the below configuration:\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003eyarn.app.mapreduce.am.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003emapreduce.map.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003emapreduce.reduce.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\n[2022-02-28 12:03:43.823]Container exited with a non-zero exit code 1. Error file: prelaunch.err.\\\\nLast 4096 bytes of prelaunch.err :\\\\nLast 4096 bytes of stderr :\\\\nError: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster\\\\nCaused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster\\\\n\\\\nPlease check whether your \\\\u003cHADOOP_HOME\\\\u003e/etc/hadoop/mapred-site.xml contains the below configuration:\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003eyarn.app.mapreduce.am.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003emapreduce.map.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\u003cproperty\\\\u003e\\\\n  \\\\u003cname\\\\u003emapreduce.reduce.env\\\\u003c/name\\\\u003e\\\\n  \\\\u003cvalue\\\\u003eHADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}\\\\u003c/value\\\\u003e\\\\n\\\\u003c/property\\\\u003e\\\\n\\\\nFor more detailed output, check the application tracking page: http://localhost:8088/cluster/app/application_1646029193591_0001 Then click on links to logs of each attempt.\\\\n. Failing the application.\",\"elapsedTime\":9925,\"finalStatus\":\"FAILED\",\"finishedTime\":1646030023835,\"id\":\"application_1646029193591_0001\",\"launchTime\":1646030016899,\"logAggregationStatus\":\"DISABLED\",\"masterNodeId\":\"localhost:45960\",\"memorySeconds\":15708,\"name\":\"wordcount\",\"numAMContainerPreempted\":0,\"numNonAMContainerPreempted\":0,\"preemptedMemorySeconds\":0,\"preemptedResourceMB\":0,\"preemptedResourceSecondsMap\":{},\"preemptedResourceVCores\":0,\"preemptedVcoreSeconds\":0,\"priority\":0,\"progress\":0,\"queue\":\"default\",\"queueUsagePercentage\":0,\"reservedMB\":-1,\"reservedVCores\":-1,\"resourceSecondsMap\":{\"entry\":{\"key\":\"vcores\",\"value\":\"7\"}},\"runningContainers\":-1,\"startedTime\":1646030013910,\"state\":\"FAILED\",\"timeouts\":{\"timeout\":[{\"expiryTime\":\"UNLIMITED\",\"remainingTimeInSeconds\":-1,\"type\":\"LIFETIME\"}]},\"trackingUI\":\"History\",\"trackingUrl\":\"http://localhost:8088/cluster/app/application_1646029193591_0001\",\"unmanagedApplication\":false,\"user\":\"bhagyaraj\",\"vcoreSeconds\":7}",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "application": {

diff --git a/packages/hadoop/data_stream/application/_dev/test/pipeline/test-common-config.yml b/packages/hadoop/data_stream/application/_dev/test/pipeline/test-common-config.yml
@@ -1,5 +1,5 @@
 dynamic_fields:
-  event.ingested: ".*"
+  "event.ingested": ".*"
 fields:
   tags:
     - preserve_original_event
diff --git a/packages/hadoop/data_stream/application/elasticsearch/ingest_pipeline/default.yml b/packages/hadoop/data_stream/application/elasticsearch/ingest_pipeline/default.yml
@@ -6,7 +6,7 @@ processors:
       value: '8.5.1'
   - set:
       field: event.type
-      value: info
+      value: [info]
   - set:
       field: event.kind
       value: metric
@@ -15,7 +15,7 @@ processors:
       value: httpjson
   - set:
       field: event.category
-      value: database
+      value: [database]
   - rename:
       field: message
       target_field: event.original

diff --git a/packages/hadoop/data_stream/application/fields/beats.yml b/packages/hadoop/data_stream/application/fields/beats.yml
@@ -1,6 +1,3 @@
 - name: input.type
   description: Type of Filebeat input.
   type: keyword
-- name: tags
-  type: keyword
-  description: User defined tags
diff --git a/packages/hadoop/data_stream/application/fields/ecs.yml b/packages/hadoop/data_stream/application/fields/ecs.yml
@@ -14,3 +14,4 @@
   name: host.ip
 - external: ecs
   name: tags
+  description: User defined tags.
diff --git a/packages/hadoop/data_stream/application/sample_event.json b/packages/hadoop/data_stream/application/sample_event.json
@@ -22,13 +22,17 @@
     },
     "event": {
         "agent_id_status": "verified",
-        "category": "database",
+        "category": [
+            "database"
+        ],
         "created": "2023-02-02T12:03:41.178Z",
         "dataset": "hadoop.application",
         "ingested": "2023-02-02T12:03:42Z",
         "kind": "metric",
         "module": "httpjson",
-        "type": "info"
+        "type": [
+            "info"
+        ]
     },
     "hadoop": {
         "application": {

diff --git a/...ges/hadoop/data_stream/cluster/_dev/test/pipeline/test-cluster-metrics.json-expected.json b/...ges/hadoop/data_stream/cluster/_dev/test/pipeline/test-cluster-metrics.json-expected.json
@@ -6,9 +6,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "cluster": {
@@ -34,9 +38,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "cluster": {

diff --git a/packages/hadoop/data_stream/cluster/elasticsearch/ingest_pipeline/default.yml b/packages/hadoop/data_stream/cluster/elasticsearch/ingest_pipeline/default.yml
@@ -6,13 +6,13 @@ processors:
       value: '8.5.1'
   - set:
       field: event.type
-      value: info
+      value: [info]
   - set:
       field: event.kind
       value: metric
   - set:
       field: event.category
-      value: database
+      value: [database]
   - rename:
       field: http
       target_field: hadoop

diff --git a/packages/hadoop/data_stream/cluster/fields/ecs.yml b/packages/hadoop/data_stream/cluster/fields/ecs.yml
@@ -42,4 +42,4 @@
   dimension: true
 - external: ecs
   name: host.name
-  dimension: true
+  dimension: true
diff --git a/packages/hadoop/data_stream/cluster/manifest.yml b/packages/hadoop/data_stream/cluster/manifest.yml
@@ -45,5 +45,6 @@ streams:
         show_user: false
         description: >
           Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
+
 elasticsearch:
   index_mode: "time_series"
diff --git a/packages/hadoop/data_stream/cluster/sample_event.json b/packages/hadoop/data_stream/cluster/sample_event.json
@@ -22,13 +22,17 @@
     },
     "event": {
         "agent_id_status": "verified",
-        "category": "database",
+        "category": [
+            "database"
+        ],
         "dataset": "hadoop.cluster",
         "duration": 50350559,
         "ingested": "2022-04-04T17:22:25Z",
         "kind": "metric",
         "module": "http",
-        "type": "info"
+        "type": [
+            "info"
+        ]
     },
     "hadoop": {
         "cluster": {

diff --git a/...s/hadoop/data_stream/datanode/_dev/test/pipeline/test-datanode-metrics.json-expected.json b/...s/hadoop/data_stream/datanode/_dev/test/pipeline/test-datanode-metrics.json-expected.json
@@ -6,9 +6,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "datanode": {
@@ -42,9 +46,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "datanode": {

diff --git a/packages/hadoop/data_stream/datanode/elasticsearch/ingest_pipeline/default.yml b/packages/hadoop/data_stream/datanode/elasticsearch/ingest_pipeline/default.yml
@@ -6,13 +6,13 @@ processors:
       value: '8.5.1'
   - set:
       field: event.type
-      value: info
+      value: [info]
   - set:
       field: event.kind
       value: metric
   - set:
       field: event.category
-      value: database
+      value: [database]
   - rename:
       field: http
       target_field: hadoop

diff --git a/packages/hadoop/data_stream/datanode/fields/ecs.yml b/packages/hadoop/data_stream/datanode/fields/ecs.yml
@@ -42,4 +42,4 @@
   dimension: true
 - external: ecs
   name: host.name
-  dimension: true
+  dimension: true
diff --git a/packages/hadoop/data_stream/datanode/manifest.yml b/packages/hadoop/data_stream/datanode/manifest.yml
@@ -45,5 +45,6 @@ streams:
         show_user: false
         description: >
           Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
+
 elasticsearch:
   index_mode: "time_series"
diff --git a/packages/hadoop/data_stream/datanode/sample_event.json b/packages/hadoop/data_stream/datanode/sample_event.json
@@ -22,13 +22,17 @@
     },
     "event": {
         "agent_id_status": "verified",
-        "category": "database",
+        "category": [
+            "database"
+        ],
         "dataset": "hadoop.datanode",
         "duration": 241651987,
         "ingested": "2023-02-02T12:05:05Z",
         "kind": "metric",
         "module": "http",
-        "type": "info"
+        "type": [
+            "info"
+        ]
     },
     "hadoop": {
         "datanode": {

diff --git a/...s/hadoop/data_stream/namenode/_dev/test/pipeline/test-namenode-metrics.json-expected.json b/...s/hadoop/data_stream/namenode/_dev/test/pipeline/test-namenode-metrics.json-expected.json
@@ -6,9 +6,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "namenode": {

diff --git a/packages/hadoop/data_stream/namenode/elasticsearch/ingest_pipeline/default.yml b/packages/hadoop/data_stream/namenode/elasticsearch/ingest_pipeline/default.yml
@@ -6,13 +6,13 @@ processors:
       value: '8.5.1'
   - set:
       field: event.type
-      value: info
+      value: [info]
   - set:
       field: event.kind
       value: metric
   - set:
       field: event.category
-      value: database
+      value: [database]
   - rename:
       field: http
       target_field: hadoop

diff --git a/packages/hadoop/data_stream/namenode/fields/ecs.yml b/packages/hadoop/data_stream/namenode/fields/ecs.yml
@@ -42,4 +42,4 @@
   dimension: true
 - external: ecs
   name: host.name
-  dimension: true
+  dimension: true
diff --git a/packages/hadoop/data_stream/namenode/fields/fields.yml b/packages/hadoop/data_stream/namenode/fields/fields.yml
@@ -1,6 +1,5 @@
 - name: hadoop.namenode
   type: group
-  release: beta
   fields:
     - name: blocks
       type: group

diff --git a/packages/hadoop/data_stream/namenode/manifest.yml b/packages/hadoop/data_stream/namenode/manifest.yml
@@ -45,5 +45,6 @@ streams:
         show_user: false
         description: >
           Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
+
 elasticsearch:
   index_mode: "time_series"
diff --git a/packages/hadoop/data_stream/namenode/sample_event.json b/packages/hadoop/data_stream/namenode/sample_event.json
@@ -22,13 +22,17 @@
     },
     "event": {
         "agent_id_status": "verified",
-        "category": "database",
+        "category": [
+            "database"
+        ],
         "dataset": "hadoop.namenode",
         "duration": 341259289,
         "ingested": "2022-03-28T11:36:09Z",
         "kind": "metric",
         "module": "http",
-        "type": "info"
+        "type": [
+            "info"
+        ]
     },
     "hadoop": {
         "namenode": {

diff --git a/.../data_stream/node_manager/_dev/test/pipeline/test-node-manager-metrics.json-expected.json b/.../data_stream/node_manager/_dev/test/pipeline/test-node-manager-metrics.json-expected.json
@@ -6,9 +6,13 @@
                 "version": "8.5.1"
             },
             "event": {
-                "category": "database",
+                "category": [
+                    "database"
+                ],
                 "kind": "metric",
-                "type": "info"
+                "type": [
+                    "info"
+                ]
             },
             "hadoop": {
                 "node_manager": {