[vSphere] Add new Cluster metricset (#40536)

* add cluster metricset * update changelog * fix lint errors * minor changes * add unit test * update fields.yml * mage update * minor changes * minor changes * mage update * Address review comment * Address review comments * address review comments * update data.json * update fields.yml * update data.json * lint changes * mage update * mage update for x-pack * Mage update after conflicts * update fields.go * Address review comments --------- Co-authored-by: Niraj Rathod <niraj.rathod@crestdatasys.com> Co-authored-by: subham sarkar <subham.sarkar@elastic.co>
elastic · Aug 31, 2024 · 1058457 · 1058457
1 parent 441c166
commit 1058457
Show file tree

Hide file tree

Showing 26 changed files with 755 additions and 116 deletions.
diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc
@@ -293,6 +293,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Improve logging in Okta Entity Analytics provider. {issue}40106[40106] {pull}40347[40347]
 - Document `winlog` input. {issue}40074[40074] {pull}40462[40462]
 - Added retry logic to websocket connections in the streaming input. {issue}40271[40271] {pull}40601[40601]
+- Add new metricset cluster for the vSphere module. {pull}40536[40536]
 - Disable event normalization for netflow input {pull}40635[40635]
 - Allow attribute selection in the Active Directory entity analytics provider. {issue}40482[40482] {pull}40662[40662]
 

diff --git a/metricbeat/docs/fields.asciidoc b/metricbeat/docs/fields.asciidoc
@@ -66854,6 +66854,107 @@ vSphere module
 
 
 
+[float]
+=== cluster
+
+Cluster information.
+
+
+
+
+*`vsphere.cluster.datastore.names`*::
++
+--
+List of all the Datastore names associated with the cluster.
+
+
+type: keyword
+
+--
+
+*`vsphere.cluster.datastore.count`*::
++
+--
+Number of Datastores associated with the cluster.
+
+
+type: long
+
+--
+
+
+*`vsphere.cluster.das_config.admission.control.enabled`*::
++
+--
+Indicates whether strict admission control is enabled.
+
+
+type: boolean
+
+--
+
+*`vsphere.cluster.das_config.enabled`*::
++
+--
+Indicates whether vSphere HA feature is enabled.
+
+
+type: boolean
+
+--
+
+
+*`vsphere.cluster.host.count`*::
++
+--
+Number of Hosts associated with the cluster.
+
+
+type: long
+
+--
+
+*`vsphere.cluster.host.names`*::
++
+--
+List of all the Host names associated with the cluster.
+
+
+type: keyword
+
+--
+
+*`vsphere.cluster.name`*::
++
+--
+Cluster name.
+
+
+type: keyword
+
+--
+
+
+*`vsphere.cluster.network.count`*::
++
+--
+Number of Networks associated with the cluster.
+
+
+type: long
+
+--
+
+*`vsphere.cluster.network.names`*::
++
+--
+List of all the Network names associated with the cluster.
+
+
+type: keyword
+
+--
+
 [float]
 === datastore
 

diff --git a/metricbeat/docs/modules/vsphere.asciidoc b/metricbeat/docs/modules/vsphere.asciidoc
@@ -11,7 +11,7 @@ This file is generated! See scripts/mage/docs_collector.go
 
 The vSphere module uses the https://github.com/vmware/govmomi[Govmomi] library to collect metrics from any Vmware SDK URL (ESXi/VCenter). This library is built for and tested against ESXi and vCenter 5.5, 6.0 and 6.5.
 
-By default it enables the metricsets `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
+By default it enables the metricsets `cluster`, `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
 
 [float]
 === Dashboard
@@ -35,7 +35,7 @@ in <<configuration-metricbeat>>. Here is an example configuration:
 metricbeat.modules:
 - module: vsphere
   enabled: true
-  metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
+  metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
   # Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
   period: 20s
   hosts: ["https://localhost/sdk"]
@@ -53,6 +53,8 @@ metricbeat.modules:
 
 The following metricsets are available:
 
+* <<metricbeat-metricset-vsphere-cluster,cluster>>
+
 * <<metricbeat-metricset-vsphere-datastore,datastore>>
 
 * <<metricbeat-metricset-vsphere-host,host>>
@@ -63,6 +65,8 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-vsphere-virtualmachine,virtualmachine>>
 
+include::vsphere/cluster.asciidoc[]
+
 include::vsphere/datastore.asciidoc[]
 
 include::vsphere/host.asciidoc[]

diff --git a/metricbeat/docs/modules/vsphere/cluster.asciidoc b/metricbeat/docs/modules/vsphere/cluster.asciidoc
@@ -0,0 +1,29 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/vsphere/cluster/_meta/docs.asciidoc
+
+
+[[metricbeat-metricset-vsphere-cluster]]
+=== vSphere cluster metricset
+
+beta[]
+
+include::../../../module/vsphere/cluster/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+:edit_url:
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-vsphere,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/vsphere/cluster/_meta/data.json[]
+----
+:edit_url!:
diff --git a/metricbeat/docs/modules_list.asciidoc b/metricbeat/docs/modules_list.asciidoc
@@ -311,7 +311,8 @@ This file is generated! See scripts/mage/docs_collector.go
 |<<metricbeat-module-uwsgi,uWSGI>>     |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
 .1+| .1+|  |<<metricbeat-metricset-uwsgi-status,status>>   
 |<<metricbeat-module-vsphere,vSphere>>     |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
-.5+| .5+|  |<<metricbeat-metricset-vsphere-datastore,datastore>>   
+.6+| .6+|  |<<metricbeat-metricset-vsphere-cluster,cluster>> beta[]  
+|<<metricbeat-metricset-vsphere-datastore,datastore>>   
 |<<metricbeat-metricset-vsphere-host,host>>   
 |<<metricbeat-metricset-vsphere-network,network>> beta[]  
 |<<metricbeat-metricset-vsphere-resourcepool,resourcepool>> beta[]  

diff --git a/metricbeat/include/list_common.go b/metricbeat/include/list_common.go
diff --git a/metricbeat/metricbeat.reference.yml b/metricbeat/metricbeat.reference.yml
@@ -1007,7 +1007,7 @@ metricbeat.modules:
 #------------------------------- VSphere Module -------------------------------
 - module: vsphere
   enabled: true
-  metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
+  metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
   # Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
   period: 20s
   hosts: ["https://localhost/sdk"]

diff --git a/metricbeat/module/vsphere/_meta/README.md b/metricbeat/module/vsphere/_meta/README.md
@@ -28,6 +28,7 @@ Now setup your metricbeat config to connect to Govcsim:
 ```
 - module: vsphere
   metricsets:
+    - cluster
     - datastore
     - host
     - virtualmachine

diff --git a/metricbeat/module/vsphere/_meta/config.reference.yml b/metricbeat/module/vsphere/_meta/config.reference.yml
@@ -1,6 +1,6 @@
 - module: vsphere
   enabled: true
-  metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
+  metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
   # Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
   period: 20s
   hosts: ["https://localhost/sdk"]

diff --git a/metricbeat/module/vsphere/_meta/config.yml b/metricbeat/module/vsphere/_meta/config.yml
@@ -1,5 +1,6 @@
 - module: vsphere
   #metricsets:
+  #  - cluster
   #  - datastore
   #  - host
   #  - virtualmachine

diff --git a/metricbeat/module/vsphere/_meta/docs.asciidoc b/metricbeat/module/vsphere/_meta/docs.asciidoc
@@ -1,6 +1,6 @@
 The vSphere module uses the https://github.com/vmware/govmomi[Govmomi] library to collect metrics from any Vmware SDK URL (ESXi/VCenter). This library is built for and tested against ESXi and vCenter 5.5, 6.0 and 6.5.
 
-By default it enables the metricsets `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
+By default it enables the metricsets `cluster`, `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
 
 [float]
 === Dashboard

diff --git a/metricbeat/module/vsphere/cluster/_meta/data.json b/metricbeat/module/vsphere/cluster/_meta/data.json
@@ -0,0 +1,47 @@
+{
+    "@timestamp": "2017-10-12T08:05:34.853Z",
+    "event": {
+        "dataset": "vsphere.cluster",
+        "duration": 115000,
+        "module": "vsphere"
+    },
+    "metricset": {
+        "name": "cluster",
+        "period": 10000
+    },
+    "service": {
+        "address": "127.0.0.1:33365",
+        "type": "vsphere"
+    },
+    "vsphere": {
+      "cluster": {
+        "name": "Cluster_1",
+        "das_config": {
+          "enabled": false,
+          "admission": {
+            "control": {
+              "enabled": true
+            }
+          }
+        },
+        "host": {
+          "count": 1,
+          "names": [
+            "Host_1"
+          ]
+        },
+        "datastore": {
+          "count": 1,
+          "names": [
+            "Datastore_1"
+          ]
+        },
+        "network": {
+          "count": 1,
+          "names": [
+            "Network_1"
+          ]
+        }
+      }
+    }
+}
diff --git a/metricbeat/module/vsphere/cluster/_meta/docs.asciidoc b/metricbeat/module/vsphere/cluster/_meta/docs.asciidoc
@@ -0,0 +1 @@
+This is the `cluster` metricset of the vSphere module.
diff --git a/metricbeat/module/vsphere/cluster/_meta/fields.yml b/metricbeat/module/vsphere/cluster/_meta/fields.yml
@@ -0,0 +1,54 @@
+- name: cluster
+  type: group
+  release: beta
+  description: >
+    Cluster information.
+  fields:
+    - name: datastore
+      type: group
+      fields:
+        - name: names
+          type: keyword
+          description: >
+            List of all the Datastore names associated with the cluster.
+        - name: count
+          type: long
+          description: >
+            Number of Datastores associated with the cluster.
+    - name: das_config
+      type: group
+      fields:
+        - name: admission.control.enabled
+          type: boolean
+          description: >
+            Indicates whether strict admission control is enabled.
+        - name: enabled
+          type: boolean
+          description: >
+            Indicates whether vSphere HA feature is enabled.
+    - name: host
+      type: group
+      fields:
+        - name: count
+          type: long
+          description: >
+            Number of Hosts associated with the cluster.
+        - name: names
+          type: keyword
+          description: >
+            List of all the Host names associated with the cluster.
+    - name: name
+      type: keyword
+      description: >
+        Cluster name.
+    - name: network
+      type: group
+      fields:
+        - name: count
+          type: long
+          description: >
+            Number of Networks associated with the cluster.
+        - name: names
+          type: keyword
+          description: >
+            List of all the Network names associated with the cluster.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		This is the `cluster` metricset of the vSphere module.