Adding cluster_stats metricset for elasticsearch module

CHANGELOG.asciidoc

@@ -320,6 +320,7 @@ https://github.com/elastic/beats/compare/v6.2.3...master[Check the HEAD diff]
 - Release munin and traefik module as beta. {pull}7660[7660]
 - Add envoyproxy module. {pull}7569[7569]
 - Release prometheus collector metricset as GA. {pull}7660[7660]
+- Add Elasticsearch `cluster_stats` metricset. {pull}7638[7638]
 
 *Packetbeat*
 

metricbeat/docs/fields.asciidoc

@@ -3450,6 +3450,114 @@ type: keyword
 Elasticsearch state id.
 
 
+--
+
+[float]
+== cluster.stats fields
+
+Cluster stats
+
+
+
+*`elasticsearch.cluster.stats.status`*::
++
+--
+type: keyword
+
+Cluster status (green, yellow, red).
+
+
+--
+
+[float]
+== nodes fields
+
+Nodes statistics.
+
+
+
+*`elasticsearch.cluster.stats.nodes.count`*::
++
+--
+type: long
+
+Total number of nodes in cluster.
+
+
+--
+
+*`elasticsearch.cluster.stats.nodes.master`*::
++
+--
+type: long
+
+Number of master-eligible nodes in cluster.
+
+
+--
+
+*`elasticsearch.cluster.stats.nodes.data`*::
++
+--
+type: long
+
+Number of data nodes in cluster.
+
+
+--
+
+[float]
+== indices fields
+
+Indices statistics.
+
+
+
+*`elasticsearch.cluster.stats.indices.count`*::
++
+--
+type: long
+
+Total number of indices in cluster.
+
+
+--
+
+[float]
+== shards fields
+
+Shard statistics.
+
+
+
+*`elasticsearch.cluster.stats.indices.shards.count`*::
++
+--
+type: long
+
+Total number of shards in cluster.
+
+
+--
+
+*`elasticsearch.cluster.stats.indices.shards.primaries`*::
++
+--
+type: long
+
+Total number of primary shards in cluster.
+
+
+--
+
+*`elasticsearch.cluster.stats.indices.fielddata.memory.bytes`*::
++
+--
+type: long
+
+Memory used for fielddata.
+
+
 --
 
 [float]

metricbeat/docs/modules/elasticsearch.asciidoc

@@ -44,6 +44,8 @@ This module supports TLS connection when using `ssl` config field, as described
 
 The following metricsets are available:
 
+* <<metricbeat-metricset-elasticsearch-cluster_stats,cluster_stats>>
+
 * <<metricbeat-metricset-elasticsearch-index,index>>
 
 * <<metricbeat-metricset-elasticsearch-index_recovery,index_recovery>>
@@ -60,6 +62,8 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-elasticsearch-shard,shard>>
 
+include::elasticsearch/cluster_stats.asciidoc[]
+
 include::elasticsearch/index.asciidoc[]
 
 include::elasticsearch/index_recovery.asciidoc[]

metricbeat/docs/modules/elasticsearch/cluster_stats.asciidoc

@@ -0,0 +1,23 @@
+////
+This file is generated! See scripts/docs_collector.py
+////
+
+[[metricbeat-metricset-elasticsearch-cluster_stats]]
+=== Elasticsearch cluster_stats metricset
+
+beta[]
+
+include::../../../module/elasticsearch/cluster_stats/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-elasticsearch,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/elasticsearch/cluster_stats/_meta/data.json[]
+----

metricbeat/docs/modules_list.asciidoc

@@ -33,7 +33,8 @@ This file is generated! See scripts/docs_collector.py
 |<<metricbeat-module-dropwizard,Dropwizard>>  beta[]   |image:./images/icon-no.png[No prebuilt dashboards]    |  
 .1+| .1+|  |<<metricbeat-metricset-dropwizard-collector,collector>> beta[]  
 |<<metricbeat-module-elasticsearch,Elasticsearch>>  beta[]   |image:./images/icon-no.png[No prebuilt dashboards]    |  
-.8+| .8+|  |<<metricbeat-metricset-elasticsearch-index,index>> beta[]  
+.9+| .9+|  |<<metricbeat-metricset-elasticsearch-cluster_stats,cluster_stats>> beta[]  
+|<<metricbeat-metricset-elasticsearch-index,index>> beta[]  
 |<<metricbeat-metricset-elasticsearch-index_recovery,index_recovery>> beta[]  
 |<<metricbeat-metricset-elasticsearch-index_summary,index_summary>> beta[]  
 |<<metricbeat-metricset-elasticsearch-ml_job,ml_job>> beta[]  

metricbeat/module/elasticsearch/cluster_stats/_meta/data.json

@@ -0,0 +1,43 @@
+{
+    "@timestamp": "2017-10-12T08:05:34.853Z",
+    "beat": {
+        "hostname": "host.example.com",
+        "name": "host.example.com"
+    },
+    "elasticsearch": {
+        "cluster": {
+            "id": "6UTQ_iuNSzWP49zv99vxDg",
+            "name": "elasticsearch",
+            "stats": {
+                "indices": {
+                    "fielddata": {
+                        "memory": {
+                            "bytes": 1208
+                        }
+                    },
+                    "shards": {
+                        "count": 18,
+                        "primaries": 18
+                    },
+                    "total": 18
+                },
+                "nodes": {
+                    "count": 1,
+                    "data": 1,
+                    "master": 1
+                },
+                "status": "yellow"
+            }
+        }
+    },
+    "metricset": {
+        "host": "127.0.0.1:9200",
+        "module": "elasticsearch",
+        "name": "cluster_stats",
+        "namespace": "elasticsearch.cluster.stats",
+        "rtt": 115
+    },
+    "service": {
+        "name": "elasticsearch"
+    }
+}

metricbeat/module/elasticsearch/cluster_stats/_meta/docs.asciidoc

@@ -0,0 +1,3 @@
+This is the `cluster_stats` metricset of the Elasticsearch module. It interrogates the
+https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-stats.html[Cluster Stats API endpoint]
+to fetch information about the Elasticsearch cluster.

metricbeat/module/elasticsearch/cluster_stats/_meta/fields.yml

@@ -0,0 +1,53 @@
+- name: cluster.stats
+  type: group
+  description: >
+    Cluster stats
+  release: beta
+  fields:
+    - name: status
+      type: keyword
+      description: >
+        Cluster status (green, yellow, red).
+    - name: nodes
+      type: group
+      description: >
+        Nodes statistics.
+      fields:
+      - name: count
+        type: long
+        description: >
+          Total number of nodes in cluster.
+      - name: master
+        type: long
+        description: >
+          Number of master-eligible nodes in cluster.
+      - name: data
+        type: long
+        description: >
+          Number of data nodes in cluster.
+    - name: indices
+      type: group
+      description: >
+        Indices statistics.
+      fields:
+      - name: count
+        type: long
+        description: >
+          Total number of indices in cluster.
+      - name: shards
+        type: group
+        description: >
+          Shard statistics.
+        fields:
+        - name: count
+          type: long
+          description: >
+            Total number of shards in cluster.
+        - name: primaries
+          type: long
+          description: >
+            Total number of primary shards in cluster.
+      - name: fielddata.memory.bytes
+        type: long
+        description: >
+          Memory used for fielddata.