Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server Metricset for Zookeeper Metricbeat module #10341

Merged
merged 10 commits into from
Jan 28, 2019
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. {pull}10222[10222]
- Add support for MySQL 8.0 and tests also for Percona and MariaDB. {pull}10261[10261]
- Rename 'db' Metricset to 'transaction_log' in MSSQL Metricbeat module {pull}10109[10109]
- Added 'server' Metricset to Zookeeper Metricbeat module {issue}8938[8938] {pull}10341[10341]

*Packetbeat*

Expand Down
124 changes: 124 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -25324,3 +25324,127 @@ Number of znodes reported by the local ZooKeeper process.

--

[float]
== server fields

server contains the metrics reported by the four-letter `srvr` command.


*`zookeeper.server.connections`*::
+
--
type: long

Connections established by the server

--


*`zookeeper.server.latency.avg`*::
+
--
type: long

Average latency of the server

--

*`zookeeper.server.latency.max`*::
+
--
type: long

Max latency reached by the server

--

*`zookeeper.server.latency.min`*::
+
--
type: long

Minimum latency that has been reached by the server

--

*`zookeeper.server.mode`*::
+
--
type: keyword

Server mode

--

*`zookeeper.server.node_count`*::
+
--
type: long

Total number of nodes

--

*`zookeeper.server.outstanding`*::
+
--
type: long

Outstanding

--

*`zookeeper.server.received`*::
+
--
type: long

Received requests to the server

--

*`zookeeper.server.sent`*::
+
--
type: long

Requests sent by the server

--

*`zookeeper.server.version_date`*::
+
--
type: date

Date of the Zookeeper release in use

--

*`zookeeper.server.zxid`*::
+
--
type: keyword

Original value of the Zookeeper transaction ID

--

*`zookeeper.server.count`*::
+
--
type: long

Total transactions of the leader in epoch

--

*`zookeeper.server.epoch`*::
+
--
type: long

Epoch value of the Zookeeper transaction ID

--

8 changes: 6 additions & 2 deletions metricbeat/docs/modules/zookeeper.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file is generated! See scripts/docs_collector.py
== ZooKeeper module

The ZooKeeper module fetches statistics from the ZooKeeper service. The default
metricset is `mntr`.
metricset is `mntr` and `server`.

[float]
=== Compatibility
Expand All @@ -26,7 +26,7 @@ in <<configuration-metricbeat>>. Here is an example configuration:
metricbeat.modules:
- module: zookeeper
enabled: true
metricsets: ["mntr"]
metricsets: ["mntr", "server"]
period: 10s
hosts: ["localhost:2181"]
----
Expand All @@ -38,5 +38,9 @@ The following metricsets are available:

* <<metricbeat-metricset-zookeeper-mntr,mntr>>

* <<metricbeat-metricset-zookeeper-server,server>>

include::zookeeper/mntr.asciidoc[]

include::zookeeper/server.asciidoc[]

21 changes: 21 additions & 0 deletions metricbeat/docs/modules/zookeeper/server.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
////
This file is generated! See scripts/docs_collector.py
////

[[metricbeat-metricset-zookeeper-server]]
=== ZooKeeper server metricset

include::../../../module/zookeeper/server/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-zookeeper,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../module/zookeeper/server/_meta/data.json[]
----
3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ This file is generated! See scripts/docs_collector.py
.2+| .2+| |<<metricbeat-metricset-windows-perfmon,perfmon>> beta[]
|<<metricbeat-metricset-windows-service,service>>
|<<metricbeat-module-zookeeper,ZooKeeper>> |image:./images/icon-no.png[No prebuilt dashboards] |
.1+| .1+| |<<metricbeat-metricset-zookeeper-mntr,mntr>>
.2+| .2+| |<<metricbeat-metricset-zookeeper-mntr,mntr>>
|<<metricbeat-metricset-zookeeper-server,server>>
|================================

--
Expand Down
1 change: 1 addition & 0 deletions metricbeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -689,7 +689,7 @@ metricbeat.modules:
#------------------------------ ZooKeeper Module -----------------------------
- module: zookeeper
enabled: true
metricsets: ["mntr"]
metricsets: ["mntr", "server"]
period: 10s
hosts: ["localhost:2181"]

Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/zookeeper/_meta/config.reference.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- module: zookeeper
enabled: true
metricsets: ["mntr"]
metricsets: ["mntr", "server"]
period: 10s
hosts: ["localhost:2181"]
1 change: 1 addition & 0 deletions metricbeat/module/zookeeper/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
- module: zookeeper
#metricsets:
# - mntr
# - server
period: 10s
hosts: ["localhost:2181"]
2 changes: 1 addition & 1 deletion metricbeat/module/zookeeper/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
The ZooKeeper module fetches statistics from the ZooKeeper service. The default
metricset is `mntr`.
metricset is `mntr` and `server`.

[float]
=== Compatibility
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/zookeeper/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 39 additions & 0 deletions metricbeat/module/zookeeper/server/_meta/data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"@timestamp": "2017-10-12T08:05:34.853Z",
"agent": {
"hostname": "host.example.com",
"name": "host.example.com"
},
"event": {
"dataset": "zookeeper.server",
"duration": 115000,
"module": "zookeeper"
},
"metricset": {
"name": "server"
},
"service": {
"address": "localhost:2181",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if this was also broken up as service.ip and service.port.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we would still keep service.address too I would assume. As ip does not match localhost for example.

BTW: This is not something related to this PR as it's done in the module logic. @webmat Could you open a separte issue for this? Nice thing here is we can always add this logic as it's not breaking but addition.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we'd keep service.address as the source of truth.

"type": "zookeeper",
"version": "3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03"
},
"zookeeper": {
"server": {
"connections": 1,
"count": 0,
"epoch": 0,
"latency": {
"avg": 0,
"max": 0,
"min": 0
},
"mode": "standalone",
"node_count": 4,
"outstanding": 0,
"received": 38,
"sent": 37,
"version_date": "06/29/2018 04:05 GMT",
"zxid": "0x0"
}
}
}
15 changes: 15 additions & 0 deletions metricbeat/module/zookeeper/server/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
`server` Metricset fetches the data returned by the `srvr` admin keyword.

* *connections*: Connections established by the server
* *latency.avg*: Average latency of the server
* *latency.max*: Max latency reached by the server
* *latency.min*: Minimum latency that has been reached by the server
* *mode*: Server mode
* *node_count*: Total number of nodes
* *outstanding*: Outstanding
* *received*: Received requests to the server
* *sent*: Requests sent by the server
* *version_date*: Date of the Zookeeper release in use
* *zxid*: Original value of the Zookeeper transaction ID
* *count*: Total transactions of the leader in epoch
* *epoch*: Epoch value of the Zookeeper transaction ID
47 changes: 47 additions & 0 deletions metricbeat/module/zookeeper/server/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
- name: server
type: group
description: 'server contains the metrics reported by the four-letter `srvr` command.'
release: ga
fields:
- name: connections
type: long
description: Connections established by the server
- name: latency
type: group
fields:
- name: avg
type: long
description: Average latency of the server
- name: max
type: long
description: Max latency reached by the server
- name: min
type: long
description: Minimum latency that has been reached by the server
- name: mode
type: keyword
description: Server mode
- name: node_count
type: long
description: Total number of nodes
- name: outstanding
type: long
description: Outstanding
- name: received
type: long
description: Received requests to the server
- name: sent
type: long
description: Requests sent by the server
- name: version_date
type: date
description: Date of the Zookeeper release in use
- name: zxid
type: keyword
description: Original value of the Zookeeper transaction ID
- name: count
type: long
description: Total transactions of the leader in epoch
- name: epoch
type: long
description: Epoch value of the Zookeeper transaction ID
Loading