Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ELU fields to metricbeat Kibana module. #35020

Merged
merged 4 commits into from
Apr 20, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Add beta ingest_pipeline metricset to Elasticsearch module for ingest pipeline monitoring {pull}34012[34012]
- Handle duplicated TYPE line for prometheus metrics {issue}18813[18813] {pull}33865[33865]
- Add GCP Carbon Footprint metricbeat data {pull}34820[34820]
- Add event loop utilization metric to Kibana module {pull}35020[35020]

*Packetbeat*

Expand Down
64 changes: 64 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -42193,6 +42193,33 @@ alias to: kibana.stats.process.event_loop_delay.ms

--

*`kibana_stats.process.event_loop_utilization.active`*::
+
--
type: alias

alias to: kibana.stats.process.event_loop_utilization.active

--

*`kibana_stats.process.event_loop_utilization.idle`*::
+
--
type: alias

alias to: kibana.stats.process.event_loop_utilization.idle

--

*`kibana_stats.process.event_loop_utilization.utilization`*::
+
--
type: alias

alias to: kibana.stats.process.event_loop_utilization.utilization

--

*`kibana_stats.requests.total`*::
+
--
Expand Down Expand Up @@ -42706,6 +42733,43 @@ type: long
Event loop delay in milliseconds


type: scaled_float

--

[float]
=== event_loop_utilization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The kibana module can either route data to metricbeat-* or .monitoring-kibana-8-mb depending on the xpack.enabled value provided. The monitoring-* template is defined in elasticsearch, could you also add the mappings there ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing - opened PR here: elastic/elasticsearch#95388


The ratio of time the event loop is not idling in the event provider to the total time the event loop is running.



*`kibana.stats.process.event_loop_utilization.active`*::
+
--
Duration of time event loop has been active since last measurement.


type: scaled_float

--

*`kibana.stats.process.event_loop_utilization.idle`*::
+
--
Duration of time event loop has been idle since last measurement.


type: scaled_float

--

*`kibana.stats.process.event_loop_utilization.utilization`*::
+
--
Computed utilization value representing ratio of active to idle time since last measurement.


type: scaled_float

--
Expand Down
9 changes: 9 additions & 0 deletions metricbeat/module/kibana/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@
- name: process.event_loop_delay
type: alias
path: kibana.stats.process.event_loop_delay.ms
- name: process.event_loop_utilization.active
type: alias
path: kibana.stats.process.event_loop_utilization.active
- name: process.event_loop_utilization.idle
type: alias
path: kibana.stats.process.event_loop_utilization.idle
- name: process.event_loop_utilization.utilization
type: alias
path: kibana.stats.process.event_loop_utilization.utilization
- name: requests.total
type: alias
path: kibana.stats.request.total
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/kibana/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions metricbeat/module/kibana/stats/_meta/data.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@
"event_loop_delay": {
"ms": 0.14306800067424774
},
"event_loop_utilization": {
"active": 629.1224170000005,
"idle": 359.23554199999995,
"utilization": 0.6365329598160299
},
"memory": {
"heap": {
"size_limit": {
Expand Down
17 changes: 17 additions & 0 deletions metricbeat/module/kibana/stats/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,23 @@
type: scaled_float
description: >
Event loop delay in milliseconds
- name: event_loop_utilization
type: group
description: >
The ratio of time the event loop is not idling in the event provider to the total time the event loop is running.
fields:
- name: active
type: scaled_float
description: >
Duration of time event loop has been active since last measurement.
- name: idle
type: scaled_float
description: >
Duration of time event loop has been idle since last measurement.
- name: utilization
type: scaled_float
description: >
Computed utilization value representing ratio of active to idle time since last measurement.
- name: memory.heap
type: group
description: >
Expand Down
162 changes: 162 additions & 0 deletions metricbeat/module/kibana/stats/_meta/test/stats.880.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
{
"kibana":{
"uuid":"5b2de169-2785-441b-ae8c-186a1936b17d",
"name":"Shaunaks-MBP-2",
"index":".kibana",
"host":"localhost",
"transport_address":"localhost:5601",
"version":"7.0.0-alpha1",
"snapshot":false,
"status":"green"
},
"last_updated":"2018-07-18T00:32:00.948Z",
"collection_interval_ms":5000,
"process":{
"memory":{
"heap":{
"total_bytes":223391744,
"used_bytes":198413592,
"size_limit":1501560832
},
"resident_set_size_bytes":347242496
},
"event_loop_delay":0.25226891040802,
"event_loop_utilization": {
"active": 629.1224170000005,
"idle": 359.23554199999995,
"utilization": 0.6365329598160299
},
"pid":46426,
"uptime_ms":1753889
},
"os":{
"load":{
"1m":3.50634765625,
"5m":3.76904296875,
"15m":3.54833984375
},
"memory":{
"total_bytes":17179869184,
"free_bytes":31711232,
"used_bytes":17148157952
},
"uptime_ms":2187246000
},
"elasticsearch_client": {
"protocol": "http",
"connected_nodes": 2,
"nodes_with_active_sockets": 2,
"nodes_with_idle_sockets": 1,
"total_active_sockets": 14,
"total_idle_sockets": 42,
"total_queued_requests": 0,
"most_active_node_sockets": 10,
"average_active_sockets_per_node": 7,
"most_idle_node_sockets": 42,
"average_idle_sockets_per_node": 21
},
"response_times":{
"max_ms":0
},
"requests":{
"total":0,
"disconnects":0,
"status_codes":{

}
},
"concurrent_connections":3,
"usage":{
"kibana":{
"index":".kibana",
"dashboard":{
"total":0
},
"visualization":{
"total":0
},
"search":{
"total":0
},
"index_pattern":{
"total":0
},
"graph_workspace":{
"total":0
},
"timelion_sheet":{
"total":0
}
},
"reporting":{
"available":true,
"enabled":true,
"browser_type":"phantom",
"all":0,
"csv":{
"available":true,
"total":0
},
"printable_pdf":{
"available":true,
"total":0,
"app":{
"visualization":0,
"dashboard":0
},
"layout":{
"print":0,
"preserve_layout":0
}
},
"status":{

},
"last_day":{
"all":0,
"csv":{
"available":true,
"total":0
},
"printable_pdf":{
"available":true,
"total":0,
"app":{
"visualization":0,
"dashboard":0
},
"layout":{
"print":0,
"preserve_layout":0
}
},
"status":{

}
},
"last_7_days":{
"all":0,
"csv":{
"available":true,
"total":0
},
"printable_pdf":{
"available":true,
"total":0,
"app":{
"visualization":0,
"dashboard":0
},
"layout":{
"print":0,
"preserve_layout":0
}
},
"status":{

}
}
}
},
"cluster_uuid":"NkfU5AinRnyFnqBD36zhEw"
}
5 changes: 5 additions & 0 deletions metricbeat/module/kibana/stats/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ var (
"event_loop_delay": s.Object{
"ms": c.Float("event_loop_delay"),
},
"event_loop_utilization": c.Dict("event_loop_utilization", s.Schema{
"active": c.Float("active"),
"idle": c.Float("idle"),
"utilization": c.Float("utilization"),
}, c.DictOptional),
lukeelmers marked this conversation as resolved.
Show resolved Hide resolved
"memory": c.Dict("memory", s.Schema{
"resident_set_size": s.Object{
"bytes": c.Int("resident_set_size_bytes"),
Expand Down