-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add metricbeat benchmark module #41801
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
//// | ||
This file is generated! See scripts/mage/docs_collector.go | ||
//// | ||
|
||
:modulename: benchmark | ||
:edit_url: https://github.com/elastic/beats/edit/main/x-pack/metricbeat/module/benchmark/_meta/docs.asciidoc | ||
|
||
|
||
[[metricbeat-module-benchmark]] | ||
[role="xpack"] | ||
== Benchmark module | ||
|
||
beta[] | ||
|
||
include::{libbeat-dir}/shared/integration-link.asciidoc[] | ||
|
||
:modulename!: | ||
|
||
The `benchmark` module is used to generate synthetic metrics at a predictable rate. This can be useful when you want to test output settings or test system sizing without using real data. | ||
|
||
The `benchmark` module metricset is `info`. | ||
|
||
[source,yaml] | ||
---- | ||
- module: benchmark | ||
metricsets: | ||
- info | ||
enabled: true | ||
period: 10s | ||
---- | ||
|
||
[float] | ||
== Metricsets | ||
|
||
[float] | ||
=== `info` | ||
A metric that includes a `counter` field which is used to keep the metric unique. | ||
|
||
[float] | ||
=== Module-specific configuration notes | ||
|
||
`count`:: number, the number of metrics to emit per fetch. | ||
|
||
|
||
|
||
|
||
|
||
:edit_url: | ||
|
||
[float] | ||
=== Example configuration | ||
|
||
The Benchmark module supports the standard configuration options that are described | ||
in <<configuration-metricbeat>>. Here is an example configuration: | ||
|
||
[source,yaml] | ||
---- | ||
metricbeat.modules: | ||
- module: benchmark | ||
metricsets: | ||
- info | ||
enabled: false | ||
period: 10s | ||
|
||
---- | ||
|
||
[float] | ||
=== Metricsets | ||
|
||
The following metricsets are available: | ||
|
||
* <<metricbeat-metricset-benchmark-info,info>> | ||
|
||
include::benchmark/info.asciidoc[] | ||
|
||
:edit_url!: |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
//// | ||
This file is generated! See scripts/mage/docs_collector.go | ||
//// | ||
:edit_url: https://github.com/elastic/beats/edit/main/x-pack/metricbeat/module/benchmark/info/_meta/docs.asciidoc | ||
|
||
|
||
[[metricbeat-metricset-benchmark-info]] | ||
[role="xpack"] | ||
=== Benchmark info metricset | ||
|
||
beta[] | ||
|
||
include::../../../../x-pack/metricbeat/module/benchmark/info/_meta/docs.asciidoc[] | ||
|
||
|
||
:edit_url: | ||
|
||
==== Fields | ||
|
||
For a description of each field in the metricset, see the | ||
<<exported-fields-benchmark,exported fields>> section. | ||
|
||
Here is an example document generated by this metricset: | ||
|
||
[source,json] | ||
---- | ||
include::../../../../x-pack/metricbeat/module/benchmark/info/_meta/data.json[] | ||
---- | ||
:edit_url!: |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -404,6 +404,14 @@ metricbeat.modules: | |
# Monitoring instead of metricbeat-* indices. | ||
#xpack.enabled: false | ||
|
||
#------------------------------ Benchmark Module ------------------------------ | ||
- module: benchmark | ||
metricsets: | ||
- info | ||
enabled: false | ||
period: 10s | ||
|
||
|
||
#--------------------------------- Ceph Module --------------------------------- | ||
# Metricsets depending on the Ceph REST API (default port: 5000) | ||
- module: ceph | ||
|
@@ -1356,9 +1364,11 @@ metricbeat.modules: | |
# Password to use when connecting to PostgreSQL. Empty by default. | ||
#password: pass | ||
|
||
#----------------------- Prometheus Typed Metrics Module ----------------------- | ||
#------------------------------ Prometheus Module ------------------------------ | ||
# Metrics collected from a Prometheus endpoint | ||
- module: prometheus | ||
period: 10s | ||
metricsets: ["collector"] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this need to change as part of this PR? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, in that it isn't part of this change. But Yes in that |
||
hosts: ["localhost:9090"] | ||
metrics_path: /metrics | ||
#metrics_filters: | ||
|
@@ -1367,45 +1377,27 @@ metricbeat.modules: | |
#username: "user" | ||
#password: "secret" | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# This can be used for service account based authorization: | ||
#bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token | ||
#ssl.certificate_authorities: | ||
# - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# Use Elasticsearch histogram type to store histograms (beta, default: false) | ||
# This will change the default layout and put metric type in the field name | ||
#use_types: true | ||
|
||
# Store counter rates instead of original cumulative counters (experimental, default: false) | ||
#rate_counters: true | ||
|
||
# Metrics sent by a Prometheus server using remote_write option | ||
#- module: prometheus | ||
# metricsets: ["remote_write"] | ||
# host: "localhost" | ||
# port: "9201" | ||
|
||
# Secure settings for the server using TLS/SSL: | ||
#ssl.certificate: "/etc/pki/server/cert.pem" | ||
#ssl.key: "/etc/pki/server/cert.key" | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# Use Elasticsearch histogram type to store histograms (beta, default: false) | ||
# This will change the default layout and put metric type in the field name | ||
#use_types: true | ||
|
||
# Store counter rates instead of original cumulative counters (experimental, default: false) | ||
#rate_counters: true | ||
|
||
# Define patterns for counter and histogram types so as to identify metrics' types according to these patterns | ||
#types_patterns: | ||
# counter_patterns: [] | ||
# histogram_patterns: [] | ||
# Secure settings for the server using TLS/SSL: | ||
#ssl.certificate: "/etc/pki/server/cert.pem" | ||
#ssl.key: "/etc/pki/server/cert.key" | ||
|
||
# Metrics that will be collected using a PromQL | ||
#- module: prometheus | ||
|
@@ -1433,11 +1425,9 @@ metricbeat.modules: | |
# params: | ||
# query: "some_value" | ||
|
||
#------------------------------ Prometheus Module ------------------------------ | ||
# Metrics collected from a Prometheus endpoint | ||
#----------------------- Prometheus Typed Metrics Module ----------------------- | ||
- module: prometheus | ||
period: 10s | ||
metricsets: ["collector"] | ||
hosts: ["localhost:9090"] | ||
metrics_path: /metrics | ||
#metrics_filters: | ||
|
@@ -1446,28 +1436,46 @@ metricbeat.modules: | |
#username: "user" | ||
#password: "secret" | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# This can be used for service account based authorization: | ||
#bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token | ||
#ssl.certificate_authorities: | ||
# - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# Use Elasticsearch histogram type to store histograms (beta, default: false) | ||
# This will change the default layout and put metric type in the field name | ||
#use_types: true | ||
|
||
# Store counter rates instead of original cumulative counters (experimental, default: false) | ||
#rate_counters: true | ||
|
||
# Metrics sent by a Prometheus server using remote_write option | ||
#- module: prometheus | ||
# metricsets: ["remote_write"] | ||
# host: "localhost" | ||
# port: "9201" | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# Secure settings for the server using TLS/SSL: | ||
#ssl.certificate: "/etc/pki/server/cert.pem" | ||
#ssl.key: "/etc/pki/server/cert.key" | ||
|
||
# Count number of metrics present in Elasticsearch document (default: false) | ||
#metrics_count: false | ||
|
||
# Use Elasticsearch histogram type to store histograms (beta, default: false) | ||
# This will change the default layout and put metric type in the field name | ||
#use_types: true | ||
|
||
# Store counter rates instead of original cumulative counters (experimental, default: false) | ||
#rate_counters: true | ||
|
||
# Define patterns for counter and histogram types so as to identify metrics' types according to these patterns | ||
#types_patterns: | ||
# counter_patterns: [] | ||
# histogram_patterns: [] | ||
|
||
# Metrics that will be collected using a PromQL | ||
#- module: prometheus | ||
# metricsets: ["query"] | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
- module: benchmark | ||
metricsets: | ||
- info | ||
enabled: false | ||
period: 10s | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
include::{libbeat-dir}/shared/integration-link.asciidoc[] | ||
|
||
:modulename!: | ||
|
||
The `benchmark` module is used to generate synthetic metrics at a predictable rate. This can be useful when you want to test output settings or test system sizing without using real data. | ||
|
||
The `benchmark` module metricset is `info`. | ||
|
||
[source,yaml] | ||
---- | ||
- module: benchmark | ||
metricsets: | ||
- info | ||
enabled: true | ||
period: 10s | ||
---- | ||
|
||
[float] | ||
== Metricsets | ||
|
||
[float] | ||
=== `info` | ||
A metric that includes a `counter` field which is used to keep the metric unique. | ||
|
||
[float] | ||
=== Module-specific configuration notes | ||
|
||
`count`:: number, the number of metrics to emit per fetch. | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
- key: benchmark | ||
title: "Benchmark" | ||
release: beta | ||
description: > | ||
benchmark module | ||
fields: | ||
- name: benchmark | ||
type: group | ||
description: > | ||
fields: |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
// or more contributor license agreements. Licensed under the Elastic License; | ||
// you may not use this file except in compliance with the Elastic License. | ||
|
||
// Package benchmark is a Metricbeat module that contains MetricSets. | ||
package benchmark |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be a numeric type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not. Even though it is numeric "data", it is unlikely we would use range searches, it is much more likely we want fast retrieval. ie you search to make sure that benchmark metric number 13 was actually indexed.
Here is the tip from elasticsearch docs I'm basing this off of.
Mapping numeric identifiers
Not all numeric data should be mapped as a numeric field data type. Elasticsearch optimizes numeric fields, such as integer or long, for range queries. However, keyword fields are better for term and other term-level queries.
Identifiers, such as an ISBN or a product ID, are rarely used in range queries. However, they are often retrieved using term-level queries.
Consider mapping a numeric identifier as a keyword if:
If you’re unsure which to use, you can use a multi-field to map the data as both a keyword and a numeric data type.