From a1d8a9e9f9b61d7cf53ffbdbd7fe936ecc54eaf5 Mon Sep 17 00:00:00 2001 From: Xavi Date: Fri, 1 Oct 2021 13:33:18 +0200 Subject: [PATCH 1/7] Add ExposingRuleMetrics doc page --- docs/source/recipes/exposing_rule_metrics.rst | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 docs/source/recipes/exposing_rule_metrics.rst diff --git a/docs/source/recipes/exposing_rule_metrics.rst b/docs/source/recipes/exposing_rule_metrics.rst new file mode 100644 index 00000000..1361c717 --- /dev/null +++ b/docs/source/recipes/exposing_rule_metrics.rst @@ -0,0 +1,58 @@ +.. _writingrules: + +Exposing Rule Metrics +===================== + +Configuration +------------- +Running ElastAlert with ``--prometheus_port`` configuration flag will expose ElastAlert 2 Prometheus metrics on the specified port. Prometheus metrics are disabled by default. + +To expose ElastAlert rule metrics on port ``9979`` run the following command: + +.. code-block:: console + + $ elastalert --config config.yaml --prometheus_port 9979 + +Rule Metrics +------------ + +The exposed metrics are in the `Prometheus text-based format `_. Metrics are of the metric type `counter `_ or `gauge `_ and follow the `Prometheus metric naming `_. + +In the standard metric definition, the metric names are structured as follows: + +.. code-block:: console + + elastalert_{metric}_{unit} + +Where: + +- ``{metric}`` is a unique name of the metric. For example, ``hits``. +- ``{unit}`` is the unit of measurement of the metric value. For example, ``total`` is a counter type metric and ``created`` is a gauge type metric. + +All metrics except ``elastalert_errors_{unit}`` have values that apply to a particular rule name. In the exported metrics, these can be identified using the ``rule_name`` `Prometheus label `_. + +Find below all available metrics: + ++---------------------------------------+-----------------+---------------------------+---------------+ +| METRIC | Type | Description | Label | ++=======================================+=================+===========================+===============+ +| ``elastalert_scrapes_{unit}`` | Counter, Gauge | Number of scrapes | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_hits_{unit}`` | Counter, Gauge | Number of hit | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_matches_{unit}`` | Counter, Gauge | Number of matches | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_time_taken_{unit}`` | Counter, Gauge | Number of time taken | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_alerts_sent_{unir}`` | Counter, Gauge | Number of alerts sent | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_alerts_not_sent_{unit}`` | Counter, Gauge | Number of alerts not sent | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_alerts_silenced_{unit}`` | Counter, Gauge | Number of silenced alerts | ``rule_name`` | ++---------------------------------------+-----------------+---------------------------+---------------+ +| ``elastalert_errors_{unit}`` | Counter, Gauge | Number of errors | | ++---------------------------------------+-----------------+---------------------------+---------------+ + + + + From 3773c541f0968d2957adfc7c7370cdce9d41b67f Mon Sep 17 00:00:00 2001 From: Xavi Date: Fri, 1 Oct 2021 13:34:33 +0200 Subject: [PATCH 2/7] Updated index --- docs/source/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/index.rst b/docs/source/index.rst index 77e9153b..4cce22bf 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -21,6 +21,7 @@ Contents: recipes/writing_filters recipes/adding_enhancements recipes/adding_loaders + recipes/exposing_rule_metrics recipes/signing_requests recipes/faq From e7de7e78f566d2b8afd147157deeceb58af3da41 Mon Sep 17 00:00:00 2001 From: Xavi Date: Tue, 5 Oct 2021 11:55:39 +0200 Subject: [PATCH 3/7] Fix typos --- docs/source/recipes/exposing_rule_metrics.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/recipes/exposing_rule_metrics.rst b/docs/source/recipes/exposing_rule_metrics.rst index 1361c717..c51e86c4 100644 --- a/docs/source/recipes/exposing_rule_metrics.rst +++ b/docs/source/recipes/exposing_rule_metrics.rst @@ -38,11 +38,11 @@ Find below all available metrics: +=======================================+=================+===========================+===============+ | ``elastalert_scrapes_{unit}`` | Counter, Gauge | Number of scrapes | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ -| ``elastalert_hits_{unit}`` | Counter, Gauge | Number of hit | ``rule_name`` | +| ``elastalert_hits_{unit}`` | Counter, Gauge | Number of hits | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ | ``elastalert_matches_{unit}`` | Counter, Gauge | Number of matches | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ -| ``elastalert_time_taken_{unit}`` | Counter, Gauge | Number of time taken | ``rule_name`` | +| ``elastalert_time_taken_{unit}`` | Counter, Gauge | Amount of time taken | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ | ``elastalert_alerts_sent_{unir}`` | Counter, Gauge | Number of alerts sent | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ From 922c784cae6c463c1ef2a328457b76e24a40681e Mon Sep 17 00:00:00 2001 From: Xavi Date: Tue, 5 Oct 2021 11:59:37 +0200 Subject: [PATCH 4/7] Update CHANGELOG.md --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa1701e7..178f3b65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ - None ## Other changes -- None +- [Docs] Add exposed metrics documentation - [#498](https://github.com/jertel/elastalert2/pull/498) - @thisisxgp # 2.2.2 From 10a50dafe7c40eb245910589e9a28144f4c7c398 Mon Sep 17 00:00:00 2001 From: Xavi Garcia Date: Tue, 5 Oct 2021 12:30:38 +0200 Subject: [PATCH 5/7] Updated rule metrics table --- docs/source/recipes/exposing_rule_metrics.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/recipes/exposing_rule_metrics.rst b/docs/source/recipes/exposing_rule_metrics.rst index c51e86c4..70c0ef15 100644 --- a/docs/source/recipes/exposing_rule_metrics.rst +++ b/docs/source/recipes/exposing_rule_metrics.rst @@ -16,7 +16,7 @@ To expose ElastAlert rule metrics on port ``9979`` run the following command: Rule Metrics ------------ -The exposed metrics are in the `Prometheus text-based format `_. Metrics are of the metric type `counter `_ or `gauge `_ and follow the `Prometheus metric naming `_. +The metrics being exposed are related to the `ElastAlert metadata indices `_. The exposed metrics are in the `Prometheus text-based format `_. Metrics are of the metric type `counter `_ or `gauge `_ and follow the `Prometheus metric naming `_. In the standard metric definition, the metric names are structured as follows: @@ -42,7 +42,7 @@ Find below all available metrics: +---------------------------------------+-----------------+---------------------------+---------------+ | ``elastalert_matches_{unit}`` | Counter, Gauge | Number of matches | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ -| ``elastalert_time_taken_{unit}`` | Counter, Gauge | Amount of time taken | ``rule_name`` | +| ``elastalert_time_taken_{unit}`` | Counter, Gauge | Time taken in seconds | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ | ``elastalert_alerts_sent_{unir}`` | Counter, Gauge | Number of alerts sent | ``rule_name`` | +---------------------------------------+-----------------+---------------------------+---------------+ From fdeaf9ddefcf820f2cea35eec816649278dde45f Mon Sep 17 00:00:00 2001 From: Xavi Garcia Date: Tue, 5 Oct 2021 12:55:20 +0200 Subject: [PATCH 6/7] Updated rst page label --- docs/source/recipes/exposing_rule_metrics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/recipes/exposing_rule_metrics.rst b/docs/source/recipes/exposing_rule_metrics.rst index 70c0ef15..40d13550 100644 --- a/docs/source/recipes/exposing_rule_metrics.rst +++ b/docs/source/recipes/exposing_rule_metrics.rst @@ -1,4 +1,4 @@ -.. _writingrules: +.. _exposingrulemetrics: Exposing Rule Metrics ===================== From 339ddf717bf8a633f91899804101e7a9534754a1 Mon Sep 17 00:00:00 2001 From: Xavi Garcia Date: Tue, 5 Oct 2021 13:39:07 +0200 Subject: [PATCH 7/7] Added link to new page from /running_elastalert --- docs/source/running_elastalert.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/running_elastalert.rst b/docs/source/running_elastalert.rst index 54b80c6c..eabcc2d6 100644 --- a/docs/source/running_elastalert.rst +++ b/docs/source/running_elastalert.rst @@ -44,7 +44,7 @@ logs `localhost:9200` instead of the actual ``es_host``:``es_port``. ``--pin_rules`` will stop ElastAlert 2 from loading, reloading or removing rules based on changes to their config files. -``--prometheus_port`` exposes ElastAlert 2 Prometheus metrics on the specified +``--prometheus_port`` exposes ElastAlert 2 `Prometheus metrics `_ on the specified port. Prometheus metrics disabled by default. ``--rule `` will only run the given rule. The rule file may be a