-
Notifications
You must be signed in to change notification settings - Fork 236
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add a prometheus label mapping component
- Loading branch information
Showing
7 changed files
with
488 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# Proposal: Add a component to perform label mapping efficiently | ||
|
||
* Author(s): Nicolas DUPEUX | ||
* Last updated: 19/11/2024 | ||
* Original issue: https://github.com/grafana/alloy/pull/2025 | ||
|
||
## Abstract | ||
|
||
Add a component to populate a label value based on a lookup table | ||
|
||
## Problem | ||
|
||
Using `prometheus.relabel` to populate a label value based on another label value is inefficient as we have to have a rule block for each source label value. | ||
|
||
If we have 1k values to map, we'll have to execute 1k regex for each datapoint resulting in an algorithm complexity of O(n). | ||
|
||
## Proposal | ||
|
||
Replace regex computing by a lookup table. Algorithm complexity goes from O(n) to O(1) | ||
|
||
## Pros and cons | ||
|
||
Pros: | ||
- resource efficient | ||
|
||
Cons: | ||
- New component | ||
|
||
## Alternative solutions | ||
|
||
- Instanciate more CPU resources to perform the task | ||
- Optimize prometheus.relabel component | ||
- Summarize regex when severals keys have to same value. | ||
|
||
## Compatibility | ||
|
||
As this is a new component, there isn't any compatibility issue as long as you don't use it. | ||
|
||
## Implementation | ||
|
||
https://github.com/grafana/alloy/pull/2025 | ||
|
||
## Related open issues | ||
|
||
None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
133 changes: 133 additions & 0 deletions
133
docs/sources/reference/components/prometheus/prometheus.mapping.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
--- | ||
canonical: https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.mapping/ | ||
description: Learn about prometheus.mapping | ||
title: prometheus.mapping | ||
--- | ||
|
||
# prometheus.mapping | ||
|
||
Prometheus metrics follow the [OpenMetrics](https://openmetrics.io/) format. | ||
Each time series is uniquely identified by its metric name, plus optional | ||
key-value pairs called labels. Each sample represents a datapoint in the | ||
time series and contains a value and an optional timestamp. | ||
|
||
```text | ||
<metric name>{<label_1>=<label_val_1>, <label_2>=<label_val_2> ...} <value> [timestamp] | ||
``` | ||
|
||
The `prometheus.mapping` component create new labels on each metric passed | ||
along to the exported receiver by applying a mapping table to a label value. | ||
|
||
The most common use of `prometheus.mapping` is to create new labels with a high | ||
cardinality source label value (>1k) when a large set of regex is inefficient. | ||
|
||
You can specify multiple `prometheus.mapping` components by givin them | ||
different labels. | ||
|
||
## Usage | ||
|
||
```alloy | ||
prometheus.mapping "LABEL" { | ||
forward_to = RECEIVER_LIST | ||
source_label = "labelA" | ||
mapping = { | ||
"from" = {"labelB" = "to"}, | ||
... | ||
} | ||
} | ||
``` | ||
|
||
## Arguments | ||
|
||
The following arguments are supported: | ||
|
||
Name | Type | Description | Default | Required | ||
---------------|---------------------------|-------------------------------------------------------------------------|---------|--------- | ||
`forward_to` | `list(MetricsReceiver)` | Where the metrics should be forwarded to, after relabeling takes place. | | yes | ||
`source_label` | `string` | Name of the source label to use for mapping. | | yes | ||
`target_label` | `string` | Name of the target label to use for mapping. | | yes | ||
`mapping` | `map(string,map(string))` | Mapping from source label value to target labels name/value. | | yes | ||
|
||
## Exported fields | ||
|
||
The following fields are exported and can be referenced by other components: | ||
|
||
Name | Type | Description | ||
-----------|-------------------|----------------------------------------------------------- | ||
`receiver` | `MetricsReceiver` | The input receiver where samples are sent to be relabeled. | ||
|
||
## Component health | ||
|
||
`prometheus.mapping` is only reported as unhealthy if given an invalid | ||
configuration. In those cases, exported fields are kept at their last healthy | ||
values. | ||
|
||
## Debug information | ||
|
||
`prometheus.mapping` doesn't expose any component-specific debug information. | ||
|
||
## Debug metrics | ||
|
||
* `prometheus_mapping_metrics_processed` (counter): Total number of metrics processed. | ||
* `prometheus_mapping_metrics_written` (counter): Total number of metrics written. | ||
|
||
## Example | ||
|
||
You can create and an instance of a see `prometheus.mapping` component and see how | ||
it acts on the following metrics. | ||
|
||
```alloy | ||
prometheus.mapping "keep_backend_only" { | ||
forward_to = [prometheus.remote_write.onprem.receiver] | ||
source_label = "app" | ||
mapping = { | ||
"frontend" = {"team" = "teamA"} | ||
"backend" = {"team" = "teamB"} | ||
"database" = {"team" = "teamC"} | ||
} | ||
} | ||
``` | ||
|
||
``` | ||
metric_a{__address__ = "localhost", instance = "development", app = "frontend"} 10 | ||
metric_a{__address__ = "localhost", instance = "development", app = "backend"} 2 | ||
metric_a{__address__ = "cluster_a", instance = "production", app = "frontend"} 7 | ||
metric_a{__address__ = "cluster_a", instance = "production", app = "backend"} 9 | ||
metric_a{__address__ = "cluster_b", instance = "production", app = "database"} 4 | ||
``` | ||
|
||
After applying the mapping a new `team` label is created based on mapping table | ||
and `app` label value. | ||
|
||
``` | ||
metric_a{team = "teamA", __address__ = "localhost", instance = "development", app = "frontend"} 10 | ||
metric_a{team = "teamB", __address__ = "localhost", instance = "development", app = "backend"} 2 | ||
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "frontend"} 7 | ||
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "backend"} 9 | ||
metric_a{team = "teamC", __address__ = "cluster_a", instance = "production", app = "database"} 4 | ||
``` | ||
|
||
The resulting metrics are then propagated to each receiver defined in the | ||
`forward_to` argument. | ||
<!-- START GENERATED COMPATIBLE COMPONENTS --> | ||
|
||
## Compatible components | ||
|
||
`prometheus.mapping` can accept arguments from the following components: | ||
|
||
- Components that export [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-exporters) | ||
|
||
`prometheus.mapping` has exports that can be consumed by the following components: | ||
|
||
- Components that consume [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-consumers) | ||
|
||
{{< admonition type="note" >}} | ||
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. | ||
Refer to the linked documentation for more details. | ||
{{< /admonition >}} | ||
|
||
<!-- END GENERATED COMPATIBLE COMPONENTS --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.