Skip to content

Commit

Permalink
Add a prometheus label mapping component
Browse files Browse the repository at this point in the history
  • Loading branch information
vaxvms committed Nov 26, 2024
1 parent 48f9206 commit d77e275
Show file tree
Hide file tree
Showing 7 changed files with 488 additions and 0 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ Main (unreleased)

- Add `otelcol.exporter.syslog` component to export logs in syslog format (@dehaansa)

- Add a `prometheus.mapping` component to add a label based on a source_label and a mapping table. (@vaxvms)

### Enhancements

- Add second metrics sample to the support bundle to provide delta information (@dehaansa)
Expand Down
45 changes: 45 additions & 0 deletions docs/design/2025-prometheus-mapping-component.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Proposal: Add a component to perform label mapping efficiently

* Author(s): Nicolas DUPEUX
* Last updated: 19/11/2024
* Original issue: https://github.com/grafana/alloy/pull/2025

## Abstract

Add a component to populate a label value based on a lookup table

## Problem

Using `prometheus.relabel` to populate a label value based on another label value is inefficient as we have to have a rule block for each source label value.

If we have 1k values to map, we'll have to execute 1k regex for each datapoint resulting in an algorithm complexity of O(n).

## Proposal

Replace regex computing by a lookup table. Algorithm complexity goes from O(n) to O(1)

## Pros and cons

Pros:
- resource efficient

Cons:
- New component

## Alternative solutions

- Instanciate more CPU resources to perform the task
- Optimize prometheus.relabel component
- Summarize regex when severals keys have to same value.

## Compatibility

As this is a new component, there isn't any compatibility issue as long as you don't use it.

## Implementation

https://github.com/grafana/alloy/pull/2025

## Related open issues

None
2 changes: 2 additions & 0 deletions docs/sources/reference/compatibility/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ The following components, grouped by namespace, _export_ Prometheus `MetricsRece
{{< /collapse >}}

{{< collapse title="prometheus" >}}
- [prometheus.mapping](../components/prometheus/prometheus.mapping)
- [prometheus.relabel](../components/prometheus/prometheus.relabel)
- [prometheus.remote_write](../components/prometheus/prometheus.remote_write)
- [prometheus.write.queue](../components/prometheus/prometheus.write.queue)
Expand All @@ -192,6 +193,7 @@ The following components, grouped by namespace, _consume_ Prometheus `MetricsRec
{{< /collapse >}}

{{< collapse title="prometheus" >}}
- [prometheus.mapping](../components/prometheus/prometheus.mapping)
- [prometheus.operator.podmonitors](../components/prometheus/prometheus.operator.podmonitors)
- [prometheus.operator.probes](../components/prometheus/prometheus.operator.probes)
- [prometheus.operator.servicemonitors](../components/prometheus/prometheus.operator.servicemonitors)
Expand Down
133 changes: 133 additions & 0 deletions docs/sources/reference/components/prometheus/prometheus.mapping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
canonical: https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.mapping/
description: Learn about prometheus.mapping
title: prometheus.mapping
---

# prometheus.mapping

Prometheus metrics follow the [OpenMetrics](https://openmetrics.io/) format.
Each time series is uniquely identified by its metric name, plus optional
key-value pairs called labels. Each sample represents a datapoint in the
time series and contains a value and an optional timestamp.

```text
<metric name>{<label_1>=<label_val_1>, <label_2>=<label_val_2> ...} <value> [timestamp]
```

The `prometheus.mapping` component create new labels on each metric passed
along to the exported receiver by applying a mapping table to a label value.

The most common use of `prometheus.mapping` is to create new labels with a high
cardinality source label value (>1k) when a large set of regex is inefficient.

You can specify multiple `prometheus.mapping` components by givin them
different labels.

## Usage

```alloy
prometheus.mapping "LABEL" {
forward_to = RECEIVER_LIST
source_label = "labelA"
mapping = {
"from" = {"labelB" = "to"},
...
}
}
```

## Arguments

The following arguments are supported:

Name | Type | Description | Default | Required
---------------|---------------------------|-------------------------------------------------------------------------|---------|---------
`forward_to` | `list(MetricsReceiver)` | Where the metrics should be forwarded to, after relabeling takes place. | | yes
`source_label` | `string` | Name of the source label to use for mapping. | | yes
`target_label` | `string` | Name of the target label to use for mapping. | | yes
`mapping` | `map(string,map(string))` | Mapping from source label value to target labels name/value. | | yes

## Exported fields

The following fields are exported and can be referenced by other components:

Name | Type | Description
-----------|-------------------|-----------------------------------------------------------
`receiver` | `MetricsReceiver` | The input receiver where samples are sent to be relabeled.

## Component health

`prometheus.mapping` is only reported as unhealthy if given an invalid
configuration. In those cases, exported fields are kept at their last healthy
values.

## Debug information

`prometheus.mapping` doesn't expose any component-specific debug information.

## Debug metrics

* `prometheus_mapping_metrics_processed` (counter): Total number of metrics processed.
* `prometheus_mapping_metrics_written` (counter): Total number of metrics written.

## Example

You can create and an instance of a see `prometheus.mapping` component and see how
it acts on the following metrics.

```alloy
prometheus.mapping "keep_backend_only" {
forward_to = [prometheus.remote_write.onprem.receiver]
source_label = "app"
mapping = {
"frontend" = {"team" = "teamA"}
"backend" = {"team" = "teamB"}
"database" = {"team" = "teamC"}
}
}
```

```
metric_a{__address__ = "localhost", instance = "development", app = "frontend"} 10
metric_a{__address__ = "localhost", instance = "development", app = "backend"} 2
metric_a{__address__ = "cluster_a", instance = "production", app = "frontend"} 7
metric_a{__address__ = "cluster_a", instance = "production", app = "backend"} 9
metric_a{__address__ = "cluster_b", instance = "production", app = "database"} 4
```

After applying the mapping a new `team` label is created based on mapping table
and `app` label value.

```
metric_a{team = "teamA", __address__ = "localhost", instance = "development", app = "frontend"} 10
metric_a{team = "teamB", __address__ = "localhost", instance = "development", app = "backend"} 2
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "frontend"} 7
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "backend"} 9
metric_a{team = "teamC", __address__ = "cluster_a", instance = "production", app = "database"} 4
```

The resulting metrics are then propagated to each receiver defined in the
`forward_to` argument.
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`prometheus.mapping` can accept arguments from the following components:

- Components that export [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-exporters)

`prometheus.mapping` has exports that can be consumed by the following components:

- Components that consume [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-consumers)

{{< admonition type="note" >}}
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.
{{< /admonition >}}

<!-- END GENERATED COMPATIBLE COMPONENTS -->
1 change: 1 addition & 0 deletions internal/component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ import (
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/statsd" // Import prometheus.exporter.statsd
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/unix" // Import prometheus.exporter.unix
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/windows" // Import prometheus.exporter.windows
_ "github.com/grafana/alloy/internal/component/prometheus/mapping" // Import prometheus.mapping
_ "github.com/grafana/alloy/internal/component/prometheus/operator/podmonitors" // Import prometheus.operator.podmonitors
_ "github.com/grafana/alloy/internal/component/prometheus/operator/probes" // Import prometheus.operator.probes
_ "github.com/grafana/alloy/internal/component/prometheus/operator/servicemonitors" // Import prometheus.operator.servicemonitors
Expand Down
Loading

0 comments on commit d77e275

Please sign in to comment.