Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure metrics exporter for loadbalancing when possible #2242

Merged
merged 11 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Main (unreleased)

### Features

- Add support for metrics in `otelcol.exporter.loadbalancing` (@madaraszg-tulip)

- Add `add_cloudwatch_timestamp` to `prometheus.exporter.cloudwatch` metrics. (@captncraig)

- Add support to `prometheus.operator.servicemonitors` to allow `endpointslice` role. (@yoyosir)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,18 @@ Name | Type | Description | Default |
`routing_key` | `string` | Routing strategy for load balancing. | `"traceID"` | no

The `routing_key` attribute determines how to route signals across endpoints. Its value could be one of the following:
* `"service"`: spans with the same `service.name` will be exported to the same backend.
- `"service"`: spans/logs/metrics with the same `service.name` will be exported to the same backend.
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved
This is useful when using processors like the span metrics, so all spans for each service are sent to consistent {{< param "PRODUCT_NAME" >}} instances
for metric collection. Otherwise, metrics for the same services would be sent to different instances, making aggregations inaccurate.
* `"traceID"`: spans belonging to the same traceID will be exported to the same backend.
- `"traceID"`: spans/logs belonging to the same traceID will be exported to the same backend.
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved
- `"resource"`: metrics belonging to the same resource will be exported to the same backend.
- `"metric"`: metrics with the same name will be exported to the same backend
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved
- `"streamID"`: metrics with the same streamID will be exported to the same backend
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved

The loadbalancer will configure the exporter for the singal types supported by the `routing_key`
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved

Metrics support in `otelcol.exporter.loadbalancing` is considered experimental, so an exporter
will be configured for metrics only if Alloy is run with `--stability-level=experimental`
madaraszg-tulip marked this conversation as resolved.
Show resolved Hide resolved

## Blocks

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
otelcolCfg "github.com/grafana/alloy/internal/component/otelcol/config"
"github.com/grafana/alloy/internal/component/otelcol/exporter"
"github.com/grafana/alloy/internal/featuregate"
"github.com/grafana/alloy/internal/runtime/logging/level"
"github.com/grafana/alloy/syntax"
"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter"
otelcomponent "go.opentelemetry.io/collector/component"
Expand All @@ -34,10 +35,33 @@ func init() {

Build: func(opts component.Options, args component.Arguments) (component.Component, error) {
fact := loadbalancingexporter.NewFactory()
//TODO(ptodev): LB exporter cannot yet work with metrics due to a limitation in Alloy:
// https://github.com/grafana/agent/pull/5684
// Once the limitation is removed, we may be able to remove the need for exporter.TypeSignal altogether.
return exporter.New(opts, fact, args.(Arguments), exporter.TypeLogs|exporter.TypeTraces)

myArgs := args.(Arguments)

var typeSignal exporter.TypeSignal

// As per https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/loadbalancingexporter/README.md
// metrics is considered "development" stability level

switch myArgs.RoutingKey {
ptodev marked this conversation as resolved.
Show resolved Hide resolved
case "traceID":
typeSignal = exporter.TypeLogs | exporter.TypeTraces
case "service":
if opts.MinStability.Permits(featuregate.StabilityExperimental) {
typeSignal = exporter.TypeLogs | exporter.TypeTraces | exporter.TypeMetrics
} else {
level.Warn(opts.Logger).Log("msg", "disabling metrics exporter as stability level does not allow it")
typeSignal = exporter.TypeLogs | exporter.TypeTraces
}
case "resource", "metric", "streamID":
if opts.MinStability.Permits(featuregate.StabilityExperimental) {
typeSignal = exporter.TypeMetrics
} else {
level.Warn(opts.Logger).Log("msg", "disabling metrics exporter as stability level does not allow it")
}
}

return exporter.New(opts, fact, myArgs, typeSignal)
},
})
}
Expand Down Expand Up @@ -69,13 +93,10 @@ func (args *Arguments) SetToDefault() {

// Validate implements syntax.Validator.
func (args *Arguments) Validate() error {
//TODO(ptodev): Add support for "resource" and "metric" routing keys later.
// The reason we can't add them yet is that otelcol.exporter.loadbalancing
// is labeled as "beta", but those routing keys are experimental.
// We need a way to label otelcol.exporter.loadbalancing as "public-preview"
// for logs and traces, but "experimental" for metrics.
// Allow routing keys for all signal types. Metrics exporter will be disabled
// if stability level is above experimental
switch args.RoutingKey {
case "service", "traceID":
case "service", "traceID", "resource", "metric", "streamID":
// The routing key is valid.
default:
return fmt.Errorf("invalid routing key %q", args.RoutingKey)
Expand Down
Loading