-
Notifications
You must be signed in to change notification settings - Fork 94
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add target allocator guidance (#642)
* initial ta doc structure * Add config example * Add TA to add-on index * fix prom casing * Update TA description * Update src/docs/getting-started/adot-eks-add-on/target-allocator.mdx
- Loading branch information
1 parent
87a4c8f
commit 39dda8b
Showing
3 changed files
with
104 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
102 changes: 102 additions & 0 deletions
102
src/docs/getting-started/adot-eks-add-on/target-allocator.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
--- | ||
title: 'Target Allocator' | ||
description: | ||
This page provides an introduction to the Target Allocator | ||
path: '/docs/getting-started/adot-eks-add-on/target-allocator' | ||
--- | ||
|
||
Installing the EKS Add-on gives you access to the Target Allocator (TA). Enabling the Target Allocator for an | ||
OpenTelemetry Collector custom resource decouples Prometheus service discovery and metrics collection. When enabled, | ||
the Target Allocator manages the configuration of the Prometheus receiver. Specifically, | ||
the Target Allocator allows for distribution of Prometheus scrape targets across a pool of Collectors. This functionality can | ||
be used to avoid the manual sharding of Prometheus scrape targets that can be required when horizontally scaling | ||
a single Prometheus receiver configuration. | ||
Visit the [Target Allocator Readme](https://github.com/open-telemetry/opentelemetry-operator/blob/main/cmd/otel-allocator/README.md) | ||
for a deeper introduction. | ||
|
||
## Out of Order Samples | ||
|
||
In some cases the Target Allocator can be used to solve Prometheus remote write out of order sample errors. A common | ||
reason Collector users will see out of order sample erorrs is due to multiple collectors scraping the same metric | ||
endpoint. Collector users may experience this when horizontally scaling Collectors with Prometheus | ||
receivers or daemonset deployments without manually sharding scrape targets. Both of these scenarios can lead to | ||
out of order sample erorrs without sharding of Prometheus scrape targets. | ||
|
||
## Getting Started | ||
|
||
Enabling the target allocator requires setting `OpenTelemeteryCollector.spec.targetAllocator.enabled` to `true`. | ||
This will cause the Prometheus configuration within the Collector CRD to be rewritten to use an `http_sd_config` | ||
from the Target Allocator. The operator will also create a new deployment and service for this http_sd_config | ||
directive. | ||
|
||
``` | ||
apiVersion: opentelemetry.io/v1alpha1 | ||
kind: OpenTelemetryCollector | ||
metadata: | ||
name: collector-with-ta | ||
spec: | ||
mode: statefulset | ||
servicAccount: adot-collector | ||
targetAllocator: | ||
enabled: true | ||
config: | | ||
receivers: | ||
prometheus: | ||
config: | ||
scrape_configs: | ||
- job_name: 'otel-collector' | ||
scrape_interval: 10s | ||
static_configs: | ||
- targets: [ '0.0.0.0:8888' ] | ||
metric_relabel_configs: | ||
- action: labeldrop | ||
regex: (id|name) | ||
replacement: $$1 | ||
- action: labelmap | ||
regex: label_(.+) | ||
replacement: $$1 | ||
processors: | ||
batch: | ||
exporters: | ||
prometheusremotewrite: | ||
endpoint: ${AMP_REMOTE_WRITE_URL} | ||
auth: | ||
authenticator: sigv4auth | ||
extensions: | ||
sigv4auth: | ||
region: ${AWS_REGION} | ||
service: "aps" | ||
service: | ||
pipelines: | ||
metrics: | ||
receivers: [prometheus] | ||
processors: [batch] | ||
exporters: [prometheusremotewrite] | ||
``` | ||
|
||
Please reference the [OpenTelemetry Operator documentation](https://github.com/open-telemetry/opentelemetry-operator/tree/main#target-allocator) | ||
for more information on using the Target Allocator. | ||
|
||
### Useful links | ||
|
||
[Target Allocator API Spec](https://github.com/open-telemetry/opentelemetry-operator/blob/main/docs/api.md#opentelemetrycollectorspectargetallocator) | ||
|
||
[Scaling OpenTelmetery Collectors](https://opentelemetry.io/docs/collector/scaling/) | ||
|
||
--- | ||
## [Previous Topic: Kubernetes Attributes Processor](/docs/getting-started/adot-eks-add-on/k8s-attr-processor) | ||
|
||
## Related Topics: | ||
|
||
### [Collector Configuration for Amazon Managed Prometheus](/docs/getting-started/adot-eks-add-on/config-amp) | ||
|
||
### [Collector Configuration for AWS CloudWatch](/docs/getting-started/adot-eks-add-on/config-cloudwatch) | ||
|
||
### [Advanced Collector Configuration for Amazon Managed Prometheus](/docs/getting-started/adot-eks-add-on/config-advanced) | ||
|
||
## [Next Topic: Updating and Cleanup](/docs/getting-started/adot-eks-add-on/update-and-cleanup) |