From 1debccc571b8dc8668d1d982cb237b2025cbe211 Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Tue, 9 Jan 2024 08:56:14 +0800 Subject: [PATCH] Modernise OpenTelemetry Collector example config (#3518) * Use OpenTelemetry Collector `debug` exporter The `logging` exporter is deprecated, and its recommended replacement is the `debug` exporter. * Use upstream-documented env var expansion syntax See https://opentelemetry.io/docs/collector/configuration/#environment-variables * Update docs link * More fixes Noticed a couple of unrelated issues while previewing changes: - broken links to OTLP and Elasticsearch exporters - unnecessary parentheses around OTLP/gRPC and OTLP/HTTP --- .../en/observability/apm/otel-direct.asciidoc | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/en/observability/apm/otel-direct.asciidoc b/docs/en/observability/apm/otel-direct.asciidoc index bfc77cf1a1..b4e9a4d293 100644 --- a/docs/en/observability/apm/otel-direct.asciidoc +++ b/docs/en/observability/apm/otel-direct.asciidoc @@ -32,31 +32,31 @@ processors: <2> batch: exporters: - logging: - loglevel: warn <3> + debug: + verbosity: detailed <3> otlp/elastic: <4> # Elastic APM server https endpoint without the "https://" prefix - endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}" <5> <7> + endpoint: "${env:ELASTIC_APM_SERVER_ENDPOINT}" <5> <7> headers: # Elastic APM Server secret token - Authorization: "Bearer ${ELASTIC_APM_SECRET_TOKEN}" <6> <7> + Authorization: "Bearer ${env:ELASTIC_APM_SECRET_TOKEN}" <6> <7> service: pipelines: traces: receivers: [otlp] - exporters: [logging, otlp/elastic] + exporters: [debug, otlp/elastic] metrics: receivers: [otlp] - exporters: [logging, otlp/elastic] + exporters: [debug, otlp/elastic] logs: <8> receivers: [otlp] - exporters: [logging, otlp/elastic] + exporters: [debug, otlp/elastic] ---- <1> The receivers, like the https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver[OTLP receiver], that forward data emitted by APM agents, or the https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver[host metrics receiver]. <2> We recommend using the https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md[Batch processor] and the https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md[memory limiter processor]. For more information, see https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/README.md#recommended-processors[recommended processors]. -<3> The https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/loggingexporter[logging exporter] is helpful for troubleshooting and supports various logging levels, like `debug`, `info`, `warn`, and `error`. +<3> The https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/debugexporter[debug exporter] is helpful for troubleshooting, and supports configurable verbosity levels: `basic` (default), `normal`, and `detailed`. <4> Elastic {observability} endpoint configuration. APM Server supports a ProtoBuf payload via both the OTLP protocol over gRPC transport {ot-grpc}[(OTLP/gRPC)] and the OTLP protocol over HTTP transport {ot-http}[(OTLP/HTTP)]. @@ -65,13 +65,13 @@ https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otl https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter[OTLP/gRPC exporter]. <5> Hostname and port of the APM Server endpoint. For example, `elastic-apm-server:8200`. <6> Credential for Elastic APM <> (`Authorization: "Bearer a_secret_token"`) or <> (`Authorization: "ApiKey an_api_key"`). -<7> Environment-specific configuration parameters can be conveniently passed in as environment variables documented https://opentelemetry.io/docs/collector/configuration/#configuration-environment-variables[here] (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_SECRET_TOKEN`). +<7> Environment-specific configuration parameters can be conveniently passed in as environment variables documented https://opentelemetry.io/docs/collector/configuration/#environment-variables[here] (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_SECRET_TOKEN`). <8> preview:[] To send OpenTelemetry logs to {stack} version 8.0+, declare a `logs` pipeline. You're now ready to export traces and metrics from your services and applications. -TIP: When using the OpenTelemetry collector, you should always prefer sending data via the [`OTLP` exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter) to an Elastic APM Server. -Other methods, like using the [`elasticsearch` exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter) to send data directly to {es} will send data to the {stack}, +TIP: When using the OpenTelemetry collector, you should always prefer sending data via the https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter[`OTLP` exporter] to an Elastic APM Server. +Other methods, like using the https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter[`elasticsearch` exporter] to send data directly to {es} will send data to the {stack}, but will bypass all of the validation and data processing that the APM Server performs. In addition, your data will not be viewable in the {kib} {observability} apps if you use the `elasticsearch` exporter. @@ -128,7 +128,7 @@ and <> in {kib}. [[open-telemetry-proxy-apm]] ==== Proxy requests to APM Server -APM Server supports both the {ot-grpc}[(OTLP/gRPC)] and {ot-http}[(OTLP/HTTP)] protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to the APM Server. +APM Server supports both the {ot-grpc}[OTLP/gRPC] and {ot-http}[OTLP/HTTP] protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to the APM Server. If you use the OTLP/gRPC protocol, requests to the APM Server must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: `"Content-Type: application/grpc"`.