Prototype: OpenTelemetry components for Flow

[rfratto]

This (large) draft PR demonstrates how OpenTelemetry components can be added as Flow components.

This is just a draft to demonstrate the capability, and needs a lot of attention before it's mergeable.

The implementation of these components completely bypasses the OpenTelemetry Collector pipeline system, instead relying on the Flow controller to bind OTel components together, passing around consumers as interfaces.

Each component which can receive telemetry data is bundled as a generic TelemetryConsumer, which accepts metrics, logs, and traces. It is a combination of the three consumer interfaces from OpenTelemetry Collector. If the underlying consumer does not support one of the telemetry types, the request will fail. In the future, this should probably be changed to be validated at load time instead of failing at runtime.

The following example components are included:

• otel.reciever_jaeger: The upstream Jaeger receiver
• otel.exporter_otlp: The upstream OTLP exporter
• otel.receiver_otlp: The upstream OTLP receiver
• otel.receiver_zipkin: The upstream Zipkin receiver.
• otel.exporter_logging: A custom component which uses go-kit's logger for logging about received traces.

There is generic code for constructing and running the various OpenTelmetry Collector components. The remainder of the code involves recreating config structs for unmarshaling and converting it to the upstream types on-demand. While this is more repetitive than the pkg/traces approach of unmarshaling into a map[string]interface{}, I personally find it preferable since it is more discoverable when examining the code and more likely to fail fast at compile time if upstream changes the type definition unexpectedly.

Example config:

otel "exporter_logging" "default" {
  // Exported fields: 
  // input: <TelemetryConsumer>
}

otel "exporter_otlp" "default" {
  client {
    endpoint = "localhost:4317" // Tempo
  }

  // Exported fields: 
  // input: <TelemetryConsumer>
}

otel "receiver_zipkin" "default" {
  http {}

  output {
    traces = [otel.exporter_logging.default.input, otel.exporter_otlp.default.input]
  }
}

After running, follow Tempo's guide to push Zipkin traces using cURL to see Spans appear in Tempo.

Note that running these components will break the /-/config endpoint, since rfratto/gohcl isn't able to properly serialize a list of capsule values. The fix for that doesn't seem straightforward, and a long-term fix may need to wait for #1839.

[rfratto]

Closing as this was just a prototype to demonstrate the capability of having OpenTelemetry components. A new PR with the intent of being merged can be opened in the future if we decide to add this functionality into Flow.