component/otelcol/receiver: create receiver component abstraction #2254

rfratto · 2022-09-30T19:31:26Z

This commit introduces a new package, component/otelcol/receiver, which exposes a generic Flow component implementation which can run OpenTelemetry Collector receiver.

Like #2227, it leaves some work unfinished for future PRs:

A Zap logging adapter needs to be created to correctly process logs from OpenTelemetry Collector components.
Component-specific metrics are currently ignored.
Component-specific traces are currently ignored.

As of this commit, there are no registered otelcol.receiver.* components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs.

Related to #2213. A third PR for component/otelcol/processor will be made following this one.

This commit introduces a new package, component/otelcol/receiver, which exposes a generic Flow component implementation which can run OpenTelemetry Collector receiver. Like grafana#2227, it leaves some work unfinished for future PRs: * A Zap logging adapter needs to be created to correctly process logs from OpenTelemetry Collector components. * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.receiver.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to grafana#2213.

tpaschalis

Looks clean!

tpaschalis · 2022-10-03T12:41:20Z

component/otelcol/consumer.go

+type ConsumerArguments struct {
+	Metrics []Consumer `river:"metrics,attr,optional"`
+	Logs    []Consumer `river:"logs,attr,optional"`
+	Traces  []Consumer `river:"traces,attr,optional"`


We'll always use the fanoutconsumer package implementations for these internally, right?

No, fanoutconsumer is only used when passing the list of consumers to the OpenTelemetry code. These will instead almost always be conusmers from the lazyconsumer package (unless one day there's different components which are otelcol-compatible that have their own approach for exposing consumers)

mattdurham

LGTM

This commit introduces a new package, component/otelcol/processor, which exposes a generic Flow component implementation which can run OpenTelemetry Collector processor. Like grafana#2227 and grafana#2254, it leaves some work unfinished for future PRs: * A Zap logging adapter needs to be created to correctly process logs from OpenTelemetry Collector components. * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.processor.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to grafana#2213.

…2284) This commit introduces a new package, component/otelcol/processor, which exposes a generic Flow component implementation which can run OpenTelemetry Collector processor. Like #2227 and #2254, it leaves some work unfinished for future PRs: * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.processor.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to #2213.

OpenTelemetry supports many extensions. Extensions are used as a generic way to put "everything else" into OpenTelemetry. There are two types of extensions relevant to us: * [Authentication extensions][auth-ext]: used for both client and server authentication. * [Storage extensions][storage-ext]: used for external storage of state. Other extensions, such as [awsproxy][] are useful but better suited as generic Flow components rather than being shoved in the otelcol namespace, since they are unrelated to telemetry pipelines and aren't referenced by other otelcol components in the upstream configuration. This commit introduces a new package, component/otelcol/auth, which exposes a generic Flow component implementation which can run OpenTelemetry Collector extensions meant for authentication. While storage extensions may end up being Flow components eventually, it's currently marked as experimental upstream. We will reevaluate storage extension components once things have stabilized a little more. Like grafana#2227, grafana#2254, and grafana#2284, it leaves some work unfinished for future PRs: * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.auth.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to grafana#2213. [auth-ext]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/config/configauth [storage-ext]: https://pkg.go.dev/go.opentelemetry.io/collector/extension/experimental/storage [awsproxy]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.61.0/extension/awsproxy

OpenTelemetry supports many extensions. Extensions are used as a generic way to put "everything else" into OpenTelemetry. To quote their [documentation][ext-docs]: > Extension is the interface for objects hosted by the OpenTelemetry > Collector that don't participate directly on data pipelines but provide > some functionality to the service, examples: health check endpoint, > z-pages, etc. There are two types of extensions relevant to us: * [Authentication extensions][auth-ext]: used for both client and server authentication. * [Storage extensions][storage-ext]: used for external storage of state. Other extensions, such as [awsproxy][] are useful but better suited as generic Flow components rather than being shoved in the otelcol namespace, since they are unrelated to telemetry pipelines and aren't referenced by other otelcol components in the upstream configuration. This commit introduces a new package, component/otelcol/auth, which exposes a generic Flow component implementation which can run OpenTelemetry Collector extensions meant for authentication. While storage extensions may end up being Flow components eventually, it's currently marked as experimental upstream. We will reevaluate storage extension components once things have stabilized a little more. Like grafana#2227, grafana#2254, and grafana#2284, it leaves some work unfinished for future PRs: * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.auth.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to grafana#2213. [ext-docs]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/component#Extension [auth-ext]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/config/configauth [storage-ext]: https://pkg.go.dev/go.opentelemetry.io/collector/extension/experimental/storage [awsproxy]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.61.0/extension/awsproxy

…2352) OpenTelemetry supports many extensions. Extensions are used as a generic way to put "everything else" into OpenTelemetry. To quote their [documentation][ext-docs]: > Extension is the interface for objects hosted by the OpenTelemetry > Collector that don't participate directly on data pipelines but provide > some functionality to the service, examples: health check endpoint, > z-pages, etc. There are two types of extensions relevant to us: * [Authentication extensions][auth-ext]: used for both client and server authentication. * [Storage extensions][storage-ext]: used for external storage of state. Other extensions, such as [awsproxy][] are useful but better suited as generic Flow components rather than being shoved in the otelcol namespace, since they are unrelated to telemetry pipelines and aren't referenced by other otelcol components in the upstream configuration. This commit introduces a new package, component/otelcol/auth, which exposes a generic Flow component implementation which can run OpenTelemetry Collector extensions meant for authentication. While storage extensions may end up being Flow components eventually, it's currently marked as experimental upstream. We will reevaluate storage extension components once things have stabilized a little more. Like #2227, #2254, and #2284, it leaves some work unfinished for future PRs: * Component-specific metrics are currently ignored. * Component-specific traces are currently ignored. As of this commit, there are no registered `otelcol.auth.*` components. Implementations for OpenTelemetry Collector Flow components will be done in future PRs. Related to #2213. [ext-docs]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/component#Extension [auth-ext]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/config/configauth [storage-ext]: https://pkg.go.dev/go.opentelemetry.io/collector@v0.61.0/extension/experimental/storage [awsproxy]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.61.0/extension/awsproxy

rfratto requested review from mattdurham and tpaschalis September 30, 2022 19:31

rfratto added 2 commits September 30, 2022 15:35

fix lint errors

693ca54

fix data race

041ced7

tpaschalis approved these changes Oct 3, 2022

View reviewed changes

tpaschalis reviewed Oct 3, 2022

View reviewed changes

mattdurham approved these changes Oct 3, 2022

View reviewed changes

rfratto merged commit 51dc11e into grafana:main Oct 3, 2022

rfratto deleted the otelcol-receiver branch October 3, 2022 14:13

rfratto mentioned this pull request Oct 3, 2022

component/otelcol/processor: create processor component abstraction #2284

Merged

rfratto mentioned this pull request Oct 12, 2022

component/otelcol/auth: create auth extension component abstraction #2352

Merged

github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Mar 17, 2024

github-actions bot locked as resolved and limited conversation to collaborators Mar 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

component/otelcol/receiver: create receiver component abstraction #2254

component/otelcol/receiver: create receiver component abstraction #2254

rfratto commented Sep 30, 2022 •

edited

Loading

tpaschalis left a comment

tpaschalis Oct 3, 2022

rfratto Oct 3, 2022

mattdurham left a comment

component/otelcol/receiver: create receiver component abstraction #2254

component/otelcol/receiver: create receiver component abstraction #2254

Conversation

rfratto commented Sep 30, 2022 • edited Loading

tpaschalis left a comment

Choose a reason for hiding this comment

tpaschalis Oct 3, 2022

Choose a reason for hiding this comment

rfratto Oct 3, 2022

Choose a reason for hiding this comment

mattdurham left a comment

Choose a reason for hiding this comment

rfratto commented Sep 30, 2022 •

edited

Loading