OpenTelemetry Output Plugin #9109

codeboten · 2021-04-08T21:39:49Z

Required for all PRs:

Updated associated README.md.
Wrote appropriate unit tests.

resolves #9108

This change takes the Stackdriver output plugin as a template and outputs the data in the OpenTelemetry protocol instead. It's currently state still needs:

refactoring to not bring in the prometheus sidecar OTLP client (this was done as part of the prototyping, I plan on removing this before marking the PR as ready for review)

telegraf-tiger

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

codeboten · 2021-04-08T22:28:49Z

!signed-cla

telegraf-tiger

🤝 ✅ CLA has been signed. Thank you!

…tput-plugin

telegraf-tiger

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

…tput-plugin

telegraf-tiger

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

README.md

codeboten · 2021-04-23T20:20:49Z

Addressed the new comments, it looks like the failing test is caused by a race condition happening in a different plugin.

…tput-plugin

telegraf-tiger · 2021-04-27T17:18:28Z

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

srebhan · 2021-04-28T12:29:12Z

plugins/outputs/opentelemetry/client.go

+func (c *client) ping(ctx context.Context) error {
+	// Loop until the context is canceled, allowing for retryable failures.
+	for {
+		err := c.connect(ctx)


To be honest, this is somewhat unexpected from my perspective. The task of this function is to test connectivity, so I'm not sure if it's a good idea to connect. However, if everybody else accepts this as is, I can live with it.

srebhan

Very nice. Can you please say a few words regarding the possibility to use the v0.19 interface of the upstream client and thus get rid of most of your client implementation!?
Another comment is about connecting in ping. To me it's unexpected, but this is not a blocker for me.

…tput-plugin

telegraf-tiger · 2021-04-29T19:38:06Z

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

codeboten · 2021-04-29T23:42:37Z

Hey @srebhan

Thanks for all the reviews! With regards to using the OTLP exporter instead of the current client code, in order to utilize it, it would require creating various implementing various components specified in the SDK (CheckpointSet, Aggregator) which are likely to change in the next iteration of the opentelemetry metrics signal specification. There's no exposed interface that allows me to use the protobuf more directly and pass in the slice of ResourceMetrics.

If you're ok w/ the current implementation, I would be happy to refactor the client code in the future, once the metrics API/SDK has stabilized. I just don't want to implement something that's not trivial (I spent some hours trying today, and i'm not sure the code is any simpler) and have to throw it away in the near future.

I've updated the code to remove the separate ping function, and instead incorporated the check into the connect function.

…tput-plugin

telegraf-tiger · 2021-04-30T17:35:11Z

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

codeboten · 2021-05-03T17:45:20Z

Looks like there's another PR implementing this as well #9228

…tput-plugin

telegraf-tiger · 2021-05-04T18:29:31Z

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

codeboten · 2021-05-07T20:33:31Z

@srebhan not sure how to move this PR forward here, any help is much appreciated.

sjwang90 · 2021-05-07T23:13:20Z

@srebhan and some other Telegraf engineers will discuss the two OpenTelemetry output plugin PRs next week. We should have an answer for you @codeboten by Weds - I believe there's nothing we need from your right now.

lizthegrey · 2021-05-07T23:22:45Z

FYI @puckpuck and @bdarfler

codeboten · 2021-05-08T00:18:28Z

Thanks @sjwang90!

sjwang90 · 2021-05-12T17:24:27Z

@codeboten The team would like to proceed with #9228 and that PR is close to being merged. If you would like to take a review at the PR and address any concerns.

Otherwise, thanks so much for your contribution. Either way we should be getting this OpenTelemetry Output plugin into master very soon!

codeboten · 2021-05-12T21:19:56Z

Thanks @sjwang90, I tested the other PR and couldn't see any data being output to the OpenTelemetry collector. See my comment here:
#9228 (comment)

Running the same test I did with the other PR with the code in this PR I see the following output:

otel-collector              | Metric #0
otel-collector              | Descriptor:
otel-collector              |      -> Name: processes.unknown
otel-collector              |      -> Description:
otel-collector              |      -> Unit:
otel-collector              |      -> DataType: IntGauge
otel-collector              | IntDataPoints #0
otel-collector              | Data point labels:
otel-collector              |      -> host: Alexs-MacBook-Pro.local
otel-collector              | StartTime: 0
otel-collector              | Timestamp: 1620854350000000000
otel-collector              | Value: 0
otel-collector              | ResourceMetrics #203
otel-collector              | InstrumentationLibraryMetrics #0
otel-collector              | InstrumentationLibrary telegraf unknown
otel-collector              | Metric #0
otel-collector              | Descriptor:
otel-collector              |      -> Name: processes.idle
otel-collector              |      -> Description:
otel-collector              |      -> Unit:
otel-collector              |      -> DataType: IntGauge
otel-collector              | IntDataPoints #0
otel-collector              | Data point labels:
otel-collector              |      -> host: Alexs-MacBook-Pro.local
otel-collector              | StartTime: 0
otel-collector              | Timestamp: 1620854350000000000
otel-collector              | Value: 0
otel-collector              | ResourceMetrics #204
otel-collector              | InstrumentationLibraryMetrics #0
otel-collector              | InstrumentationLibrary telegraf unknown
otel-collector              | Metric #0
otel-collector              | Descriptor:
otel-collector              |      -> Name: processes.blocked
otel-collector              |      -> Description:
otel-collector              |      -> Unit:
otel-collector              |      -> DataType: IntGauge
otel-collector              | IntDataPoints #0
otel-collector              | Data point labels:
otel-collector              |      -> host: Alexs-MacBook-Pro.local
otel-collector              | StartTime: 0
otel-collector              | Timestamp: 1620854350000000000
otel-collector              | Value: 0

sjwang90 · 2021-05-17T18:16:55Z

@codeboten Thanks for your feedback. @jacobmarble is going to take a look at your comments. Let's provide any updates on #9228 to keep it all the discussion in the same place.

sjwang90 · 2021-08-04T21:37:00Z

Closing since #9228 has been merged. Thanks for help!

initial checkin of otlp output plugin

bd54546

telegraf-tiger bot added new plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins labels Apr 8, 2021

telegraf-tiger bot requested changes Apr 8, 2021

View reviewed changes

telegraf-tiger bot approved these changes Apr 8, 2021

View reviewed changes

Alex Boten added 8 commits April 8, 2021 17:31

update to use attributes from config

7dc413a

removing prom sidecar dependency

bba15cf

add tests around configuration

496e080

adding write test

0eff888

update dependencies list

c67d18f

adding more tests

1e6b5e4

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

66f3e31

…tput-plugin

mod tidy

2c0d9ec

telegraf-tiger bot reviewed Apr 9, 2021

View reviewed changes

Alex Boten added 5 commits April 9, 2021 15:13

lint fixes

a8bf7c3

fix list of dependencies

b0e7b92

remove use of log

6331204

fix test

dd9c8dc

adding client tests

90901ec

codeboten mentioned this pull request Apr 16, 2021

feat(influx2otel): add influx->otel package influxdata/influxdb-observability#1

Merged

Alex Boten added 3 commits April 16, 2021 15:39

minor cleanup

81772d7

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

a44744f

…tput-plugin

mod tidy

1aa9a00

codeboten marked this pull request as ready for review April 16, 2021 22:48

telegraf-tiger bot reviewed Apr 16, 2021

View reviewed changes

Alex Boten added 3 commits April 16, 2021 16:17

fix example

464c2d4

update default endpoint

7648ce6

cleaning up tests

357d793

srebhan reviewed Apr 19, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

Alex Boten added 5 commits April 23, 2021 08:24

review feedback

f07e4b9

cleanup go.mod

52b685b

tidy

3ac6824

fix dependencies list

6a585e1

fix test

86704fb

Alex Boten added 2 commits April 27, 2021 09:45

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

b3d87fe

…tput-plugin

tidy

84adf2b

srebhan reviewed Apr 28, 2021

View reviewed changes

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

aed6860

…tput-plugin

remove separate ping function

8fedd60

Alex Boten added 2 commits April 30, 2021 08:13

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

37f85cd

…tput-plugin

update otlp import

b447006

Merge remote-tracking branch 'upstream/master' into codeboten/otlp-ou…

8aee642

…tput-plugin

sjwang90 closed this Aug 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenTelemetry Output Plugin #9109

OpenTelemetry Output Plugin #9109

codeboten commented Apr 8, 2021 •

edited

Loading

telegraf-tiger bot left a comment

codeboten commented Apr 8, 2021

telegraf-tiger bot left a comment

telegraf-tiger bot left a comment

telegraf-tiger bot left a comment

codeboten commented Apr 23, 2021

telegraf-tiger bot commented Apr 27, 2021

Artifact URLs

srebhan Apr 28, 2021

srebhan left a comment

telegraf-tiger bot commented Apr 29, 2021

Artifact URLs

codeboten commented Apr 29, 2021

telegraf-tiger bot commented Apr 30, 2021

Artifact URLs

codeboten commented May 3, 2021

telegraf-tiger bot commented May 4, 2021

Artifact URLs

codeboten commented May 7, 2021

sjwang90 commented May 7, 2021

lizthegrey commented May 7, 2021

codeboten commented May 8, 2021

sjwang90 commented May 12, 2021

codeboten commented May 12, 2021

sjwang90 commented May 17, 2021

sjwang90 commented Aug 4, 2021 •

edited

Loading

OpenTelemetry Output Plugin #9109

OpenTelemetry Output Plugin #9109

Conversation

codeboten commented Apr 8, 2021 • edited Loading

Required for all PRs:

telegraf-tiger bot left a comment

Choose a reason for hiding this comment

codeboten commented Apr 8, 2021

telegraf-tiger bot left a comment

Choose a reason for hiding this comment

telegraf-tiger bot left a comment

Choose a reason for hiding this comment

Artifact URLs

telegraf-tiger bot left a comment

Choose a reason for hiding this comment

Artifact URLs

codeboten commented Apr 23, 2021

telegraf-tiger bot commented Apr 27, 2021

Artifact URLs

srebhan Apr 28, 2021

Choose a reason for hiding this comment

srebhan left a comment

Choose a reason for hiding this comment

telegraf-tiger bot commented Apr 29, 2021

Artifact URLs

codeboten commented Apr 29, 2021

telegraf-tiger bot commented Apr 30, 2021

Artifact URLs

codeboten commented May 3, 2021

telegraf-tiger bot commented May 4, 2021

Artifact URLs

codeboten commented May 7, 2021

sjwang90 commented May 7, 2021

lizthegrey commented May 7, 2021

codeboten commented May 8, 2021

sjwang90 commented May 12, 2021

codeboten commented May 12, 2021

sjwang90 commented May 17, 2021

sjwang90 commented Aug 4, 2021 • edited Loading

codeboten commented Apr 8, 2021 •

edited

Loading

sjwang90 commented Aug 4, 2021 •

edited

Loading