Added tracegen utility #956

jpkrohling · 2020-09-04T11:51:38Z

Signed-off-by: Juraci Paixão Kröhling juraci@kroehling.de

Description: Added tracegen

Link to tracking Issue: #955

Testing: Manual tests

Documentation: readme included

jpkrohling · 2020-09-07T09:42:15Z

@yurishkuro could you take a look at this? The one notable missing thing is the client metrics. I also wanted to check if the inject/extract part looks sane, as I couldn't really get why the original tracegen is doing that.

yurishkuro

iirc @tigrannajaryan had another load generator that he used for benchmarking OTLP

yurishkuro · 2020-09-07T19:21:40Z

tracegen/internal/tracegen/config.go

@@ -0,0 +1,86 @@
+// Copyright (c) 2018 The Jaeger Authors.


since this is not the code from Jaeger verbatim, a better copyright reference is

// Copyright (c) 2020 The OpenTelemetry Authors. // Copyright (c) 2018 The Jaeger Authors.

This is currently under discussion here open-telemetry/community#475

yurishkuro · 2020-09-07T19:24:38Z

tracegen/internal/tracegen/config.go

+	}
+
+	wg := sync.WaitGroup{}
+	var running uint32 = 1


this one is never used to signal the workers to stop

Isn't it used a few lines below?

if c.Duration > 0 { time.Sleep(c.Duration) atomic.StoreUint32(&running, 0) }

tigrannajaryan · 2020-09-08T22:37:36Z

iirc @tigrannajaryan had another load generator that he used for benchmarking OTLP

Thanks @yurishkuro

@jpkrohling Here is the generator I used in the OTLP benchmarks, it is a primitive one: https://github.com/tigrannajaryan/exp-otelproto/blob/master/encodings/otlp/generator.go

@jpkrohling we also have a more sophisticated Span data generator in https://github.com/open-telemetry/opentelemetry-collector/blob/master/internal/goldendataset/span_generator.go

Depending on what sort of spans you are looking for this one may be a better choice since it can generate a variety of data which is good for testing (but it may also be slower to generate data - not sure by how much).

jpkrohling · 2020-09-09T07:30:36Z

I'm looking primarily to having a tool that is able to generate a specific number of traces with predictable outcome. This tool, ideally a binary packaged in a container, would be used by e2e tests to assert that the whole system is working properly.

Load testing would be a secondary goal.

tigrannajaryan · 2020-09-09T17:53:58Z

I'm looking primarily to having a tool that is able to generate a specific number of traces with predictable outcome. This tool, ideally a binary packaged in a container, would be used by e2e tests to assert that the whole system is working properly.

Load testing would be a secondary goal.

Are we talking about some other e2e tests that are different from e2e tests that we already have in this repo?

jpkrohling · 2020-09-09T19:15:45Z

Potentially, yes. For the Jaeger Operator, we have a different set of e2e tests, especially for the case where we inject a sidecar into a "business application". The same would happen for OpenTelemetry Operator, and the e2e test would make sure that everything is in the right place and the networking is properly configured. The actual business application is irrelevant, hence, something simple like the tracegen.

jpkrohling · 2020-09-09T19:17:03Z

For clarity: this first iteration of the tracegen would not be used for the case I mentioned above, but I do plan on adding a web interface to it.

tigrannajaryan

Blocking until license/copyright issue is clarified.

tigrannajaryan · 2020-09-09T22:43:05Z

tracegen/internal/tracegen/config.go

@@ -0,0 +1,86 @@
+// Copyright (c) 2018 The Jaeger Authors.


This is currently under discussion here open-telemetry/community#475

tracegen/internal/tracegen/worker.go

tracegen/internal/tracegen/config.go

jpkrohling · 2020-09-10T08:34:52Z

Once the copyright part is figured out, I'll address most of the review comments.

tigrannajaryan · 2020-09-17T15:28:45Z

Copyright is clarified, we can move forward.

jpkrohling · 2020-09-18T09:52:13Z

PR updated

jpkrohling · 2020-09-18T19:47:18Z

CI is failing due to flaky tests.

tigrannajaryan · 2020-09-22T14:53:58Z

@jpkrohling please fix CHANGELOG.md conflict. Can you also add some tests for the new code?

codecov · 2020-09-22T15:00:04Z

Codecov Report

Merging #956 into master will increase coverage by 0.87%.
The diff coverage is 78.12%.

@@            Coverage Diff             @@
##           master     #956      +/-   ##
==========================================
+ Coverage   87.97%   88.85%   +0.87%     
==========================================
  Files         251      254       +3     
  Lines       12012    12150     +138     
==========================================
+ Hits        10568    10796     +228     
+ Misses       1109     1006     -103     
- Partials      335      348      +13

Flag	Coverage Δ
#integration	`75.42% <ø> (?)`
#unit	`87.89% <78.12%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
tracegen/internal/tracegen/config.go	`72.22% <72.22%> (ø)`
tracegen/internal/tracegen/worker.go	`85.71% <85.71%> (ø)`
...eiver/awsxrayreceiver/internal/udppoller/poller.go	`97.61% <0.00%> (-2.39%)`	⬇️
receiver/k8sclusterreceiver/watcher.go	`95.29% <0.00%> (-2.36%)`	⬇️
internal/common/testing/container/container.go	`72.97% <0.00%> (ø)`
receiver/dockerstatsreceiver/docker.go	`92.30% <0.00%> (+39.05%)`	⬆️
receiver/dockerstatsreceiver/receiver.go	`96.82% <0.00%> (+49.20%)`	⬆️
receiver/redisreceiver/receiver.go	`87.50% <0.00%> (+87.50%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9082de...6a1a481. Read the comment docs.

tracegen/internal/tracegen/config.go

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

jpkrohling · 2020-09-23T09:48:34Z

Conflict solved, test added.

* Consolidate stdout exporter * Move config to own file and match project standard * Abstract Exporter into unified struct * Rename trace part of the exporter * Update import paths and configuration * Update tests * Update InstallNewPipeline to not return traceProvider It is a registered global, access it that way. * Update example_test * Update docs * Update example to be for whole package * Update metric output Closer match the span output. * Clean up span output Print as a batch and cleanup marshaling. * Correct spelling error in doc * Add Exporters README * Update Changelog * Propagate changes to rest of project * Lint fixes * Fix example test in metric SDK * Add disable config options for trace and metric Co-authored-by: Liz Fong-Jones <lizf@honeycomb.io>

* [instrumentation/wsgi] fix NonRecordingSpan bug There was a bug caused by accessing `.kind` on a NonRecordingSpan. Added a test to validate the fix. Fix #956 * fix lint * use is_recording * fix lint * fix lint * fix lint

jpkrohling requested a review from a team September 4, 2020 11:51

yurishkuro reviewed Sep 7, 2020

View reviewed changes

tigrannajaryan self-assigned this Sep 9, 2020

tigrannajaryan requested changes Sep 9, 2020

View reviewed changes

jpkrohling mentioned this pull request Sep 15, 2020

How to handle MIT licence open-telemetry/community#475

Closed

jpkrohling force-pushed the jpkrohling/issue955 branch from 3981314 to bed0ea2 Compare September 18, 2020 09:52

tigrannajaryan reviewed Sep 22, 2020

View reviewed changes

tracegen/internal/tracegen/config.go Show resolved Hide resolved

jpkrohling added 2 commits September 23, 2020 11:47

Added tracegen utility

034e75b

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Added test

6a1a481

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

jpkrohling force-pushed the jpkrohling/issue955 branch from bed0ea2 to 6a1a481 Compare September 23, 2020 09:47

jpkrohling requested a review from tigrannajaryan September 23, 2020 09:48

jpkrohling mentioned this pull request Sep 23, 2020

Deadlock in groupbytrace processor open-telemetry/opentelemetry-collector#1811

Closed

tigrannajaryan approved these changes Sep 23, 2020

View reviewed changes

tigrannajaryan merged commit 6145a10 into open-telemetry:master Sep 23, 2020

crobert-1 mentioned this pull request Feb 29, 2024

[cmd/telemetrygen] Duration flag is not respected when tls handshake fails #31401

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added tracegen utility #956

Added tracegen utility #956

jpkrohling commented Sep 4, 2020

jpkrohling commented Sep 7, 2020

yurishkuro left a comment

yurishkuro Sep 7, 2020

tigrannajaryan Sep 9, 2020

yurishkuro Sep 7, 2020

jpkrohling Sep 8, 2020

tigrannajaryan commented Sep 8, 2020

jpkrohling commented Sep 9, 2020

tigrannajaryan commented Sep 9, 2020

jpkrohling commented Sep 9, 2020

jpkrohling commented Sep 9, 2020

tigrannajaryan left a comment

tigrannajaryan Sep 9, 2020

jpkrohling commented Sep 10, 2020

tigrannajaryan commented Sep 17, 2020

jpkrohling commented Sep 18, 2020

jpkrohling commented Sep 18, 2020

tigrannajaryan commented Sep 22, 2020

codecov bot commented Sep 22, 2020 •

edited

Loading

jpkrohling commented Sep 23, 2020

Added tracegen utility #956

Added tracegen utility #956

Conversation

jpkrohling commented Sep 4, 2020

jpkrohling commented Sep 7, 2020

yurishkuro left a comment

Choose a reason for hiding this comment

yurishkuro Sep 7, 2020

Choose a reason for hiding this comment

tigrannajaryan Sep 9, 2020

Choose a reason for hiding this comment

yurishkuro Sep 7, 2020

Choose a reason for hiding this comment

jpkrohling Sep 8, 2020

Choose a reason for hiding this comment

tigrannajaryan commented Sep 8, 2020

jpkrohling commented Sep 9, 2020

tigrannajaryan commented Sep 9, 2020

jpkrohling commented Sep 9, 2020

jpkrohling commented Sep 9, 2020

tigrannajaryan left a comment

Choose a reason for hiding this comment

tigrannajaryan Sep 9, 2020

Choose a reason for hiding this comment

jpkrohling commented Sep 10, 2020

tigrannajaryan commented Sep 17, 2020

jpkrohling commented Sep 18, 2020

jpkrohling commented Sep 18, 2020

tigrannajaryan commented Sep 22, 2020

codecov bot commented Sep 22, 2020 • edited Loading

Codecov Report

jpkrohling commented Sep 23, 2020

codecov bot commented Sep 22, 2020 •

edited

Loading