Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kafka OTEL receiver/ingester #2221

Merged
merged 3 commits into from
May 6, 2020

Conversation

pavolloffay
Copy link
Member

Related to #2205

This PR adds Kafka OTEL receiver. This receiver will be used in OTEL ingester that will come in a follow-up PR.

PR for kafka exporter #2135

@pavolloffay pavolloffay requested a review from a team as a code owner May 5, 2020 16:04
@pavolloffay pavolloffay requested review from vprithvi and removed request for vprithvi May 5, 2020 16:04
@pavolloffay
Copy link
Member Author

@rubenvp8510 or and @objectiser could you please review?

@codecov
Copy link

codecov bot commented May 5, 2020

Codecov Report

Merging #2221 into master will increase coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2221      +/-   ##
==========================================
+ Coverage   96.16%   96.20%   +0.03%     
==========================================
  Files         219      219              
  Lines       10632    10632              
==========================================
+ Hits        10224    10228       +4     
+ Misses        352      350       -2     
+ Partials       56       54       -2     
Impacted Files Coverage Δ
cmd/ingester/app/flags.go 100.00% <ø> (ø)
cmd/query/app/static_handler.go 88.59% <0.00%> (+1.75%) ⬆️
cmd/query/app/server.go 94.52% <0.00%> (+2.73%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cdaea8...69ec7d6. Read the comment docs.

Copy link
Contributor

@objectiser objectiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - just some minor comments.

Main question, not for this PR, is how we work with/transition to a pure OTel/Kafka impl - would be better if using OTel model for kafka messages.

cmd/opentelemetry-collector/app/defaults/defaults.go Outdated Show resolved Hide resolved

kafkaCfg := cfg.Receivers[TypeStr].(*Config)
assert.Equal(t, TypeStr, kafkaCfg.Name())
assert.Equal(t, "jaeger-prod", kafkaCfg.Topic)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to allow one of the flag values to be passed to the final config?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kafkaCfg.Options.Kerberos.Username is passed from jaeger config file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that none of the CLI flag values are making it into the final config - would be good if atleast one did for completeness.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flags or config file are the same source for viper. A couple of lines above we test that flags are passed to the factory.

@pavolloffay
Copy link
Member Author

Main question, not for this PR, is how we work with/transition to a pure OTel/Kafka impl - would be better if using OTel model for kafka messages.

Yeah, this is not related to this PR, but it is definitely something on the roadmap.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Copy link
Contributor

@objectiser objectiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of comments to consider.

cmd/opentelemetry-collector/app/defaults/defaults.go Outdated Show resolved Hide resolved

kafkaCfg := cfg.Receivers[TypeStr].(*Config)
assert.Equal(t, TypeStr, kafkaCfg.Name())
assert.Equal(t, "jaeger-prod", kafkaCfg.Topic)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that none of the CLI flag values are making it into the final config - would be good if atleast one did for completeness.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay pavolloffay merged commit 16d7e09 into jaegertracing:master May 6, 2020
pavolloffay added a commit to pavolloffay/jaeger that referenced this pull request May 6, 2020
* Add Kafka OTEL receiver/ingester

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Fix review

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Use tls.cert from flags

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants