Add support for kafka 2.0.0 #8399
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
houndci-bot
reviewed
Sep 23, 2018
libbeat/common/kafka/version.go
Outdated
| return nil | ||
| } | ||
|
|
||
| func (v *Version) Get() (sarama.KafkaVersion, bool) { |
There was a problem hiding this comment.
exported method Version.Get should have comment or be unexported
libbeat/common/kafka/version.go
Outdated
| return nil | ||
| } | ||
|
|
||
| func (v *Version) Unpack(s string) error { |
There was a problem hiding this comment.
exported method Version.Unpack should have comment or be unexported
libbeat/common/kafka/version.go
Outdated
| @@ -78,3 +88,26 @@ func parseKafkaVersion(s string) sarama.KafkaVersion { | |||
| } | |||
| return v | |||
| } | |||
|
|
|||
| func (v *Version) Validate() error { | |||
There was a problem hiding this comment.
exported method Version.Validate should have comment or be unexported
libbeat/common/kafka/version.go
Outdated
| "github.com/Shopify/sarama" | ||
| ) | ||
|
|
||
| type Version struct { |
There was a problem hiding this comment.
exported type Version should have comment or be unexported
jsoriano
added
module
libbeat
Metricbeat
ph
reviewed
Sep 24, 2018
libbeat/common/kafka/version.go
Outdated
| // Version is a kafka version | ||
| type Version struct { | ||
| String string | ||
| } |
There was a problem hiding this comment.
We could have the same features of the validation by using type Version string without having to have a String Field.
libbeat/outputs/kafka/config.go
Outdated
| @@ -96,7 +97,7 @@ func defaultConfig() kafkaConfig { | |||
| BrokerTimeout: 10 * time.Second, | |||
| Compression: "gzip", | |||
| CompressionLevel: 4, | |||
| Version: "1.0.0", | |||
| Version: kafka.Version{String: "1.0.0"}, | |||
There was a problem hiding this comment.
+1 to have the default previous behavior, I was looking into that.
| @@ -4,7 +4,7 @@ ENV KAFKA_HOME /kafka | |||
| # The advertised host is kafka. This means it will not work if container is started locally and connected from localhost to it | |||
| ENV KAFKA_ADVERTISED_HOST kafka | |||
| ENV KAFKA_LOGS_DIR="/kafka-logs" | |||
| ENV KAFKA_VERSION 1.1.1 | |||
| ENV KAFKA_VERSION 2.0.0 | |||
There was a problem hiding this comment.
It would be still pending to test the output with multiple versions, but I leave this for a future change, maybe related with #7957
ph
suggested changes
Sep 24, 2018
ph
approved these changes
Sep 24, 2018
jsoriano
removed
the
needs_backport
jsoriano
added a commit
to jsoriano/beats
that referenced
this pull request
Sep 27, 2018
Add support to kafka 2.0.0 in kafka output and metricbeat module. Merge kafka versioning helpers of output and metricbeat module. Set version in kafka module configuration of metricbeat system tests (cherry picked from commit 1bfd445)
jsoriano
added a commit
that referenced
this pull request
Oct 1, 2018
…8467) Test metricbeat kafka module with kafka 2.0.0 Upgrade sarama client to 1.18.0. Add support to kafka 2.0.0 in kafka output and metricbeat module. Merge kafka versioning helpers of output and metricbeat module. Set version in kafka module configuration of metricbeat system tests (cherry picked from commits 7635731, 9749f3d and 1bfd445)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #8330 and #7992, fixes #6823