Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration tests for kafka sink hang for Kafka 0.11.0.0 #1984

Closed
ghost opened this issue Mar 4, 2020 · 1 comment · Fixed by #15722
Closed

Integration tests for kafka sink hang for Kafka 0.11.0.0 #1984

ghost opened this issue Mar 4, 2020 · 1 comment · Fixed by #15722
Labels
domain: tests Anything related to Vector's internal tests have: must We must have this feature, it is critical to the project's success. It is high priority. sink: kafka Anything `kafka` sink related type: bug A code related bug.

Comments

@ghost
Copy link

ghost commented Mar 4, 2020 •
edited by ghost
Loading

We use the latest version of wurstmeister/kafka Docker image for the integration tests. Currently it corresponds to Kafka 2.12.

However, it turns out that the integration tests pass only for Kafka 2.x. Setting the version tag in docker-compose.yml to any version earlier than 2.x makes the integration tests hang.

Because Vector claims to support all versions of Kafka since 0.9, we need to fix this and potentially add integration tests using earlier versions of Kafka to CI.
@ghost ghost added the sink: kafka Anything `kafka` sink related label Mar 4, 2020
@ghost ghost mentioned this issue Mar 4, 2020
Closed
@binarylogic
Copy link
Contributor

I'd say this is high priority. To add to this, we should be erroring when an unsupported version is detected. I'm curious about general strategies around this. Ex: doing the same with Elasticsearch and so on.

@binarylogic binarylogic assigned ghost Mar 4, 2020
@binarylogic binarylogic added domain: docs domain: tests Anything related to Vector's internal tests labels Mar 4, 2020
@ghost ghost changed the title Integration tests for Kafka sinks hang for Kafka 0.11.0.0 Integration tests for kafka sink hang for Kafka 0.11.0.0 Mar 4, 2020
@binarylogic binarylogic unassigned ghost Apr 15, 2020
@binarylogic binarylogic added have: must We must have this feature, it is critical to the project's success. It is high priority. type: bug A code related bug. labels Aug 7, 2020
@jamtur01 jamtur01 added type: bug A code related bug. and removed type: bug A code related bug. labels Sep 10, 2020
jszwedko added a commit that referenced this issue Dec 23, 2022
@jszwedko
chore(kafka): Update Kafka support
bcd431f 
Updating integration tests to target oldest non-EOL Kafka version (using Confluent as the benchmark)
and docs to match.

Closes: #1984

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
@jszwedko jszwedko mentioned this issue Dec 23, 2022
Merged
jszwedko added a commit that referenced this issue Dec 23, 2022
@jszwedko
chore(kafka): Update Kafka support (#15722)
a29c5fb 
Updating integration tests to target oldest non-EOL Kafka version (using Confluent as the benchmark)
and docs to match.

Closes: #1984

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
domain: tests Anything related to Vector's internal tests have: must We must have this feature, it is critical to the project's success. It is high priority. sink: kafka Anything `kafka` sink related type: bug A code related bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(kafka): Update Kafka support vectordotdev/vector
2 participants
@jamtur01 @binarylogic