Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MO] - replace EmbeddedKafkaCluster with Strimzi test container #6337

Merged
merged 5 commits into from
Feb 14, 2022

Conversation

see-quick
Copy link
Member

@see-quick see-quick commented Feb 9, 2022

Type of change

  • Enhancement / new feature
  • Refactoring

Description

This PR replaces EmbeddedKafkaCluster with Strimzi test container in the user-operator, cluster-operator and topic-operator module. Thus completely removing dependency for EmbeddedKafkaCluster

Checklist

  • Make sure all tests pass

morsak added 3 commits February 9, 2022 17:40
… - cluster module

Signed-off-by: morsak <xorsak02@stud.fit.vutbr.cz>
… - user module

Signed-off-by: morsak <xorsak02@stud.fit.vutbr.cz>
… - topic module

Signed-off-by: morsak <xorsak02@stud.fit.vutbr.cz>
@see-quick see-quick force-pushed the cluster-operator-module branch from 848ff60 to 3153cd0 Compare February 9, 2022 16:40
@see-quick see-quick added this to the 0.29.0 milestone Feb 9, 2022
@see-quick see-quick self-assigned this Feb 9, 2022
@see-quick see-quick requested a review from scholzj February 9, 2022 16:41
morsak added 2 commits February 9, 2022 18:11
Signed-off-by: morsak <xorsak02@stud.fit.vutbr.cz>
Signed-off-by: morsak <xorsak02@stud.fit.vutbr.cz>
Copy link
Member

@scholzj scholzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits. But LGTM overall.

pom.xml Show resolved Hide resolved
topic-operator/pom.xml Show resolved Hide resolved
@scholzj scholzj requested a review from tombentley February 9, 2022 23:18
Copy link
Member

@tombentley tombentley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits, but otherwise looks good.

cluster = new EmbeddedKafkaCluster(3);
final Map<String, String> kafkaClusterConfiguration = new HashMap<>();
kafkaClusterConfiguration.put("zookeeper.connect", "zookeeper:2181");
cluster = new StrimziKafkaCluster(3, 1, kafkaClusterConfiguration);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use Map.of to make this a 1 liner?

cluster = new EmbeddedKafkaCluster(3);
final Map<String, String> kafkaClusterConfiguration = new HashMap<>();
kafkaClusterConfiguration.put("zookeeper.connect", "zookeeper:2181");
cluster = new StrimziKafkaCluster(3, 1, kafkaClusterConfiguration);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, Map.of

Comment on lines +79 to +81
Map<String, String> p = new HashMap<>();
p.put(KafkaConfig$.MODULE$.AutoCreateTopicsEnableProp(), "false");
p.put("zookeeper.connect", "zookeeper:2181");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Map.of

cluster = new EmbeddedKafkaCluster(1);
cluster.start();
Map<String, String> config = new HashMap<>();
config.put("zookeeper.connect", "zookeeper:2181");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know what I'm going to say, right?

protected static Map<String, String> kafkaClusterConfig() {
Map<String, String> p = new HashMap<>();
p.put(KafkaConfig$.MODULE$.DeleteTopicEnableProp(), "false");
p.put("zookeeper.connect", "zookeeper:2181");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again

return config;
protected static Map<String, String> kafkaClusterConfig() {
Map<String, String> p = new HashMap<>();
p.put(KafkaConfig$.MODULE$.DeleteTopicEnableProp(), "false");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only reason we have a dependency on the kafka broker code? If so, maybe test-container should expose constants for broker configs, to avoid needing that dependency? We probably don't need constants for all the broker configs, just the ones which are typically overridden. Just an idea.


ZkClient zkc = new ZkClient(cluster.zKConnectString());
ZkClient zkc = new ZkClient(cluster.getZookeeper().getHost() + ":" + cluster.getZookeeper().getFirstMappedPort());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a StrimziKafkaCluster.getZookeeper().getConnectString() or similar to avoid the caller having to concatenate this themselves?

"super.users", "User:ANONYMOUS");
kafkaContainer = new StrimziKafkaContainer()
.withBrokerId(1)
.withKafkaConfigurationMap(additionalConfiguration);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless you need a variable, I think it's clearer to inline here.

@see-quick
Copy link
Member Author

Hey @tombentley, ∀ of these suggestions need another release of test-container and I simplicity I would rather just mark ∀ these suggestions with to after f.e. test-container with 0.101.0 is released. WDYT?

@tombentley
Copy link
Member

If you want to open an issue to come back to these after release that's fine with me.

@see-quick see-quick added the ready for merge Label for PRs which are ready for merge label Feb 14, 2022
@scholzj scholzj merged commit 617a83c into strimzi:main Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for merge Label for PRs which are ready for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants