Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-17543: Improve and clarify the error message about generated broker IDs in migration #17210

Conversation

scholzj
Copy link
Member

@scholzj scholzj commented Sep 16, 2024

This PR tries to improve the error message when broker.id is set to -1 when migration is enabled. It is not needed to disable the broker.id.generation.enable option. It is sufficient to just not use it (by not setting the broker.id to -1). This was discussed in

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

…ion should not be used, it does not have to be disabled)

Signed-off-by: Jakub Scholz <www@scholzj.com>
Copy link
Contributor

@chia7712 chia7712 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scholzj thanks for your patch

@@ -967,7 +967,7 @@ class KafkaConfig private(doLog: Boolean, val props: util.Map[_, _])
// ZK-based
if (migrationEnabled) {
require(brokerId >= 0,
"broker broker.id.generation.enable is incompatible with migration to ZK. Please disable it before enabling migration")
"broker.id generation is incompatible with ZooKeeper migration. Please stop using it before enabling migration (set broker.id to a value greater or equal to 0).")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this path is unrelated to broker.id generation, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure why it is here -> I just updated the message. The test does not trigger it. But to be honest, my Kafka code knowledge is not good enough to understand if under some circumstances it might get here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path and the above one (L850) are both under validateValues method. The above one is validate for ZK brokers (i.e. process.role is null). And this line is to validate KRaft, or migrating brokers. Both are related to this fix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should just say "please define the broker.id for migration"? It can fail due to undefined broker.if even though we have disabled generation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean in the error message? Something like Please define the broker.id for ZooKeeper migration?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never mind. Current version is good enough. I will merge it later

Copy link
Contributor

@showuon showuon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -967,7 +967,7 @@ class KafkaConfig private(doLog: Boolean, val props: util.Map[_, _])
// ZK-based
if (migrationEnabled) {
require(brokerId >= 0,
"broker broker.id.generation.enable is incompatible with migration to ZK. Please disable it before enabling migration")
"broker.id generation is incompatible with ZooKeeper migration. Please stop using it before enabling migration (set broker.id to a value greater or equal to 0).")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path and the above one (L850) are both under validateValues method. The above one is validate for ZK brokers (i.e. process.role is null). And this line is to validate KRaft, or migrating brokers. Both are related to this fix.

core/src/main/scala/kafka/server/KafkaConfig.scala Outdated Show resolved Hide resolved
Signed-off-by: Jakub Scholz <www@scholzj.com>
Signed-off-by: Jakub Scholz <www@scholzj.com>
@chia7712
Copy link
Contributor

3.9 RC is almost there and this PR should be backport to 3.9, hence I will wait for RC 1 ...

@cmccabe cmccabe merged commit 09e3c12 into apache:trunk Sep 18, 2024
8 of 9 checks passed
cmccabe pushed a commit that referenced this pull request Sep 18, 2024
…oker IDs in migration (#17210)

This PR tries to improve the error message when broker.id is set to -1 and ZK migration is enabled. It is not
needed to disable the broker.id.generation.enable option. It is sufficient to just not use it (by not setting
the broker.id to -1).

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Luke Chen <showuon@gmail.com>
@scholzj scholzj deleted the improve-error-message-when-generated-broker-id-is-enabled-during-migration branch September 18, 2024 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants