KAFKA-19596: Improve visibility when topic auto-creation fails #20340
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
triage
Contributor
Author
|
For example, if I:
Now I will see logs in the server like: |
showuon
reviewed
Aug 12, 2025
Member
showuon
left a comment
showuon
left a comment
There was a problem hiding this comment.
It looks like the AutoTopicCreateManager just wants to try to create the topics once without retry, which is fine to me because users can create the topic manually later. Left a comment. Thanks.
| warn(s"Auto topic creation failed for ${topicResult.name()} with error '${error.name()}': ${topicResult.errorMessage()}") | ||
| } | ||
| }) | ||
| case _ => // do nothing |
Member
There was a problem hiding this comment.
Don't you think this should be also an error we should log?
Contributor
Author
There was a problem hiding this comment.
true, have added a warn log in this case
robobario
force-pushed
the
stronger-log
branch
from
8cff9ef to
f22ae59
Compare
showuon
approved these changes
Aug 12, 2025
Member
showuon
left a comment
showuon
left a comment
There was a problem hiding this comment.
LGTM! Thanks for the improvement.
brandboat
approved these changes
Aug 12, 2025
Member
brandboat
left a comment
brandboat
left a comment
There was a problem hiding this comment.
LGTM, thanks for the enhancement!
| createTopicsResponse.data().topics().forEach(topicResult => { | ||
| val error = Errors.forCode(topicResult.errorCode()) | ||
| if (error != Errors.NONE) { | ||
| warn(s"Auto topic creation failed for ${topicResult.name()} with error '${error.name()}': ${topicResult.errorMessage()}") |
Member
There was a problem hiding this comment.
tiny nit: we can skip () for getters
Suggested change
| warn(s"Auto topic creation failed for ${topicResult.name()} with error '${error.name()}': ${topicResult.errorMessage()}") | |
| warn(s"Auto topic creation failed for ${topicResult.name} with error '${error.name}': ${topicResult.errorMessage}") |
Contributor
Author
There was a problem hiding this comment.
thanks, have killed off ()
Log a warning for each topic that failed to be created as a result of an automatic creation. Previously, at the default log level, you could only see logs that the broker was attempting to autocreate topics. If the creation failed it was logged at debug. This makes it more visible. Signed-off-by: Robert Young <robertyoungnz@gmail.com>
robobario
force-pushed
the
stronger-log
branch
from
f22ae59 to
5924673
Compare
chia7712
reviewed
Aug 25, 2025
| createTopicsResponse.data().topics().forEach(topicResult => { | ||
| val error = Errors.forCode(topicResult.errorCode) | ||
| if (error != Errors.NONE) { | ||
| warn(s"Auto topic creation failed for ${topicResult.name} with error '${error.name}': ${topicResult.errorMessage}") |
Member
There was a problem hiding this comment.
Have you considered logging the request to provide a more useful debug message?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Log a warning for each topic that failed to be created as a result of an automatic creation. This makes the underlying cause more visible so users can take action.
Previously, at the default log level, you could only see logs that the broker was attempting to autocreate topics. If the creation failed, then it was logged at debug.