-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Fix up to a 1s delay in first produce to new topic #5032
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (1)
tests/0055-producer_latency.c:496
- [nitpick] The variable 'wait_time_mss' appears ambiguous; consider renaming it to 'wait_times_ms' for clarity.
int wait_time_mss[] = {500, 1200, 2500, 3700, 4900};
a500cd0 to
7cf8368
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (1)
tests/0055-producer_latency.c:500
- [nitpick] The variable name 'wait_time_mss' appears to contain a typo; consider renaming it to 'wait_time_ms' for clarity.
int wait_time_mss[] = {500, 1200, 2500, 3700, 4900};
emasab
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix Milind, sent some improvements
tests/0055-producer_latency.c
Outdated
| rd_kafka_conf_set_dr_msg_cb(conf, dr_msg_cb_first_message); | ||
| rk = test_create_handle(RD_KAFKA_PRODUCER, conf); | ||
|
|
||
| switch (case_number) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's repeat this 50 times instead of changing the sleep. Every time with a different topic name. this way we can also evaluate the number of metadata requests done
a32284a to
7638b5f
Compare
emasab
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Milind! Approving
Currently, when creating a new rkt via the public api, rd_kafka_topic_new(), causes us to immediately make a leader query if the topic is unknown.
On the other hand, using the internal rd_kafka_topic_new0() does not do this. producev and produceva both call rd_kafka_topic_new0 in case topic is provided by name. So this delays produce until scan_all is run, which runs on a 1s timer.
Definitely affects JS client and I think it affects python client and dotnet client as well.