-
Notifications
You must be signed in to change notification settings - Fork 287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sink: set max-message-bytes to initialize producer. (#3080) #3106
sink: set max-message-bytes to initialize producer. (#3080) #3106
Conversation
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
[REVIEW NOTIFICATION] This pull request has not been approved. To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
/run-integration-tests |
Codecov Report
@@ Coverage Diff @@
## release-4.0 #3106 +/- ##
===================================================
+ Coverage 53.7661% 57.1978% +3.4316%
===================================================
Files 153 164 +11
Lines 15958 19485 +3527
===================================================
+ Hits 8580 11145 +2565
- Misses 6475 7281 +806
- Partials 903 1059 +156 |
/merge |
This pull request has been accepted and is ready to merge. Commit hash: 36841c2
|
This is an automated cherry-pick of #3080
What problem does this PR solve?
Close #3081
This is a fix after #3002.
In #3002, we set 512M as the minimum
max-message-size
to initialize the sarama producer, which would batch small message then larger than broker'smessage.max.bytes
, make the message failed to send.If the user does not pass
max-message-bytes
when creating then changefeed, the producer use 512M, it's very likely larger than the broker's value. so we have to set the default value smaller, to 1M.Also, set encoder's defaultMaxMessageBytes to 1M, to prevent encoded message too large, then reject by the client.
What is changed and how it works?
max-message-size
to initialize the producer.MaxMessageSize
to 1M.Check List
Tests
Code changes
Side effects
Related changes
Release note