Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

codec(cdc): fix encoder max-message-bytes (#4074) #4077

Merged
merged 46 commits into from
Dec 27, 2021
Merged

codec(cdc): fix encoder max-message-bytes (#4074) #4077

merged 46 commits into from
Dec 27, 2021

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #4074

What problem does this PR solve?

when initializing the encoder, make sure max-message-bytes in the opts, and set identical to producer's configuration.

What is changed and how it works?

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

Release note 

`None`.

3AceShowHand and others added 30 commits November 22, 2021 05:33
@3AceShowHand @ti-chi-bot
This is an automated cherry-pick of pingcap#3192
0187fdd 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@3AceShowHand
fix conflicts.
e1a50d6
@3AceShowHand @ti-chi-bot
This is an automated cherry-pick of pingcap#4036
806ceae 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@3AceShowHand
remove craft.
f51e1a9
@3AceShowHand
remove useless files.
1996bd6
@3AceShowHand
fix conflicts in json.
9dd01b8
@ti-chi-bot @3AceShowHand
test(ticdc): separate statistics for unit tests and integration tests (…
2913f7b 
…pingcap#3570) (pingcap#3579)
@ti-chi-bot @3AceShowHand
ticdc/metrics: fix processor checkpoint-ts lag expr (pingcap#3543) (p…
9f3f8cf 
…ingcap#3559)
@ti-chi-bot @3AceShowHand
test: add new ci collation without old value test (pingcap#3114) (pin…
6fad381 
…gcap#3368)
@ti-chi-bot @3AceShowHand
processor,sink(cdc): let sink report resolved ts and do not skip buff…
3136a20 
…er sink flush (pingcap#3540) (pingcap#3563)
@ti-chi-bot @3AceShowHand
integration_tests(ticdc): move integration tests into integration_tes…
d928a2f 
…ts dir (pingcap#3646) (pingcap#3690)
@ti-chi-bot @3AceShowHand
sink(ticdc): Set config.Metadata.Timeout correctly (pingcap#3665) (pi…
50545de 
…ngcap#3671)
@ti-chi-bot @3AceShowHand
*: fix changefeed checkpoint lag negative value error (pingcap#3013) (p…
47b2b2d 
…ingcap#3534)
@ti-chi-bot @3AceShowHand
ticdc.rules: fix alertmanager rules (pingcap#3422) (pingcap#3595)
666ac9e
@ti-chi-bot @3AceShowHand
Fix handling of JSON columns (pingcap#3643) (pingcap#3652)
643fb1b
@ti-chi-bot @3AceShowHand
workerpool: limit the rate to output deadlock warning (pingcap#3775) (p…
974aaa1 
…ingcap#3798)
@ti-chi-bot @3AceShowHand
ticdc/alert: add no owner alert rule (pingcap#3809) (pingcap#3835)
e5155ed
@ti-chi-bot @3AceShowHand
pkg,cdc: do not use log package (pingcap#3902) (pingcap#3943)
daf1adb
@amyangfei @3AceShowHand
*: rename repo from pingcap/ticdc to pingcap/tiflow (pingcap#3956)
ab1661e
@ti-chi-bot @3AceShowHand
tz (ticdc): fix timezone error (pingcap#3887) (pingcap#3909)
f7a8919
@ti-chi-bot @3AceShowHand
kvclient(ticdc): fix kvclient takes too long time to recover (pingcap…
92b5021 
…#3612) (pingcap#3661)
@ti-chi-bot @3AceShowHand
etcd_worker: batch etcd patch (pingcap#3277) (pingcap#3392)
fea07be
@ti-chi-bot @3AceShowHand
Clean old owner and old processor in release 5.2 branch (pingcap#4019) (
b5178e6 
pingcap#4023)
@ti-chi-bot @3AceShowHand
tests(ticdc): set up the sync diff output directory correctly (pingca…
4ef3deb 
…p#3725) (pingcap#3744)
@ti-chi-bot @3AceShowHand
owner,scheduler(cdc): fix nil pointer panic in owner scheduler (pingc…
af3c44f 
…ap#2980) (pingcap#4007) (pingcap#4014)
@ti-chi-bot @3AceShowHand
config(ticdc): Fix old value configuration check for maxwell protocol (…
f51db06 
…pingcap#3747) (pingcap#3781)
@3AceShowHand
This is an automated cherry-pick of pingcap#4036
8b5360c 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@3AceShowHand
remove craft.
f35ae50
@3AceShowHand
remove useless files.
9da3fb7
@3AceShowHand
fix conflicts in json.
380feac
amyangfei
amyangfei reviewed Dec 27, 2021
View reviewed changes
Copy link
Contributor

@amyangfei amyangfei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

@3AceShowHand @amyangfei
Update cdc/sink/codec/json.go
4bf83dc 
Co-authored-by: amyangfei <amyangfei@gmail.com>
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed status/LGT1 Indicates that a PR has LGTM 1. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 27, 2021
@ti-chi-bot ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 27, 2021
@codecov-commenter
Copy link

codecov-commenter commented Dec 27, 2021 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (release-5.1@1ac656b). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release-5.1      #4077   +/-   ##
================================================
  Coverage               ?   64.1878%           
================================================
  Files                  ?        160           
  Lines                  ?      16782           
  Branches               ?          0           
================================================
  Hits                   ?      10772           
  Misses                 ?       5071           
  Partials               ?        939

@3AceShowHand
Copy link
Contributor

/run-integration-test
/run-kafka-integration-test

@Rustin170506
Copy link
Member

/merge

@ti-chi-bot
Copy link
Member Author

This pull request has been accepted and is ready to merge.

Commit hash: bd8a6e1

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 27, 2021
@zhouqiang-cl zhouqiang-cl added the cherry-pick-approved Cherry pick PR approved by release team. label Dec 27, 2021
@Rustin170506
Copy link
Member

Rustin170506 commented Dec 27, 2021 •
edited
Loading

/hold

Waiting for manual testing

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 27, 2021
@3AceShowHand
Copy link
Contributor

/run-all-tests

@3AceShowHand
Copy link
Contributor

/run-kafka-integration-test

@overvenus
Copy link
Member

/unhold

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 27, 2021
@overvenus
Copy link
Member

/merge

@ti-chi-bot ti-chi-bot merged commit d94cf7c into pingcap:release-5.1 Dec 27, 2021
@overvenus overvenus added this to the v5.1.4 milestone Jan 6, 2022
@3AceShowHand 3AceShowHand mentioned this pull request Jan 20, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rustin170506 Rustin170506 Rustin170506 approved these changes

@overvenus overvenus overvenus approved these changes

@amyangfei amyangfei amyangfei approved these changes

Assignees

@3AceShowHand 3AceShowHand

Labels
cherry-pick-approved Cherry pick PR approved by release team. release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-5.1 This PR is cherry-picked to release-5.1 from a source PR.
Projects
None yet
Milestone
v5.1.4
Development

Successfully merging this pull request may close these issues.
7 participants
@ti-chi-bot @3AceShowHand @amyangfei @codecov-commenter @Rustin170506 @overvenus @zhouqiang-cl