Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ticdc: support graceful upgrade TiCDC pods #4647

Merged
merged 10 commits into from
Aug 3, 2022
Merged

ticdc: support graceful upgrade TiCDC pods #4647

merged 10 commits into from
Aug 3, 2022

Conversation

overvenus
Copy link
Member

What problem does this PR solve?

This PR supports upgrade TiCDC pods without causing latency spike.

Close #4646

What is changed and how does it work?

Before update TiCDC statefuleset, we first drain the TiCDC in that pod.

Code changes

  • Has Go code change
  • Has CI related scripts change

Tests

  • Unit test
  • E2E test
  • Manual test
  • No code
Upgrade TiCDC via editting config
image

Side effects

  • Breaking backward compatibility
  • Other side effects:

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

Support graceful upgrade TiCDC

@overvenus
pkg: add ticdc graceful shutdown timeout option
88eec01 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus
update generated files
47e9500 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus
ticdc: support graceful upgrade TiCDC pods
a480752 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 27, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • DanielZhangQD
  • KanShiori

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@overvenus overvenus mentioned this pull request Jul 27, 2022
Closed
37 tasks
@codecov-commenter
Copy link

codecov-commenter commented Jul 27, 2022 •
edited
Loading

Codecov Report

Merging #4647 (bdcfd11) into master (c59fdf4) will increase coverage by 6.38%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4647      +/-   ##
==========================================
+ Coverage   62.65%   69.03%   +6.38%     
==========================================
  Files         186      190       +4     
  Lines       20849    23346    +2497     
==========================================
+ Hits        13062    16116    +3054     
+ Misses       6591     5986     -605     
- Partials     1196     1244      +48
Flag Coverage Δ
e2e 48.22% <66.66%> (?)
unittest 62.64% <100.00%> (-0.01%) ⬇️

@overvenus
Merge branch 'master' into cdc-graceful-upgrade
58496c2
@overvenus
fix misspell
194e28f 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus
Copy link
Member Author

/test pull-e2e-kind pull-e2e-kind-across-kubernetes pull-e2e-kind-pull-e2e-kind-serial

@overvenus
Copy link
Member Author

/test pull-e2e-kind-br

@overvenus
use metav1.Duration
c2316e8 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@ti-chi-bot
Copy link
Member

@liubog2008: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

overvenus and others added 2 commits August 2, 2022 13:20
@overvenus
address lints
2a063f1 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@DanielZhangQD
Merge branch 'master' into cdc-graceful-upgrade
7a90ab2
@DanielZhangQD
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@DanielZhangQD: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

DanielZhangQD
DanielZhangQD approved these changes Aug 2, 2022
View reviewed changes
Copy link
Contributor

@DanielZhangQD DanielZhangQD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DanielZhangQD
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 7a90ab2

@DanielZhangQD
Copy link
Contributor

/test pull-e2e-kind-serial pull-e2e-kind-br

@DanielZhangQD
Copy link
Contributor

/test pull-e2e-kind-serial

@ti-chi-bot
Copy link
Member

@overvenus: Your PR was out of date, I have automatically updated it for you.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@overvenus
Copy link
Member Author

/test pull-e2e-kind

1 similar comment
@DanielZhangQD
Copy link
Contributor

/test pull-e2e-kind

@ti-chi-bot ti-chi-bot merged commit 5c7e1a4 into pingcap:master Aug 3, 2022
overvenus added a commit to overvenus/tidb-operator that referenced this pull request Sep 8, 2022
@overvenus
ticdc: support graceful upgrade TiCDC pods (pingcap#4647)
2c86337
csuzhangxc pushed a commit that referenced this pull request Sep 13, 2022
@overvenus
Support graceful upgrade TiCDC (>=6.3.0) pods (#4697)
a735acc 
* Support graceful shutdown TiCDC node (#4624)

* ticdc: support graceful upgrade TiCDC pods (#4647)

* ticdc: wait TiCDC cluster becomes healthy after resigning owner (#4665)

Signed-off-by: Neil Shen <overvenus@gmail.com>
xhebox pushed a commit to KanShiori/tidb-operator that referenced this pull request Sep 16, 2022
@overvenus @xhebox
ticdc: support graceful upgrade TiCDC pods (pingcap#4647)
2df0ecd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DanielZhangQD DanielZhangQD DanielZhangQD approved these changes

@KanShiori KanShiori KanShiori approved these changes

@liubog2008 liubog2008 liubog2008 approved these changes

Assignees
No one assigned
Labels
status/can-merge status/LGT2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support graceful upgrade TiCDC pods
6 participants
@overvenus @ti-chi-bot @codecov-commenter @DanielZhangQD @liubog2008 @KanShiori