Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv/client: fix token counter in region scan limiter (#1979) #2037

Merged
merged 13 commits into from
Jun 22, 2021
Merged

kv/client: fix token counter in region scan limiter (#1979) #2037

merged 13 commits into from
Jun 22, 2021

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #1979

What problem does this PR solve?

  • Fix token is not correctly updated in some corner case
  • Add more metrics

What is changed and how it works?

  • Fix token is not correctly updated in some corner case
  • Add more metrics
  • Fix a bug that context is not used correctly when calling onRegionFail, which may lead to region loss, the error region should be re-dispatched, but ignore in error channel. (This Fix has been verified by manual test in the next part)

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    • Setup a cdc replication cluster with 3 TiKVs, in the upstream prepare more than 10k regions.
    • Reload one of the TiKV
    • The resolved ts of TiCDC should recovery in 2 minutes.

Release note

  • No release note

@ti-chi-bot ti-chi-bot added do-not-merge/cherry-pick-not-approved status/ptal Could you please take a look? release-blocker This issue blocks a release. Please solve it ASAP. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. type/cherry-pick-for-release-5.0 This PR is cherry-picked to release-5.0 from a source PR. labels Jun 11, 2021
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 11, 2021
Merged
@ti-chi-bot ti-chi-bot requested review from zier-one and overvenus June 11, 2021 05:06
@amyangfei amyangfei added this to the v5.0.3 milestone Jun 11, 2021
@amyangfei amyangfei removed the release-blocker This issue blocks a release. Please solve it ASAP. label Jun 11, 2021
overvenus
overvenus approved these changes Jun 17, 2021
View reviewed changes
Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve in advance, please merge this PR once it's ready.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 17, 2021
@amyangfei amyangfei force-pushed the cherry-pick-1979-to-release-5.0 branch from 160ddb2 to e95954d Compare June 21, 2021 07:04
amyangfei
amyangfei reviewed Jun 21, 2021
View reviewed changes
pkg/config/kvclient.go
// background workerpool size, the workrpool is shared by all goroutines in cdc server
WorkerPoolSize int `toml:"worker-pool-size" json:"worker-pool-size"`
// region incremental scan limit for one table in a single store
RegionScanLimit int `toml:"region-scan-limit" json:"region-scan-limit"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is part of cherry-pick from #1899

@amyangfei
Copy link
Contributor

/run-all-tests

@amyangfei
Copy link
Contributor

/run-all-tests

@lonng lonng added cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Jun 22, 2021
@ti-chi-bot
Copy link
Member Author

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • amyangfei
  • overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 22, 2021
@amyangfei
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member Author

This pull request has been accepted and is ready to merge.

Commit hash: 3e09b02

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 22, 2021
@codecov-commenter
Copy link

codecov-commenter commented Jun 22, 2021 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (release-5.0@49c684b). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release-5.0      #2037   +/-   ##
================================================
  Coverage               ?   53.9680%           
================================================
  Files                  ?        164           
  Lines                  ?      17540           
  Branches               ?          0           
================================================
  Hits                   ?       9466           
  Misses                 ?       7077           
  Partials               ?        997

@amyangfei amyangfei force-pushed the cherry-pick-1979-to-release-5.0 branch from 2b07a0a to 3255626 Compare June 22, 2021 07:28
@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jun 22, 2021
@amyangfei
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member Author

This pull request has been accepted and is ready to merge.

Commit hash: 3255626

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 22, 2021
@amyangfei
Copy link
Contributor

/run-unit-tests

@amyangfei
Copy link
Contributor

/merge

@amyangfei
Copy link
Contributor

/run-all-tests

@amyangfei
Copy link
Contributor

/run-unit-tests

@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jun 22, 2021
@amyangfei
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member Author

This pull request has been accepted and is ready to merge.

Commit hash: fdf1b3e

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 22, 2021
@ti-chi-bot ti-chi-bot merged commit 3a6b37b into pingcap:release-5.0 Jun 22, 2021
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 22, 2021
@ti-chi-bot
Copy link
Member Author

@ti-chi-bot: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus approved these changes

@amyangfei amyangfei amyangfei approved these changes

@zier-one zier-one Awaiting requested review from zier-one

Assignees

@amyangfei amyangfei

Labels
cherry-pick-approved Cherry pick PR approved by release team. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look? type/cherry-pick-for-release-5.0 This PR is cherry-picked to release-5.0 from a source PR.
Projects
None yet
Milestone
v5.0.3
Development

Successfully merging this pull request may close these issues.
5 participants
@ti-chi-bot @amyangfei @codecov-commenter @overvenus @lonng