Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv/client: add incremental scan region count limit #1926

Merged
merged 8 commits into from
Jun 9, 2021
Merged

kv/client: add incremental scan region count limit #1926

merged 8 commits into from
Jun 9, 2021

Conversation

amyangfei
Copy link
Contributor

What problem does this PR solve?

cherry-pick #1899 into release-4.0, for test only

What is changed and how it works?

cherry-pick #1899 into release-4.0, for test only

Check List

Tests

  • Unit test
  • Integration test

Release note

  • Add concurrency limit to the region incremental scan in kv client.

@amyangfei amyangfei added component/kv-client TiKV kv log client component. type/cherry-pick-for-release-4.0 This PR is cherry-picked to release-4.0 from a source PR. labels Jun 3, 2021
@amyangfei amyangfei added this to the v4.0.14 milestone Jun 3, 2021
@ti-chi-bot ti-chi-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jun 3, 2021
@amyangfei
Copy link
Contributor Author

/run-all-tests

overvenus
overvenus approved these changes Jun 6, 2021
View reviewed changes
Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 6, 2021
@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-unit-tests

1 similar comment
@amyangfei
Copy link
Contributor Author

/run-unit-tests

@amyangfei amyangfei added the release-blocker This issue blocks a release. Please solve it ASAP. label Jun 9, 2021
@lonng lonng added cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Jun 9, 2021
@lonng
Copy link
Contributor

lonng commented Jun 9, 2021

/merge

@ti-chi-bot
Copy link
Member

@lonng: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@lonng
Copy link
Contributor

lonng commented Jun 9, 2021

/lgtm

@ti-chi-bot
Copy link
Member

@lonng: Please use GitHub review feature instead of /lgtm [cancel] when you want to submit review to the pull request.
For how to use GitHub review feature, see also this document provided by GitHub.

For the reason we drop support to the commands, see also this page.
This reply is being used as a temporary reply during the migration of review process and will be removed on July 1st.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@amyangfei
Copy link
Contributor Author

/run-unit-tests

@overvenus
Copy link
Member

/run-kafka-integration-test

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • liuzix
  • overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot removed the status/LGT1 Indicates that a PR has LGTM 1. label Jun 9, 2021
@ti-chi-bot ti-chi-bot added the status/LGT2 Indicates that a PR has LGTM 2. label Jun 9, 2021
@overvenus
Copy link
Member

/run-kafka-integration-tests

@overvenus
Copy link
Member

/run-kafka-tests

overvenus
overvenus reviewed Jun 9, 2021
View reviewed changes
cdc/kv/client.go Outdated Show resolved Hide resolved
@overvenus
Copy link
Member

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 5c2d80b

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 9, 2021
@overvenus
Copy link
Member

/run-kafka-tests
/run-unit-tests

@overvenus
Copy link
Member

/run-integration-test
/run-leak-test

@overvenus
Copy link
Member

/run-all-tests

1 similar comment
@amyangfei
Copy link
Contributor Author

/run-all-tests

@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jun 9, 2021
@amyangfei
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: f881572

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 9, 2021
amyangfei
amyangfei commented Jun 9, 2021
View reviewed changes
cdc/kv/client_test.go
eventCh := make(chan *model.RegionFeedEvent, 10)
// The buffer size of event channel must be large enough because in the test
// case we send events first, and then retrive all events from this channel.
eventCh := make(chan *model.RegionFeedEvent, 100)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the fix for this bug should be picked to master @amyangfei

@amyangfei
Copy link
Contributor Author

/run-leak-tests

@ti-chi-bot
Copy link
Member

@amyangfei: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit ffbdae3 into pingcap:release-4.0 Jun 9, 2021
@amyangfei amyangfei deleted the chery-pick-4.0-region-scan-limit branch June 10, 2021 03:21
amyangfei added a commit to amyangfei/tiflow that referenced this pull request Jun 24, 2021
@amyangfei
kv/client: add incremental scan region count limit (pingcap#1899)
b712e95 
Note pingcap#1926 has picked part of pingcap#1899, this PR picks the remaining change
amyangfei added a commit to amyangfei/tiflow that referenced this pull request Jun 24, 2021
@amyangfei
kv/client: add incremental scan region count limit (pingcap#1899)
13a817d 
Note pingcap#1926 has picked part of pingcap#1899, this PR picks the remaining change
amyangfei added a commit to amyangfei/tiflow that referenced this pull request Jul 7, 2021
@amyangfei
kv/client: add incremental scan region count limit (pingcap#1899)
66921e8 
Note pingcap#1926 has picked part of pingcap#1899, this PR picks the remaining change
amyangfei added a commit to amyangfei/tiflow that referenced this pull request Jul 7, 2021
@amyangfei
kv/client: add incremental scan region count limit (pingcap#1899)
8a05bfe 
Note pingcap#1926 has picked part of pingcap#1899, this PR picks the remaining change
ti-chi-bot pushed a commit that referenced this pull request Jul 8, 2021
@amyangfei @ti-chi-bot
kv/client: add incremental scan region count limit (#1899)
c37854d 
Note #1926 has picked part of #1899, this PR picks the remaining change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus approved these changes

@liuzix liuzix liuzix approved these changes

@zier-one zier-one Awaiting requested review from zier-one

Assignees
No one assigned
Labels
cherry-pick-approved Cherry pick PR approved by release team. component/kv-client TiKV kv log client component. release-blocker This issue blocks a release. Please solve it ASAP. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/cherry-pick-for-release-4.0 This PR is cherry-picked to release-4.0 from a source PR.
Projects
None yet
Milestone
v4.0.14
Development

Successfully merging this pull request may close these issues.
5 participants
@amyangfei @lonng @ti-chi-bot @overvenus @liuzix