Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scheduler(ticdc): split more evenly #8070

Merged
merged 7 commits into from
Jan 17, 2023
Merged

scheduler(ticdc): split more evenly #8070

merged 7 commits into from
Jan 17, 2023

Conversation

overvenus
Copy link
Member

@overvenus overvenus commented Jan 12, 2023 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #7720

What is changed and how it works?

  • Enable split span by default.
  • Split span evenly.

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?

No.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note 

None

@overvenus
config,scheduler(ticdc): enable split span by default and split evenly
22ff733 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus overvenus added the area/ticdc Issues or PRs related to TiCDC. label Jan 12, 2023
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jan 12, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • 3AceShowHand
  • asddongmen

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 12, 2023
@overvenus overvenus mentioned this pull request Jan 12, 2023
Closed
22 tasks
@overvenus
split more evely
ede527d 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus
Copy link
Member Author

/run-all-tests

@overvenus overvenus marked this pull request as ready for review January 13, 2023 07:23
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 13, 2023
@overvenus
Copy link
Member Author

/run-all-tests

@overvenus
Copy link
Member Author

/run-integration-test
/run-kafka-integration-test

1 similar comment
@overvenus
Copy link
Member Author

/run-integration-test
/run-kafka-integration-test

nongfushanquan
nongfushanquan reviewed Jan 17, 2023
View reviewed changes
Copy link
Contributor

@nongfushanquan nongfushanquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend not enabling split span by default .the user can enable this feature in the changefeed configure if necessary

@3AceShowHand
Copy link
Contributor

I recommend not enabling split span by default .the user can enable this feature in the changefeed configure if necessary

I agreed. In the most use case, there are multiple tables in the changefeed so that split span is not necessary.

We can update the documents to tell users this feature, and let them to decide whether to enable it or not.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 17, 2023
@overvenus
Copy link
Member Author

I recommend not enabling split span by default .the user can enable this feature in the changefeed configure if necessary

How about setting a larger default value? E.g., 100K regions. It's will disable the feature for most cases (smaller than 100k regions).

@overvenus
revert enable split by default
11adcdd 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus
Merge remote-tracking branch 'github/span-replication/enable' into sp…
3c5dd02 
…an-replication/enable
@overvenus overvenus changed the title config,scheduler(ticdc): enable split span by default and split evenly scheduler(ticdc): split more evenly Jan 17, 2023
@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jan 17, 2023
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 17, 2023
@overvenus
Copy link
Member Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 3c5dd02

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 17, 2023
@ti-chi-bot
Copy link
Member

@overvenus: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

trigger some heavy tests which will not run always when PR updated.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@overvenus
fix processor scheduler config issue
b1a0545 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jan 17, 2023
@overvenus
Copy link
Member Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: b1a0545

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 17, 2023
@ti-chi-bot ti-chi-bot merged commit 0dc39e0 into pingcap:master Jan 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nongfushanquan nongfushanquan nongfushanquan left review comments

@asddongmen asddongmen asddongmen approved these changes

@3AceShowHand 3AceShowHand 3AceShowHand approved these changes

@sdojjy sdojjy Awaiting requested review from sdojjy

Assignees
No one assigned
Labels
area/ticdc Issues or PRs related to TiCDC. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@overvenus @ti-chi-bot @3AceShowHand @asddongmen @nongfushanquan