mcs: support forwarding split and scatter request #7190

rleungx · 2023-10-11T06:50:24Z

What problem does this PR solve?

Issue Number: Ref #5839.

What is changed and how does it work?

Before this PR, the API service was responsible for scatter and split. After this PR, the request will be forwarded to the scheduling service. Once we implement the scheduling service discovery, we can directly send requests to the scheduling service.

Check List

Tests

Manual test

Release note

None.

ti-chi-bot · 2023-10-11T06:50:27Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JmPotato
lhy1024

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

codecov · 2023-10-11T07:27:18Z

Codecov Report

Merging #7190 (49d864c) into master (8b2896b) will decrease coverage by 0.27%.
The diff coverage is 1.45%.

@@            Coverage Diff             @@
##           master    #7190      +/-   ##
==========================================
- Coverage   74.65%   74.39%   -0.27%     
==========================================
  Files         442      442              
  Lines       47744    47870     +126     
==========================================
- Hits        35642    35611      -31     
- Misses       8957     9103     +146     
- Partials     3145     3156      +11

Flag	Coverage Δ
unittests	`74.39% <1.45%> (-0.27%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

lhy1024 · 2023-10-11T07:31:49Z

pkg/mcs/scheduling/server/grpc_service.go

@@ -191,6 +185,48 @@ func (s *Service) StoreHeartbeat(ctx context.Context, request *schedulingpb.Stor
 	return &schedulingpb.StoreHeartbeatResponse{Header: &schedulingpb.ResponseHeader{ClusterId: s.clusterID}}, nil
 }

+// SplitRegions split regions by the given split keys
+func (s *Service) SplitRegions(ctx context.Context, request *schedulingpb.SplitRegionsRequest) (*schedulingpb.SplitRegionsResponse, error) {


Do we need to add some tests about forward requests?

Yes, I have added a TODO in the PR description. Will do it later.

lhy1024 · 2023-10-12T03:05:02Z

pkg/mcs/scheduling/server/grpc_service.go

@@ -191,6 +185,48 @@ func (s *Service) StoreHeartbeat(ctx context.Context, request *schedulingpb.Stor
 	return &schedulingpb.StoreHeartbeatResponse{Header: &schedulingpb.ResponseHeader{ClusterId: s.clusterID}}, nil
 }

+// SplitRegions split regions by the given split keys
+func (s *Service) SplitRegions(ctx context.Context, request *schedulingpb.SplitRegionsRequest) (*schedulingpb.SplitRegionsResponse, error) {


BTW, we seem to need to support AskSplit, AskBatchSplit, and GetOperator with grpc method too.

We don't need AskSplit/AskBatchSplit. But GetOperator is necessary.

HandleAskBatchSplit will call GetMergeChecker()

Oh, you are right.

Will do it in the next PR

JmPotato

LGTM. Please update the kvproto first.

lhy1024

the rest LGTM

lhy1024 · 2023-10-17T01:43:11Z

server/grpc_service.go

@@ -2175,7 +2283,7 @@ func (s *GrpcServer) SplitRegions(ctx context.Context, request *pdpb.SplitRegion
 }

 // SplitAndScatterRegions split regions by the given split keys, and scatter regions.
-// Only regions which splited successfully will be scattered.
+// Only regions which split successfully will be scattered.
 // scatterFinishedPercentage indicates the percentage of successfully splited regions that are scattered.
 func (s *GrpcServer) SplitAndScatterRegions(ctx context.Context, request *pdpb.SplitAndScatterRegionsRequest) (*pdpb.SplitAndScatterRegionsResponse, error) {


Do we need to forward this functions?

AFAIK, this API is not used for now.

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx · 2023-10-17T06:32:32Z

/merge

ti-chi-bot · 2023-10-17T06:32:35Z

@rleungx: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-10-17T06:32:36Z

This pull request has been accepted and is ready to merge.

Commit hash: 49d864c

ref tikv#5839 Signed-off-by: Ryan Leung <rleungx@gmail.com>

ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Oct 11, 2023

ti-chi-bot bot requested review from HunDunDM and nolouch October 11, 2023 06:50

ti-chi-bot bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Oct 11, 2023

rleungx requested review from JmPotato and lhy1024 and removed request for nolouch and HunDunDM October 11, 2023 06:50

rleungx mentioned this pull request Oct 11, 2023

Support forwarding split and scatter request pingcap/kvproto#1198

Merged

lhy1024 reviewed Oct 11, 2023

View reviewed changes

lhy1024 reviewed Oct 12, 2023

View reviewed changes

JmPotato approved these changes Oct 16, 2023

View reviewed changes

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 16, 2023

lhy1024 reviewed Oct 17, 2023

View reviewed changes

lhy1024 approved these changes Oct 17, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 17, 2023

rleungx added 3 commits October 17, 2023 14:00

support forwarding split and scatter request

a50e66f

Signed-off-by: Ryan Leung <rleungx@gmail.com>

add get operator rpc

0cc6054

Signed-off-by: Ryan Leung <rleungx@gmail.com>

update kvproto

49d864c

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx force-pushed the forward-split-scatter branch from 58fa4dd to 49d864c Compare October 17, 2023 06:01

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 17, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 17, 2023

ti-chi-bot bot merged commit c9a97a8 into tikv:master Oct 17, 2023
24 of 26 checks passed

rleungx deleted the forward-split-scatter branch October 17, 2023 06:38

rleungx mentioned this pull request Oct 23, 2023

Separate scheduling into an independent service #5839

Closed

rleungx added a commit to rleungx/pd that referenced this pull request Dec 1, 2023

mcs: support forwarding split and scatter request (tikv#7190)

0117dee

ref tikv#5839 Signed-off-by: Ryan Leung <rleungx@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mcs: support forwarding split and scatter request #7190

mcs: support forwarding split and scatter request #7190

rleungx commented Oct 11, 2023 •

edited

Loading

ti-chi-bot bot commented Oct 11, 2023 •

edited

Loading

codecov bot commented Oct 11, 2023 •

edited

Loading

lhy1024 Oct 11, 2023

rleungx Oct 11, 2023

lhy1024 Oct 12, 2023

rleungx Oct 12, 2023

lhy1024 Oct 12, 2023

rleungx Oct 12, 2023

rleungx Oct 13, 2023

JmPotato left a comment

lhy1024 left a comment

lhy1024 Oct 17, 2023

rleungx Oct 17, 2023

rleungx commented Oct 17, 2023

ti-chi-bot bot commented Oct 17, 2023

ti-chi-bot bot commented Oct 17, 2023

mcs: support forwarding split and scatter request #7190

mcs: support forwarding split and scatter request #7190

Conversation

rleungx commented Oct 11, 2023 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Oct 11, 2023 • edited Loading

codecov bot commented Oct 11, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JmPotato left a comment

Choose a reason for hiding this comment

lhy1024 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rleungx commented Oct 17, 2023

ti-chi-bot bot commented Oct 17, 2023

ti-chi-bot bot commented Oct 17, 2023

rleungx commented Oct 11, 2023 •

edited

Loading

ti-chi-bot bot commented Oct 11, 2023 •

edited

Loading

codecov bot commented Oct 11, 2023 •

edited

Loading