Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedulers: add role config to shuffleRegionScheduler #2219

Merged
merged 7 commits into from
Mar 12, 2020

Conversation

disksing
Copy link
Contributor

What problem does this PR solve?

Fix #2177

What is changed and how it works?

Add role configuration to shuffle-region-scheduler.

Use it to balance learners:

1. add shuffle-region-scheduler

pd-ctl scheduler add shuffle-region-scheduler

or

curl -XPOST -d '{"name": "shuffle-region-scheduler"}' http://pd:2379/pd/api/v1/schedulers

2. config roles

pd-ctl scheduler config shuffle-region-scheduler set-roles learner

or

curl -XPOST -d '["learner"]'  http://pd:2379/pd/api/v1/scheduler-config/shuffle-region-scheduler/roles

Check List

Tests

  • Unit test
  • Manual test

Code changes

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
@disksing disksing added the component/schedule Scheduling logic. label Mar 10, 2020
@disksing disksing added this to the v4.0.0-rc milestone Mar 10, 2020
@disksing disksing changed the title Shuffle region schedulers: add role config to shuffleRegionScheduler Mar 10, 2020
@disksing
Copy link
Contributor Author

/run-test

@disksing disksing added the needs-cherry-pick-release-3.1 The PR needs to cherry pick to release-3.1 branch. label Mar 12, 2020

func (conf *shuffleRegionSchedulerConfig) ServeHTTP(w http.ResponseWriter, r *http.Request) {
router := mux.NewRouter()
router.HandleFunc("/roles", conf.handleGetRoles).Methods("GET")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the root path of PD's API? Maybe add the PD API prefix is better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it is a sub router to be registered under scheduler_config/shuffle-region-scheduler/.

if cluster.IsPlacementRulesEnabled() {
scoreGuard = filter.NewRuleFitFilter(s.GetName(), cluster, region, oldPeer.GetStoreId())
} else {
scoreGuard = filter.NewDistinctScoreFilter(s.GetName(), cluster.GetLocationLabels(), cluster.GetRegionStores(region), cluster.GetStore(oldPeer.GetStoreId()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it different from the original behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, different. Priviously it does not consider any constraints.

roles = append(roles, f)
}
}
b, _ := json.Marshal(roles)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to handle this error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should not return any error as it only contains basic types.

Copy link
Member

@rleungx rleungx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rleungx
Copy link
Member

rleungx commented Mar 12, 2020

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 12, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

@disksing merge failed.

@disksing
Copy link
Contributor Author

/run-test

@disksing
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

@disksing merge failed.

@disksing
Copy link
Contributor Author

CI failure cased by #2231

@disksing
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

@disksing merge failed.

@disksing
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

/run-all-tests

@sre-bot sre-bot merged commit addf27a into tikv:master Mar 12, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Mar 12, 2020

cherry pick to release-3.1 failed

@disksing disksing deleted the shuffle-region branch March 12, 2020 07:41
disksing added a commit to oh-my-tidb/pd that referenced this pull request Mar 12, 2020
disksing added a commit to oh-my-tidb/pd that referenced this pull request Mar 12, 2020
disksing added a commit that referenced this pull request Mar 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/schedule Scheduling logic. needs-cherry-pick-release-3.1 The PR needs to cherry pick to release-3.1 branch. status/can-merge Indicates a PR has been approved by a committer.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support balance learners
5 participants