Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: add table option pre_split_regions for pre-split region when create table with shard_row_id_bits. #10138

Merged
merged 22 commits into from
Apr 24, 2019

Conversation

crazycs520
Copy link
Contributor

@crazycs520 crazycs520 commented Apr 12, 2019

What problem does this PR solve?

Pre-split table region when creat table with shard_row_id_bits.

If we do not pre-split region, there is only 1 region of the table at first, then maybe still have write hot spots at the beginning.

What is changed and how it works?

  1. Do pre-split region when create table with shard_row_id_bits and pre_split_regions.

  2. scatter split region. Wait PD PR merge: client: supports scatter region and get operator status tikv/pd#1501

Pre-split region will split 2^(pre_split_regions-1) for table row data, split 1 region for every index separately.

The pre-split process was executed asynchronous.

Check List

Tests

  • Adding test...

Code changes

  • Has exported function/method change

Side effects

@codecov
Copy link

codecov bot commented Apr 12, 2019

Codecov Report

Merging #10138 into master will increase coverage by 0.0178%.
The diff coverage is 4.8543%.

@@               Coverage Diff                @@
##             master     #10138        +/-   ##
================================================
+ Coverage   77.8697%   77.8876%   +0.0178%     
================================================
  Files           409        407         -2     
  Lines         84188      82732      -1456     
================================================
- Hits          65557      64438      -1119     
+ Misses        13742      13511       -231     
+ Partials       4889       4783       -106

@winkyao
Copy link
Contributor

winkyao commented Apr 15, 2019

Please update the desc Adding test, do you finish adding?

ddl/table.go Show resolved Hide resolved
ddl/table.go Outdated Show resolved Hide resolved
ddl/table.go Show resolved Hide resolved
@nolouch
Copy link
Member

nolouch commented Apr 16, 2019

Can we try to detect the scattering operator is finished then return the DDL job is finished, like https://github.com/tikv/tikv/pull/4423/files#diff-14a21dd3ee41f0c7bb3547557e5c815cR223?

@bb7133
Copy link
Member

bb7133 commented Apr 17, 2019

Please update go.sum / go.mod for the parser

ddl/table.go Outdated
@@ -74,6 +74,7 @@ func onCreateTable(d *ddlCtx, t *meta.Meta, job *model.Job) (ver int64, _ error)
// TODO: Add restrictions to this operation.
go splitTableRegion(d.store, tbInfo.ID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we modify splitTableRegion strategy here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is not to do this.

ddl/table.go Show resolved Hide resolved
store/tikv/backoff.go Outdated Show resolved Hide resolved
@nolouch
Copy link
Member

nolouch commented Apr 23, 2019

Can we show the pre-split-regions in show create table xxx?

@crazycs520
Copy link
Contributor Author

@nolouch Done.

Copy link
Member

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

}

func (s *tikvStore) SplitRegionAndScatter(splitKey kv.Key) (uint64, error) {
left, err := s.splitRegion(splitKey)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If splitRegion failed will retry to split? such as epoch is not matched.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a Backoff in splitRegion, so it will retry.

Copy link
Contributor

@winkyao winkyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520
Copy link
Contributor Author

/run-all-tests

Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bb7133 bb7133 merged commit 732f430 into pingcap:master Apr 24, 2019
crazycs520 added a commit to crazycs520/tidb that referenced this pull request Jun 19, 2019
winkyao pushed a commit that referenced this pull request Jun 19, 2019
@you06 you06 added the sig/sql-infra SIG: SQL Infra label Mar 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/sql-infra SIG: SQL Infra
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants