Create tables in parallel in load unit

[wjhuang2016]

Feature Request

Is your feature request related to a problem? Please describe:

Currently, DM Sends the DDL request serially. If the TiDB is not the owner, it will take about 1s to deal with this request. For example, the create table statement is about to take 1s. But if the request is sent to the owner, it will take only 100ms.

Describe the feature you'd like:

DM can automatically send the ddl request to the owner.

Describe alternatives you've considered:

Send the del request concurrently, it's better, but still slower than to the owner directly.

Teachability, Documentation, Adoption, Migration Strategy:

No

Score

• 600

Mentor

• @lance6716

[csuzhangxc]

In fact, I don't think DM should do this special thing. and it may not work in many cases, like a proxy between DM and TiDB.

[wjhuang2016]

Now I want the alternative: Send the DDL request concurrently. It would be useful when the number of the database's table is large but the data is not much. @lance6716 PTAL.

[lance6716]

(only for full data migration phase) I think we could implement it after 2.0 GA. And lightning may also use concurrent create table? @kennytm @glorv @YuJuncen

[glorv]

Yes, if there are a lot of table, the performance of sending ddl tasks in parallel will be much better.
There is a ticase of using lightning to restore 10k small tables, the current performance is quite slow.

[hidehalo]

/pick-up

[ti-challenge-bot]

It is not a pickable issue!

Details

Tip : If you want this issue to be picked, you need to add a challenge-program label to it.

Warning: None

[hidehalo]

Can I pick up this issue?

[ti-challenge-bot]

It has been added to the challenge program, but your issue description has some problems.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning: The description format for this issue is wrong.

[lance6716]

Can I pick up this issue?

added, please check again

[hidehalo]

Can I pick up this issue?

added, please check again

Thx!

/pick-up

[ti-challenge-bot]

Pick up success.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning:
Pick up success, but the issue miss mentor or score information.
cc: @wjhuang2016

[ti-challenge-bot]

The description of issue updated, but still has some problems.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning: The description format for this issue is wrong.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning:
Pick up success, but the issue miss mentor or score information.
cc: @wjhuang2016

[ti-challenge-bot]

The description of issue updated, but still has some problems.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning: The description format for this issue is wrong.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning:
Pick up success, but the issue miss mentor or score information.
cc: @wjhuang2016

[ti-challenge-bot]

The description of issue updated, but still has some problems.

Details

Tip :
You need to ensure that the description of the issue follows the following template:

```
## Score

- ${score}

## Mentor

- ${mentor}
```

Warning: The description format for this issue is wrong.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.

[ti-challenge-bot]

@hidehalo You did not submit PR within 7 days, so give up automatically.

[hidehalo]

/pick-up

[ti-challenge-bot]

Pick up success.