Drainer should not retry on long time ddls directly

[lichunzhu]

What did you do?

Use Drainer to replicate some long time costing ddls to downstream TiDB. For example, adding index ddls.

What did you expect to see?

Drainer can replicate ddls successfully.

What did you see instead?

Drainer fails because of i/o timeout and keeps retrying to replicate these ddls again.

Versions of the cluster

master(b0214a2)

[lichunzhu]

Root Cause

When Drainer executes some time-costing ddls, especially for adding index, Drainer may fail to get result because it won't return any result until it succeeds. If this time cost is larger than syncer's read-timeout, Drainer will fail and try to execute ddl again. This will make the situation even worse.

Workaround

For these long time costing ddls, we can use a special connection to execute them and keep watching the result asynchronously through admin show ddl jobs.