Skip to content
This repository has been archived by the owner on Jan 3, 2025. It is now read-only.

zh: add handle-error-sql-statements.md #227

Merged
merged 22 commits into from
Aug 18, 2020

Conversation

GMHDBJD
Copy link
Collaborator

@GMHDBJD GMHDBJD commented Aug 14, 2020

What is changed, added or deleted? (Required)

add handle-error-sql-statements.md

sql-skip/sql-replace 重构为 handle-error, 删除了原有的文档,新增的 handle-failed-sql-statements 借鉴了原来文档的结构和部分使用场景。其余地方几乎都不一样。

Which DM version(s) do your changes apply to? (Required)

  • master (the latest development version, including v2.0 changes for now)
  • v1.0 (TiDB DM 1.0 versions)

What is the related PR or file link(s)?

DM#850

  • This PR is translated from:
  • Other reference link(s):

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Have version specific changes
  • Might cause conflicts

@ireneontheway ireneontheway self-assigned this Aug 14, 2020
@ireneontheway ireneontheway added status/PTAL This PR is ready for reviewing. translation/doing This PR’s assignee is translating this PR. v2.0 This PR/issue applies to TiDB DM v2.0. labels Aug 14, 2020
@yikeke yikeke self-requested a review August 14, 2020 06:06
Copy link
Member

@csuzhangxc csuzhangxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

zh/handle-error-sql-statements.md Outdated Show resolved Hide resolved

### 命令介绍

使用 dmctl 手动处理出错的 SQL 语句时,主要使用的命令包括 `query-status`、`handle-error`。
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handle-error 淡化掉吧?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

啥意思?😂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我看错了,无视我,我以为是 query-error. 😢

| Table | Create Table |
+-------+-----------------------------------------------------------------------------------------------------------+
| tb | CREATE TABLE `shard_table` (
`id` int(11) DEFAULT NULL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

要不要加一下 PK。


- 比如:`DROP PRIMARY KEY`,这种情况下,只能在下游重建一个(DDL 执行完后的)新表结构对应的表,并将原表的全部数据重新导入该新表。

### 支持场景
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感觉这个文档的 三/四级标题,可以提升为 二/三级标题?

zh/handle-error-sql-statements.md Outdated Show resolved Hide resolved
"ErrClass": "schema-tracker",
"ErrScope": "internal",
"ErrLevel": "high",
"Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4]: cannot track DDL: ALTER TABLE `shard_db_1`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这文档里的 query-status 结果的 block,我们要不要省略掉一些信息(比如用 ...),只保留关键的信息,方便用户抓住重点。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

或者可以用 <details> <summary> HTML 标签进行折叠(不过尽量能省则省吧,这么长就算折叠起来也很难看):

执行结果
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-replica-01",
                "worker": "worker1",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 44006,
                                "ErrClass": "schema-tracker",
                                "ErrScope": "internal",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4]: cannot track DDL: ALTER TABLE `shard_db_1`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
                                "RawCause": "[ddl:8200]Unsupported modify charset from latin1 to utf8",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "2",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(DESKTOP-T561TSO-bin.000001, 1848)",
                        "masterBinlogGtid": "143bdef3-dd4a-11ea-8b00-00155de45f57:1-8",
                        "syncerBinlog": "(DESKTOP-T561TSO-bin.000001, 1420)",
                        "syncerBinlogGtid": "143bdef3-dd4a-11ea-8b00-00155de45f57:1-4",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                            {
                                "target": "`shard_db`.`shard_table`",
                                "DDLs": [
                                    "ALTER TABLE `shard_db`.`shard_table` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI"
                                ],
                                "firstLocation": "position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4",
                                "synced": [
                                    "`shard_db_1`.`shard_table_1`"
                                ],
                                "unsynced": [
                                    "`shard_db_1`.`shard_table_2`"
                                ]
                            }
                        ],
                        "synced": false,
                        "binlogType": "remote"
                    }
                }
            ]
        },
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-replica-02",
                "worker": "worker2",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 44006,
                                "ErrClass": "schema-tracker",
                                "ErrScope": "internal",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-7]: cannot track DDL: ALTER TABLE `shard_db_2`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
                                "RawCause": "[ddl:8200]Unsupported modify charset from latin1 to utf8",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "2",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(DESKTOP-T561TSO-bin.000001, 1848)",
                        "masterBinlogGtid": "1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-8",
                        "syncerBinlog": "(DESKTOP-T561TSO-bin.000001, 1420)",
                        "syncerBinlogGtid": "1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                            {
                                "target": "`shard_db`.`shard_table`",
                                "DDLs": [
                                    "ALTER TABLE `shard_db`.`shard_table` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI"
                                ],
                                "firstLocation": "position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6",
                                "synced": [
                                    "`shard_db_2`.`shard_table_1`"
                                ],
                                "unsynced": [
                                    "`shard_db_2`.`shard_table_2`"
                                ]
                            }
                        ],
                        "synced": false,
                        "binlogType": "remote"
                    }
                }
            ]
        }
    ]
}

| Table | Create Table |
+-------+-----------------------------------------------------------------------------------------------------------+
| tb | CREATE TABLE `shard_table` (
`id` int(11) DEFAULT NULL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加 PK?


##### 应用场景

假设现在存在如下四个上游表需要合并同步到下游的同一个表 ``` `shard_db`.`shard_table` ```,任务模式为悲观协调模式:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们要不要加一个简单的,没有合库合表的示例?那个可能反而是大多数场景

@yikeke
Copy link
Contributor

yikeke commented Aug 14, 2020

hi @GMHDBJD ,可以在 pr 描述里概括下你做的改动吗?两个修改的文件太大且没有 diff,不确定你修改的地方,不方便 review

zh/handle-error-sql-statements.md Outdated Show resolved Hide resolved
@@ -0,0 +1,1020 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议把文件名换成 handle-failed-sql-statements.md,这样更有助于从字面上理解

zh/handle-error-sql-statements.md Outdated Show resolved Hide resolved
"ErrClass": "schema-tracker",
"ErrScope": "internal",
"ErrLevel": "high",
"Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4]: cannot track DDL: ALTER TABLE `shard_db_1`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

"ErrClass": "schema-tracker",
"ErrScope": "internal",
"ErrLevel": "high",
"Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4]: cannot track DDL: ALTER TABLE `shard_db_1`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

或者可以用 <details> <summary> HTML 标签进行折叠(不过尽量能省则省吧,这么长就算折叠起来也很难看):

执行结果
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-replica-01",
                "worker": "worker1",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 44006,
                                "ErrClass": "schema-tracker",
                                "ErrScope": "internal",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4]: cannot track DDL: ALTER TABLE `shard_db_1`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
                                "RawCause": "[ddl:8200]Unsupported modify charset from latin1 to utf8",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "2",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(DESKTOP-T561TSO-bin.000001, 1848)",
                        "masterBinlogGtid": "143bdef3-dd4a-11ea-8b00-00155de45f57:1-8",
                        "syncerBinlog": "(DESKTOP-T561TSO-bin.000001, 1420)",
                        "syncerBinlogGtid": "143bdef3-dd4a-11ea-8b00-00155de45f57:1-4",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                            {
                                "target": "`shard_db`.`shard_table`",
                                "DDLs": [
                                    "ALTER TABLE `shard_db`.`shard_table` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI"
                                ],
                                "firstLocation": "position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 143bdef3-dd4a-11ea-8b00-00155de45f57:1-4",
                                "synced": [
                                    "`shard_db_1`.`shard_table_1`"
                                ],
                                "unsynced": [
                                    "`shard_db_1`.`shard_table_2`"
                                ]
                            }
                        ],
                        "synced": false,
                        "binlogType": "remote"
                    }
                }
            ]
        },
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-replica-02",
                "worker": "worker2",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 44006,
                                "ErrClass": "schema-tracker",
                                "ErrScope": "internal",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6], endLocation: [position: (DESKTOP-T561TSO-bin.000001, 1634), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-7]: cannot track DDL: ALTER TABLE `shard_db_2`.`shard_table_1` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI",
                                "RawCause": "[ddl:8200]Unsupported modify charset from latin1 to utf8",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "2",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(DESKTOP-T561TSO-bin.000001, 1848)",
                        "masterBinlogGtid": "1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-8",
                        "syncerBinlog": "(DESKTOP-T561TSO-bin.000001, 1420)",
                        "syncerBinlogGtid": "1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                            {
                                "target": "`shard_db`.`shard_table`",
                                "DDLs": [
                                    "ALTER TABLE `shard_db`.`shard_table` CHARACTER SET UTF8 COLLATE UTF8_UNICODE_CI"
                                ],
                                "firstLocation": "position: (DESKTOP-T561TSO-bin.000001, 1420), gtid-set: 1d66ce48-dd4a-11ea-8ef7-00155de45f57:1-6",
                                "synced": [
                                    "`shard_db_2`.`shard_table_1`"
                                ],
                                "unsynced": [
                                    "`shard_db_2`.`shard_table_2`"
                                ]
                            }
                        ],
                        "synced": false,
                        "binlogType": "remote"
                    }
                }
            ]
        }
    ]
}

}
```

可以看到 MySQL 实例1的 `shard_db_1`.`shard_table_1`表和 MySQL 实例2的 `shard_db_2`.`shard_table_1` 表报错。假设业务上可以接受下游 TiDB 不执行此 DDL 语句(即继续保持原有的表结构),则可以通过使用 `handle-error <task-name> skip` 命令跳过该 DDL 语句以恢复同步任务。操作步骤如下:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
可以看到 MySQL 实例1的 `shard_db_1`.`shard_table_1`表和 MySQL 实例2的 `shard_db_2`.`shard_table_1` 表报错。假设业务上可以接受下游 TiDB 不执行此 DDL 语句(即继续保持原有的表结构),则可以通过使用 `handle-error <task-name> skip` 命令跳过该 DDL 语句以恢复同步任务。操作步骤如下:
可以看到 MySQL 实例 1 的 `shard_db_1`.`shard_table_1` 表和 MySQL 实例 2 的 `shard_db_2`.`shard_table_1` 表报错。假设业务上可以接受下游 TiDB 不执行此 DDL 语句(即继续保持原有的表结构),则可以通过使用 `handle-error <task-name> skip` 命令跳过该 DDL 语句以恢复同步任务。操作步骤如下:

文档风格统一的问题:中英文之间、中文和数字之间是需要空格的,麻烦批量改下

GMHDBJD and others added 9 commits August 14, 2020 15:45
Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com>
Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com>
Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com>
Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com>
@GMHDBJD
Copy link
Collaborator Author

GMHDBJD commented Aug 14, 2020

PTAL again. 我直接把错误信息提炼出来了,会不会太简洁了。。。


+ `replace`:替代错误的 DDL 语句

+ `revert`:重置该错误先前的 skip/replace 操作
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

要说明一下仅在先前的操作没有最终生效前执行吗?怕用户以为能够 rollback 到更早的地方去(虽然显然是不会的

zh/handle-failed-sql-statements.md Show resolved Hide resolved
zh/handle-failed-sql-statements.md Outdated Show resolved Hide resolved
zh/handle-failed-sql-statements.md Outdated Show resolved Hide resolved
zh/handle-failed-sql-statements.md Outdated Show resolved Hide resolved
zh/handle-failed-sql-statements.md Outdated Show resolved Hide resolved
» query-status test
```

```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这种太长的结果考虑折叠下不? @GMHDBJD @csuzhangxc 可以使整篇文章看起来更简洁些😂

GMHDBJD and others added 3 commits August 14, 2020 19:47
Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com>
Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com>
Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com>
GMHDBJD and others added 5 commits August 14, 2020 19:48
@ti-srebot
Copy link
Contributor

@csuzhangxc, @yikeke, @ireneontheway, PTAL.

Copy link
Member

@csuzhangxc csuzhangxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 17, 2020
@ireneontheway ireneontheway added the size/large Changes of a large size. label Aug 17, 2020
Copy link
Contributor

@yikeke yikeke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Aug 18, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Aug 18, 2020
@yikeke
Copy link
Contributor

yikeke commented Aug 18, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 18, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit 08f1ead into pingcap:master Aug 18, 2020
yikeke pushed a commit to yikeke/docs-dm that referenced this pull request Aug 21, 2020
@yikeke yikeke mentioned this pull request Aug 21, 2020
6 tasks
@yikeke yikeke added translation/done This PR has been translated. and removed translation/doing This PR’s assignee is translating this PR. labels Aug 21, 2020
yikeke added a commit that referenced this pull request Aug 21, 2020
* align #227

* Update en/handle-failed-sql-statements.md

* Apply suggestions from code review

Co-authored-by: GMHDBJD <35025882+GMHDBJD@users.noreply.github.com>

* Update en/handle-failed-sql-statements.md

* Apply suggestions from code review

Co-authored-by: Lilian Lee <lilin@pingcap.com>

* Update en/handle-failed-sql-statements.md

Co-authored-by: GMHDBJD <35025882+GMHDBJD@users.noreply.github.com>
Co-authored-by: Lilian Lee <lilin@pingcap.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
size/large Changes of a large size. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/PTAL This PR is ready for reviewing. translation/done This PR has been translated. v2.0 This PR/issue applies to TiDB DM v2.0.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants