From 31b811f0a084b7b7f819e44aa8a2f0564d2407fb Mon Sep 17 00:00:00 2001 From: lilin90 Date: Wed, 22 Aug 2018 16:00:30 +0800 Subject: [PATCH 1/4] tools: add two commands in tikv-ctl --- tools/tikv-control.md | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/tools/tikv-control.md b/tools/tikv-control.md index b7ee14c2afc3b..66de8b5d7c5a8 100644 --- a/tools/tikv-control.md +++ b/tools/tikv-control.md @@ -6,7 +6,9 @@ category: tools # TiKV Control User Guide -TiKV Control (`tikv-ctl`) is a command line tool of TiKV, used to manage the cluster. When you compile TiKV, the `tikv-ctl` command is also compiled at the same time. If the cluster is deployed using Ansible, the `tikv-ctl` binary file exists in the corresponding `tidb-ansible/resources/bin` directory. If the cluster is deployed using the binary, the `tikv-ctl` file is in the `bin` directory together with other files such as `tidb-server`, `pd-server`, `tikv-server`, etc. +TiKV Control (`tikv-ctl`) is a command line tool of TiKV, used to manage the cluster. + +When you compile TiKV, the `tikv-ctl` command is also compiled at the same time. If the cluster is deployed using Ansible, the `tikv-ctl` binary file exists in the corresponding `tidb-ansible/resources/bin` directory. If the cluster is deployed using the binary, the `tikv-ctl` file is in the `bin` directory together with other files such as `tidb-server`, `pd-server`, `tikv-server`, etc. ## General options @@ -103,6 +105,23 @@ key: zmDB:29\000\000\377\000\374\000\000\000\000\000\000\377\000H\000\000\000\00 In this command, the key is also the escaped form of raw key. +### Recover from MVCC data corruption + +Use the `recover-mvcc` command in circumstances where TiKV cannot run normally caused by MVCC data corruption. It cross-checks 3 CFs ("default", "write", "lock") to recover from various kinds of inconsistency. + +Use the `--regions` option to specify involved Regions by `region_id`. Use the `--pd`option to specify PD endpoints. + +```bash +$ tikv-ctl --db /path/to/tikv/db recover-mvcc --regions 1001,1002 --pd 127.0.0.1:2379 +success! +``` + +> **Note**: +> +> - This command only supports the local mode. It prints `success!` when successfully run. +> - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether the specified `region_id` is validated or not. +> - You need to run this command for all stores that specify Regions' peers. + ### Print a specific key value To print the value of a key, use the `print` command. @@ -157,16 +176,21 @@ success! > - This command only supports the local mode. > - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether PD can securely switch to Tombstone. Therefore, before setting a PD instance to Tombstone, you need to take off the corresponding Peer of this Region on the machine in `pd-ctl`. -### Force Region to recover the service from multiple replicas failure +### Force Region to recover the service from failure of multiple replicas -Use the `unsafe-recover remove-fail-stores` command to remove the failed machines from the peers list of all Regions. Then after you restart TiKV, these Regions can continue to provide services using the other healthy replicas. This command is usually used in circumstances where multiple TiKV stores are damaged or deleted. +Use the `unsafe-recover remove-fail-stores` command to remove the failed machines from the peer list of Regions. Then after you restart TiKV, these Regions can continue to provide services using the other healthy replicas. This command is usually used in circumstances where multiple TiKV stores are damaged or deleted. + +The `--stores` option accepts multiple `store_id` separated by comma and uses the `--regions` flag to specify involved Regions. Otherwise, all Regions' peers located on these stores will be removed by default. ```bash -$ tikv-ctl --db /path/to/tikv/db unsafe-recover remove-fail-stores 3,4,5 +$ tikv-ctl --db /path/to/tikv/db unsafe-recover remove-fail-stores --stores 3 --regions 1001,1002 success! ``` -> **Note:** This command only supports the local mode. It prints `success!` when successfully run. +> **Note:** +> +> - This command only supports the local mode. It prints `success!` when successfully run. +> - You must run this command for all stores that specify Regions' peers. If `--regions` is not set,all Regions are involved, and you need to run this command for all stores. ### Send a `consistency-check` request to TiKV From f4dc8925364c017f1799baf6361296446f3a524d Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 23 Aug 2018 11:16:00 +0800 Subject: [PATCH 2/4] tools: address the comment --- tools/tikv-control.md | 66 +++++++++++++++++++++---------------------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/tools/tikv-control.md b/tools/tikv-control.md index 66de8b5d7c5a8..f137c9d0e59ad 100644 --- a/tools/tikv-control.md +++ b/tools/tikv-control.md @@ -105,23 +105,6 @@ key: zmDB:29\000\000\377\000\374\000\000\000\000\000\000\377\000H\000\000\000\00 In this command, the key is also the escaped form of raw key. -### Recover from MVCC data corruption - -Use the `recover-mvcc` command in circumstances where TiKV cannot run normally caused by MVCC data corruption. It cross-checks 3 CFs ("default", "write", "lock") to recover from various kinds of inconsistency. - -Use the `--regions` option to specify involved Regions by `region_id`. Use the `--pd`option to specify PD endpoints. - -```bash -$ tikv-ctl --db /path/to/tikv/db recover-mvcc --regions 1001,1002 --pd 127.0.0.1:2379 -success! -``` - -> **Note**: -> -> - This command only supports the local mode. It prints `success!` when successfully run. -> - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether the specified `region_id` is validated or not. -> - You need to run this command for all stores that specify Regions' peers. - ### Print a specific key value To print the value of a key, use the `print` command. @@ -176,22 +159,6 @@ success! > - This command only supports the local mode. > - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether PD can securely switch to Tombstone. Therefore, before setting a PD instance to Tombstone, you need to take off the corresponding Peer of this Region on the machine in `pd-ctl`. -### Force Region to recover the service from failure of multiple replicas - -Use the `unsafe-recover remove-fail-stores` command to remove the failed machines from the peer list of Regions. Then after you restart TiKV, these Regions can continue to provide services using the other healthy replicas. This command is usually used in circumstances where multiple TiKV stores are damaged or deleted. - -The `--stores` option accepts multiple `store_id` separated by comma and uses the `--regions` flag to specify involved Regions. Otherwise, all Regions' peers located on these stores will be removed by default. - -```bash -$ tikv-ctl --db /path/to/tikv/db unsafe-recover remove-fail-stores --stores 3 --regions 1001,1002 -success! -``` - -> **Note:** -> -> - This command only supports the local mode. It prints `success!` when successfully run. -> - You must run this command for all stores that specify Regions' peers. If `--regions` is not set,all Regions are involved, and you need to run this command for all stores. - ### Send a `consistency-check` request to TiKV Use the `consistency-check` command to execute a consistency check among replicas in the corresponding Raft of a specific Region. If the check fails, TiKV itself panics. If the TiKV instance specified by `--host` is not the Region leader, an error is reported. @@ -255,3 +222,36 @@ success! $ tikv-ctl modify-tikv-config -m raftdb -n default.disable_auto_compactions -v true success! ``` + +### Force Region to recover the service from failure of multiple replicas + +Use the `unsafe-recover remove-fail-stores` command to remove the failed machines from the peer list of Regions. Then after you restart TiKV, these Regions can continue to provide services using the other healthy replicas. This command is usually used in circumstances where multiple TiKV stores are damaged or deleted. + +The `--stores` option accepts multiple `store_id` separated by comma and uses the `--regions` flag to specify involved Regions. Otherwise, all Regions' peers located on these stores will be removed by default. + +```bash +$ tikv-ctl --db /path/to/tikv/db unsafe-recover remove-fail-stores --stores 3 --regions 1001,1002 +success! +``` + +> **Note:** +> +> - This command only supports the local mode. It prints `success!` when successfully run. +> - You must run this command for all stores that specify Regions' peers. If `--regions` is not set,all Regions are involved, and you need to run this command for all stores. + +### Recover from MVCC data corruption + +Use the `recover-mvcc` command in circumstances where TiKV cannot run normally caused by MVCC data corruption. It cross-checks 3 CFs ("default", "write", "lock") to recover from various kinds of inconsistency. + +Use the `--regions` option to specify involved Regions by `region_id`. Use the `--pd`option to specify PD endpoints. + +```bash +$ tikv-ctl --db /path/to/tikv/db recover-mvcc --regions 1001,1002 --pd 127.0.0.1:2379 +success! +``` + +> **Note**: +> +> - This command only supports the local mode. It prints `success!` when successfully run. +> - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether the specified `region_id` is validated or not. +> - You need to run this command for all stores that specify Regions' peers. From b3791d01d16ae378fb7c8e18085b0a9c4a357266 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 23 Aug 2018 11:29:17 +0800 Subject: [PATCH 3/4] tools: address comments --- tools/tikv-control.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/tikv-control.md b/tools/tikv-control.md index f137c9d0e59ad..45a4d71c33c68 100644 --- a/tools/tikv-control.md +++ b/tools/tikv-control.md @@ -237,7 +237,7 @@ success! > **Note:** > > - This command only supports the local mode. It prints `success!` when successfully run. -> - You must run this command for all stores that specify Regions' peers. If `--regions` is not set,all Regions are involved, and you need to run this command for all stores. +> - You must run this command for all stores that have specified Regions' peers. If `--regions` is not set, all Regions are involved, and you need to run this command for all stores. ### Recover from MVCC data corruption @@ -254,4 +254,4 @@ success! > > - This command only supports the local mode. It prints `success!` when successfully run. > - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether the specified `region_id` is validated or not. -> - You need to run this command for all stores that specify Regions' peers. +> - You need to run this command for all stores that have specified Regions' peers. From d566e2239271fa47be7008d7da6d6a87dd4af7fe Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Thu, 23 Aug 2018 12:03:06 +0800 Subject: [PATCH 4/4] tools: update wording --- tools/tikv-control.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/tikv-control.md b/tools/tikv-control.md index 45a4d71c33c68..144ea870682d8 100644 --- a/tools/tikv-control.md +++ b/tools/tikv-control.md @@ -237,7 +237,7 @@ success! > **Note:** > > - This command only supports the local mode. It prints `success!` when successfully run. -> - You must run this command for all stores that have specified Regions' peers. If `--regions` is not set, all Regions are involved, and you need to run this command for all stores. +> - You must run this command for all stores where specified Regions' peers locate. If `--regions` is not set, all Regions are involved, and you need to run this command for all stores. ### Recover from MVCC data corruption @@ -254,4 +254,4 @@ success! > > - This command only supports the local mode. It prints `success!` when successfully run. > - The argument of the `--pd/-p` option specifies the PD endpoints without the `http` prefix. Specifying the PD endpoints is to query whether the specified `region_id` is validated or not. -> - You need to run this command for all stores that have specified Regions' peers. +> - You need to run this command for all stores where specified Regions' peers locate.