tools, readme: add pd recover usage document

README.md

@@ -112,6 +112,7 @@
   - [Loader](tools/loader.md)
   - [TiDB-Binlog](tools/tidb-binlog-kafka.md)
   - [PD Control](tools/pd-control.md)
+  - [PD Recover](tools/pd-recover.md)
   - [TiKV Control](tools/tikv-control.md)
   - [TiDB Controller](tools/tidb-controller.md)
 + TiKV Documentation

tools/pd-control.md

@@ -11,7 +11,7 @@ As a command line tool of PD, PD Control obtains the state information of the cl
 ## Source code compiling
 
 1. [Go](https://golang.org/) Version 1.9 or later
-2. In the PD root directory, use the `make` command to compile and generate `bin/pd-ctl`
+2. In the root directory of the [PD project](https://github.com/pingcap/pd), use the `make` command to compile and generate `bin/pd-ctl`
 
 > **Note:** Generally, you don't need to compile source code as the PD Control tool already exists in the released Binary or Docker. However, dev users can refer to the above instruction for compiling source code.
 

tools/pd-recover.md

@@ -0,0 +1,47 @@
+---
+title: PD Recover User Guide
+summary: Use PD Recover to recover a PD cluster which cannot start or provide services normally.
+category: tools
+---
+
+# PD Recover User Guide
+
+PD Recover is a disaster recovery tool of PD, used to recover the PD cluster which cannot start or provide services normally.
+
+## Source code compiling
+
+1. [Go](https://golang.org/) Version 1.9 or later
+2. In the root directory of the [PD project](https://github.com/pingcap/pd), use the `make` command to compile and generate `bin/pd-recover`
+
+## Usage
+
+This section describes how to recover a PD cluster which cannot start or provide services normally.
+
+### Flags description
+
+```
+-alloc-id uint
+      Specify a number larger than the allocated ID of the original cluster
+-cacert string
+      Specify the path to the trusted CA certificate file in PEM format
+-cert string
+      Specify the path to the SSL certificate file in PEM format
+-key string
+      Specify the path to the SSL certificate key file in PEM format, which is the private key of the certificate specified by `--cert`
+-cluster-id uint
+      Specify the Cluster ID of the original cluster
+-endpoints string
+      Specify the PD address (default: "http://127.0.0.1:2379")
+```
+
+### Recovery flow
+
+1. Obtain the Cluster ID and the Alloc ID from the current cluster. 
+
+     - Obtain the Cluster ID from the PD, TiKV and TiDB log.
+     - Obtain the allocated Alloc ID from either the PD log or the `Metadata Information` in the PD monitoring panel. 
+
+     Specifying `alloc-id` requires a number larger than the current largest Alloc ID. If you fail to obtain the Alloc ID, you can make an estimate of a larger number according to the number of Regions and Stores in the cluster. Generally, you can specify a number that is several orders of magnitude larger.
+2. Stop the whole cluster, clear the PD data directory, and restart the PD cluster.
+3. Use PD Recover to recover and make sure that you use the correct `cluster-id` and appropriate `alloc-id`.
+4. When the recovery success information is prompted, restart the whole cluster.