diff --git a/alert-rules.md b/alert-rules.md index a6917a2bd64a4..c284f7ebe5dd3 100644 --- a/alert-rules.md +++ b/alert-rules.md @@ -425,15 +425,11 @@ This section gives the alert rules for the TiKV component. * Alert rule: - `sum(increase(tidb_tikvclient_gc_action_result{type="success"}[6h])) < 1` - - > **Note:** - > - > In TiDB 3.* versions, the `tidb_tikvclient_gc_action_result` metric exists but does not have a value. It's because distributed garbage collection (GC) is introduced in the TiDB 3.0 version but will not be performed in TiDB. + `sum(increase(tikv_gcworker_gc_tasks_vec{task="gc"}[1d])) < 1 and sum(increase(tikv_gc_compaction_filter_perform[1d])) < 1` * Description: - GC is not performed successfully in a Region within 6 hours, which indicates that GC is not working properly. If GC does not run in a short term, it will not cause much trouble; but if GC keeps down, more and more versions are retained, which slows down the query. + GC is not performed successfully on a TiKV instance within 24 hours, which indicates that GC is not working properly. If GC does not run in a short term, it will not cause much trouble; but if GC keeps down, more and more versions are retained, which slows down the query. * Solution: diff --git a/dumpling-overview.md b/dumpling-overview.md index a7b388a55ee83..d7440a2d6a936 100644 --- a/dumpling-overview.md +++ b/dumpling-overview.md @@ -237,7 +237,7 @@ Examples: The exported file is stored in the `./export-` directory by default. Commonly used options are as follows: - `-o` is used to select the directory where the exported files are stored. -- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). +- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). It is recommended to keep its value to 256 MiB or less if you plan to use TiDB Lightning to load this file into a TiDB instance. - `-r` option is used to specify the maximum number of records (or the number of rows in the database) for a single file. When it is enabled, Dumpling enables concurrency in the table to improve the speed of exporting large tables. With the above options specified, Dumpling can have a higher degree of parallelism. diff --git a/error-codes.md b/error-codes.md index e47f2eb57de7c..d59320bc0ee6e 100644 --- a/error-codes.md +++ b/error-codes.md @@ -98,7 +98,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the * Error Number: 8027 - The table schema version is outdated. TiDB uses the F1 online schema change algorithm to execute DDL statements. When the table schema version of the TiDB server is earlier than that of the entire system, this error is returned if you execute a SQL statement. + The table schema version is outdated. TiDB applies schema changes online. When the table schema version of the TiDB server is earlier than that of the entire system, this error is returned if you execute a SQL statement. When this error occurs, check the network between the TiDB server and the PD Leader. diff --git a/grafana-tikv-dashboard.md b/grafana-tikv-dashboard.md index ab6a516cdfc15..03bca8a699a35 100644 --- a/grafana-tikv-dashboard.md +++ b/grafana-tikv-dashboard.md @@ -223,20 +223,17 @@ This document provides a detailed description of these key metrics on the **TiKV ## GC -- MVCC versions: The number of versions for each key -- MVCC delete versions: The number of versions deleted by GC for each key - GC tasks: The count of GC tasks processed by gc_worker - GC tasks Duration: The time consumed when executing GC tasks -- GC keys (write CF): The count of keys in write CF affected during GC -- TiDB GC worker actions: The count of TiDB GC worker actions - TiDB GC seconds: The GC duration -- GC speed: The number of keys deleted by GC per second -- TiKV AutoGC Working: The status of Auto GC +- TiDB GC worker actions: The count of TiDB GC worker actions - ResolveLocks Progress: The progress of the first phase of GC (Resolve Locks) - TiKV Auto GC Progress: The progress of the second phase of GC +- GC speed: The number of keys deleted by GC per second - TiKV Auto GC SafePoint: The value of TiKV GC safe point. The safe point is the current GC timestamp - GC lifetime: The lifetime of TiDB GC - GC interval: The interval of TiDB GC +- GC in Compaction Filter: The count of filtered versions in the compaction filter of write CF. ## Snapshot diff --git a/sql-statements/sql-statement-admin.md b/sql-statements/sql-statement-admin.md index f442839f6988e..8bdf6836adc94 100644 --- a/sql-statements/sql-statement-admin.md +++ b/sql-statements/sql-statement-admin.md @@ -194,7 +194,7 @@ admin show ddl jobs 5 where state!='synced' and db_name='test'; * `JOB_TYPE`: the type of the DDL operations. * `SCHEMA_STATE`: the current state of the schema. If the `JOB_TYPE` is `add index`, it is the state of the index; if the `JOB_TYPE` is `add column`, it is the state of the column; if the `JOB_TYPE` is `create table`, it is the state of the table. The common states include: * `none`: it indicates not existing. When the `drop` or `create` operation fails and rolls back, it usually becomes the `none` state. - * `delete only`, `write only`, `delete reorganization`, `write reorganization`: these four states are intermediate states. For details, see the paper [Online, Asynchronous Schema Change in F1](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/41376.pdf). These states are not visible in common operations, because the conversion from the intermediate states is so quick. You can see the `write reorganization` state only in `add index` operations, which means that the index data is being added. + * `delete only`, `write only`, `delete reorganization`, `write reorganization`: these four states are intermediate states. These states are not visible in common operations, because the conversion from the intermediate states is so quick. You can see the `write reorganization` state only in `add index` operations, which means that the index data is being added. * `public`: it indicates existing and usable. When operations like `create table` and `add index/column` are finished, it usually becomes the `public` state, which means that the created table/column/index can be normally read and written now. * `SCHEMA_ID`: the ID of the database on which the DDL operations are performed. * `TABLE_ID`: the ID of the table on which the DDL operations are performed. diff --git a/tiup/tiup-cluster-topology-reference.md b/tiup/tiup-cluster-topology-reference.md new file mode 100644 index 0000000000000..257894e5b31d2 --- /dev/null +++ b/tiup/tiup-cluster-topology-reference.md @@ -0,0 +1,769 @@ +--- +title: Topology Configuration File for TiDB Deployment Using TiUP +--- + +# Topology Configuration File for TiDB Deployment Using TiUP + +To deploy or scale TiDB using TiUP, you need to provide a topology file ([sample](https://github.com/pingcap/tiup/blob/master/embed/templates/examples/topology.example.yaml)) to describe the cluster topology. + +Similarly, to modify the cluster topology, you need to modify the topology file. The difference is that, after the cluster is deployed, you can only modify a part of the fields in the topology file. This document introduces each section of the topology file and each field in each section. + +## File structure + +A topology configuration file for TiDB deployment using TiUP might contain the following sections: + +- [global](#global): The cluster's global configuration. Some of the configuration items use the default values and you can configure them separately in each instance. +- [monitored](#monitored): Configuration for monitoring services, namely, the blackbox_exporter and the `node_exporter`. On each machine, a `node_exporter` and a `blackbox_exporter` are deployed. +- [server_configs](#server_configs): Components' global configuration. You can configure each component separately. If an instance has a configuration item with the same name, the instance's configuration item will take effect. +- [pd_servers](#pd_servers): The configuration of the PD instance. This configuration specifies the machines to which the PD component is deployed. +- [tidb_servers](#tidb_servers): The configuration of the TiDB instance. This configuration specifies the machines to which the TiDB component is deployed. +- [tikv_servers](#tikv_servers): The configuration of the TiKV instance. This configuration specifies the machines to which the TiKV component is deployed. +- [tiflash_servers](#tiflash_servers): The configuration of the TiFlash instance. This configuration specifies the machines to which the TiFlash component is deployed. +- [pump_servers](#pump_servers): The configuration of the Pump instance. This configuration specifies the machines to which the Pump component is deployed. +- [drainer_servers](#drainer_servers): The configuration of the Drainer instance. This configuration specifies the machines to which the Drainer component is deployed. +- [cdc_servers](#cdc_servers): The configuration of the TiCDC instance. This configuration specifies the machines to which the TiCDC component is deployed. +- [tispark_masters](#tispark_masters): The configuration of the TiSpark master instance. This configuration specifies the machines to which the TiSpark master component is deployed. Only one node of TiSpark master can be deployed. +- [tispark_workers](#tispark_workers): The configuration of the TiSpark worker instance. This configuration specifies the machines to which the TiSpark worker component is deployed. +- [monitoring_servers](#monitoring_servers): Specifies the machines to which Prometheus is deployed. TiUP supports deploying multiple Prometheus instances but only the first instance is used. +- [grafana_servers](#grafana_servers): The configuration of the Grafana instance. This configuration specifies the machines to which Grafana is deployed. +- [alertmanager_servers](#alertmanager_servers): The configuration of the Alertmanager instance. This configuration specifies the machines to which Alertmanager is deployed. + +### `global` + +The `global` section corresponds to the cluster's global configuration and has the following fields: + +- `user`: The user used to start the deployed cluster. The default value is `"tidb"`. If the user specified in the `` field does not exist on the target machine, this user is automatically created. + +- `group`: The user group to which a user belongs. It is specified when the user is created. The value defaults to that of the `` field. If the specified group does not exist, it is automatically created. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. The default value is `22`. + +- `enable_tls`: Specifies whether to enable TLS for the cluster. After TLS is enabled, the generated TLS certificate must be used for connections between components or between the client and the component. **Once it is enabled, it cannot be disabled**. The default value is `false`. + +- `deploy_dir`: The deployment directory of each component. The default value is `"deployed"`. Its application rules are as follows: + + - If the absolute path of `deploy_dir` is configured at the instance level, the actual deployment directory is `deploy_dir` configured for the instance. + + - For each instance, if you do not configure `deploy_dir`, its default value is the relative path `-`. + + - If `global.deploy_dir` is an absolute path, the component is deployed to the `/` directory. + + - If `global.deploy_dir` is a relative path, the component is deployed to the `/home///` directory. + +- `data_dir`: The data directory. Default value: `"data"`. Its application rules are as follows: + + - If the absolute path of `data_dir` is configured at the instance level, the actual deployment directory is `data_dir` configured for the instance. + + - For each instance, if you do not configure `data_dir`, its default value is ``. + + - If `data_dir` is a relative path, the component data is placed in `/`. For the calculation rules of ``, see the application rules of the `deploy_dir` field. + +- `log_dir`: The data directory. Default value: `"log"`. Its application rules are as follows: + + - If the absolute path `log_dir` is configured at the instance level, the actual log directory is the `log_dir` configured for the instance. + + - For each instance, if you not configure `log_dir`, its default value is ``. + + - If `log_dir` is a relative path, the component log is placed in `/`. For the calculation rules of ``, see the application rules of the `deploy_dir` field. + +- `os`: The operating system of the target machine. The field controls which operating system to adapt to for the components pushed to the target machine. The default value is "linux". + +- `arch`: The CPU architecture of the target machine. The field controls which platform to adapt to for the binary packages pushed to the target machine. The supported values are "amd64" and "arm64". The default value is "amd64". + +- `resource_control`: Runtime resource control. All configurations in this field are written into the service file of systemd. There is no limit by default. The resources that can be controlled are as follows: + + - `memory_limit`: Limits the maximum runtime memory. For example, "2G" means that the maximum memory of 2 GB can be used. + + - `cpu_quota`: Limits the maximum CPU usage at runtime. For example, "200%". + + - `io_read_bandwidth_max`: Limits the maximum I/O bandwidth for disk reads. For example, `"/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0 100M"`. + + - `io_write_bandwidth_max`: Limits maximum I/O bandwidth for disk writes. For example, `/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0 100M`. + + - `limit_core`: Controls the size of core dump. + +A `global` configuration example is as follows: + +```yaml +global: + user: "tidb" + resource_control: + memory_limit: "2G" +``` + +In the above configuration, the `tidb` user is used to start the cluster. At the same time, each component is restricted to a maximum of 2 GB of memory when it is running. + +### `monitored` + +`monitored` is used to configure the monitoring service on the target machine: [`node_exporter`](https://github.com/prometheus/node_exporter) and [`blackbox_exporter`](https://github.com/prometheus/blackbox_exporter). The following fields are included: + +- `node_exporter_port`: The service port of `node_exporter`. The default value is `9100`. + +- `blackbox_exporter_port`: The service port of `blackbox_exporter`. The default value is `9115`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +A `monitored` configuration example is as follows: + +```yaml +monitored: + node_exporter_port: 9100 + blackbox_exporter_port: 9115 +``` + +The above configuration specifies that `node_exporter` uses the `9100` port and `blackbox_exporter` uses the `9115` port. + +### `server_configs` + +`server_configs` is used to configure services and to generate configuration files for each component. Similar to the `global` section, the configuration of this section can be overwritten by the configurations with the same names in an instance. `server_configs` mainly includes the following fields: + +- `tidb`: TiDB service-related configuration. For the complete configuration, see [TiDB configuration file](/tidb-configuration-file.md). + +- `tikv`: TiKV service-related configuration. For the complete configuration, see [TiKV configuration file](/tikv-configuration-file.md). + +- `pd`: PD service-related configuration. For the complete configuration, see [PD configuration file](/pd-configuration-file.md). + +- `tiflash`: TiFlash service-related configuration. For the complete configuration, see [TiFlash configuration file](/tiflash/tiflash-configuration.md). + +- `tiflash_learner`: Each TiFlash node has a special built-in TiKV. This configuration item is used to configure this special TiKV. It is generally not recommended to modify the content under this configuration item. + +- `pump`: Pump service-related configuration. For the complete configuration, see [TiDB Binlog configuration file](/tidb-binlog/tidb-binlog-configuration-file.md#pump). + +- `drainer`: Drainer service-related configuration. For the complete configuration, see [TiDB Binlog configuration file](/tidb-binlog/tidb-binlog-configuration-file.md#drainer). + +- `cdc`: TiCDC service-related configuration. For the complete configuration, see [Deploy TiCDC](/ticdc/deploy-ticdc.md). + +A `server_configs` configuration example is as follows: + +```yaml +server_configs: + tidb: + run-ddl: true + lease: "45s" + split-table: true + token-limit: 1000 + tikv: + log-level: "info" + readpool.unified.min-thread-count: 1 +``` + +The above configuration specifies the global configuration of TiDB and TiKV. + +### `pd_servers` + +`pd_servers` specifies the machines to which PD services are deployed. It also specifies the service configuration on each machine. `pd_servers` is an array, and each element of the array contains the following fields: + +- `host`: Specifies the machine to which the PD services are deployed. The field value is an IP address and is mandatory. + +- `listen_host`: When the machine has multiple IP addresses, `listen_host` specifies the listening IP address of the service. The default value is `0.0.0.0`. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `name`: Specifies the name of the PD instance. Different instances must have unique names; otherwise, instances cannot be deployed. + +- `client_port`: Specifies the port that PD uses to connect to the client. The default value is `2379`. + +- `peer_port`: Specifies the port for communication between PDs. The default value is `2380`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `pd` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `pd` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `listen_host` +- `name` +- `client_port` +- `peer_port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `pd_servers` configuration example is as follows: + +```yaml +pd_servers: + - host: 10.0.1.11 + config: + schedule.max-merge-region-size: 20 + schedule.max-merge-region-keys: 200000 + - host: 10.0.1.12 +``` + +The above configuration specifies that PD will be deployed on `10.0.1.11` and `10.0.1.12`, and makes specific configurations for the PD of `10.0.1.11`. + +### `tidb_servers` + +`tidb_servers` specifies the machines to which TiDB services are deployed. It also specifies the service configuration on each machine. `tidb_servers` is an array, and each element of the array contains the following fields: + +- `host`: Specifies the machine to which the TiDB services are deployed. The field value is an IP address and is mandatory. + +- `listen_host`: When the machine has multiple IP addresses, `listen_host` specifies the listening IP address of the service. The default value is `0.0.0.0`. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of TiDB services, which is used to provide connection to the MySQL client. The default value is `4000`. + +- `status_port`: The listening port of the TiDB status service, which is used to view the status of the TiDB services from the external via HTTP requests. The default value is `10080`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `tidb` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `tidb` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `listen_host` +- `port` +- `status_port` +- `deploy_dir` +- `log_dir` +- `arch` +- `os` + +A `tidb_servers` configuration example is as follows: + +```yaml +tidb_servers: + - host: 10.0.1.14 + config: + log.level: warn + log.slow-query-file: tidb-slow-overwrited.log + - host: 10.0.1.15 +``` + +### `tikv_servers` + +`tikv_servers` specifies the machines to which TiKV services are deployed. It also specifies the service configuration on each machine. `tikv_servers` is an array, and each element of the array contains the following fields: + +- `host`: Specifies the machine to which the TiKV services are deployed. The field value is an IP address and is mandatory. + +- `listen_host`: When the machine has multiple IP addresses, `listen_host` specifies the listening IP address of the service. The default value is `0.0.0.0`. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of the TiKV services. The default value is `20160`. + +- `status_port`: The listening port of the TiKV status service. The default value is `20180`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `tikv` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `tikv` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `listen_host` +- `port` +- `status_port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `tikv_servers` configuration example is as follows: + +```yaml +tikv_servers: + - host: 10.0.1.14 + config: + server.labels: { zone: "zone1", host: "host1" } + - host: 10.0.1.15 + config: + server.labels: { zone: "zone1", host: "host2" } +``` + +### `tiflash_servers` + +`tiflash_servers` specifies the machines to which TiFlash services are deployed. It also specifies the service configuration on each machine. This section is an array, and each element of the array contains the following fields: + +- `host`: Specifies the machine to which the TiFlash services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `tcp_port`: The port of the TiFlash TCP service. The default value is `9000`. + +- `http_port`: The port of the TiFlash HTTP service. The default value is `8123`. + +- `flash_service_port`: The port via which TiFlash provides services. TiDB reads data from TiFlash via this port. The default value is `3930`. + +- `metrics_port`: TiFlash's status port, which is used to output metric data. The default value is `8234`. + +- `flash_proxy_port`: The port of the built-in TiKV. The default value is `20170`. + +- `flash_proxy_status_port`: The status port of the built-in TiKV. The default value is `20292`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. TiFlash supports multiple `data_dir` directories separated by commas. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `tmp_path`: The storage path of TiFlash temporary files. The default value is [`path` or the first directory of `storage.latest.dir`] + "/tmp". + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `tiflash` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `tiflash` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `learner_config`: Each TiFlash node has a special built-in TiKV. This configuration item is used to configure this special TiKV. It is generally not recommended to modify the content under this configuration item. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +After the deployment, for the fields above, you can only add directories to `data_dir`; for the fields below, you cannot modified these fields: + +- `host` +- `tcp_port` +- `http_port` +- `flash_service_port` +- `flash_proxy_port` +- `flash_proxy_status_port` +- `metrics_port` +- `deploy_dir` +- `log_dir` +- `tmp_path` +- `arch` +- `os` + +A `tiflash_servers` configuration example is as follows: + +```yaml +tiflash_servers: + - host: 10.0.1.21 + - host: 10.0.1.22 +``` + +### `pump_servers` + +`pump_servers` specifies the machines to which the Pump services of TiDB Binlog are deployed. It also specifies the service configuration on each machine. `pump_servers` is an array, and each element of the array contains the following fields: + +- `host`: Specifies the machine to which the Pump services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of the Pump services. The default value is `8250`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `pump` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `pump` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `pump_servers` configuration example is as follows: + +```yaml +pump_servers: + - host: 10.0.1.21 + config: + gc: 7 + - host: 10.0.1.22 +``` + +### `drainer_servers` + +`drainer_servers` specifies the machines to which the Drainer services of TiDB Binlog are deployed. It also specifies the service configuration on each machine. `drainer_servers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the Drainer services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of Drainer services. The default value is `8249`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `commit_ts`: When Drainer starts, it reads the checkpoint. If Drainer cannot read the checkpoint, it uses this field as the replication time point for the initial startup. This field defaults to `-1` (Drainer always gets the latest timestamp from the PD as the commit_ts). + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The configuration rule of this field is the same as the `drainer` configuration rule in `server_configs`. If this field is configured, the field content is merged with the `drainer` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `commit_ts` +- `arch` +- `os` + +A `drainer_servers` configuration example is as follows: + +```yaml +drainer_servers: + - host: 10.0.1.21 + config: + syncer.db-type: "mysql" + syncer.to.host: "127.0.0.1" + syncer.to.user: "root" + syncer.to.password: "" + syncer.to.port: 3306 + syncer.ignore-table: + - db-name: test + tbl-name: log + - db-name: test + tbl-name: audit +``` + +### `cdc_servers` + +`cdc_servers` specifies the machines to which the TiCDC services are deployed. It also specifies the service configuration on each machine. `cdc_servers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the TiCDC services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of the TiCDC services. The default value is `8300`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `gc-ttl`: The Time To Live (TTL) duration of the service level GC safepoint set by TiCDC in PD, in seconds. The default value is `86400`, which is 24 hours. + +- `tz`: The time zone that the TiCDC services use. TiCDC uses this time zone when internally converting time data types such as timestamp and when replicating data to the downstream. The default value is the local time zone where the process runs. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config`: The field content is merged with the `cdc` content in `server_configs` (if the two fields overlap, the content of this field takes effect). Then, a configuration file is generated and sent to the machine specified in `host`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `log_dir` +- `gc-ttl` +- `tz` +- `arch` +- `os` + +A `cdc_servers` configuration example is as follows: + +```yaml +cdc_servers: + - host: 10.0.1.20 + gc-ttl: 86400 + - host: 10.0.1.21 +``` + +### `tispark_masters` + +`tispark_masters` specifies the machines to which the master node of TiSpark is deployed. It also specifies the service configuration on each machine. `tispark_masters` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the TiSpark master is deployed. The field value is an IP address and is mandatory. + +- `listen_host`: When the machine has multiple IP addresses, `listen_host` specifies the listening IP address of the service. The default value is `0.0.0.0`. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: Spark's listening port, used for communication before the node. The default value is `7077`. + +- `web_port`: Spark's web port, which provides web services and the task status. The default value is `8080`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `java_home`: Specifies the path of the JRE environment to be used. This parameter corresponds to the `JAVA_HOME` system environment variable. + +- `spark_config`: Configures to configure the TiSpark services. Then, a configuration file is generated and sent to the machine specified in `host`. + +- `spark_env`: Configures the environment variables when Spark starts. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `listen_host` +- `port` +- `web_port` +- `deploy_dir` +- `arch` +- `os` + +A `tispark_masters` configuration example is as follows: + +```yaml +tispark_masters: + - host: 10.0.1.21 + spark_config: + spark.driver.memory: "2g" + spark.eventLog.enabled: "False" + spark.tispark.grpc.framesize: 2147483647 + spark.tispark.grpc.timeout_in_sec: 100 + spark.tispark.meta.reload_period_in_sec: 60 + spark.tispark.request.command.priority: "Low" + spark.tispark.table.scan_concurrency: 256 + spark_env: + SPARK_EXECUTOR_CORES: 5 + SPARK_EXECUTOR_MEMORY: "10g" + SPARK_WORKER_CORES: 5 + SPARK_WORKER_MEMORY: "10g" + - host: 10.0.1.22 +``` + +### `tispark_workers` + +`tispark_workers` specifies the machines to which the worker nodes of TiSpark are deployed. It also specifies the service configuration on each machine. `tispark_workers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the TiSpark workers are deployed. The field value is an IP address and is mandatory. + +- `listen_host`: When the machine has multiple IP addresses, `listen_host` specifies the listening IP address of the service. The default value is `0.0.0.0`. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: Spark's listening port, used for communication before the node. The default value is `7077`. + +- `web_port`: Spark's web port, which provides web services and the task status. The default value is `8080`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `java_home`: Specifies the path in which the JRE environment to be used is located. This parameter corresponds to the `JAVA_HOME` system environment variable. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `listen_host` +- `port` +- `web_port` +- `deploy_dir` +- `arch` +- `os` + +A `tispark_workers` configuration example is as follows: + +```yaml +tispark_workers: + - host: 10.0.1.22 + - host: 10.0.1.23 +``` + +### `monitoring_servers` + +`monitoring_servers` specifies the machines to which the Prometheus services are deployed. It also specifies the service configuration on each machine. `monitoring_servers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the monitoring services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of the Prometheus services. The default value is `9090`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `storage_retention`: The retention time of the Prometheus monitoring data. The default value is `"15d"`. + +- `rule_dir`: Specifies a local directory that should contain complete `*.rules.yml` files. These files are transferred to the target machine during the initialization phase of the cluster configuration as the rules for Prometheus. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `monitoring_servers` configuration example is as follows: + +```yaml +monitoring_servers: + - host: 10.0.1.11 + rule_dir: /local/rule/dir +``` + +### `grafana_servers` + +`grafana_servers` specifies the machines to which the Grafana services are deployed. It also specifies the service configuration on each machine. `grafana_servers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the Grafana services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `port`: The listening port of the Grafana services. The default value is `3000`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `username`: The user name on the Grafana login interface. + +- `password`: The password corresponding to Grafana. + +- `dashboard_dir`: Specifies a local directory that should contain complete `dashboard(*.json)` files. These files are transferred to the target machine during the initialization phase of the cluster configuration as the dashboards for Grafana. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +> **Note:** +> +> If the `dashboard_dir` field of `grafana_servers` is configured, after executing the `tiup cluster rename` command to rename the cluster, you need to perform the following operations: +> +> 1. For the `*.json` files in the local dashboards directory, update the value of the `datasource` field to the new cluster name (because `datasource` is named after the cluster name). +> 2. Execute the `tiup cluster reload -R grafana` command. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `port` +- `deploy_dir` +- `arch` +- `os` + +A `grafana_servers` configuration example is as follows: + +```yaml +grafana_servers: + - host: 10.0.1.11 + dashboard_dir: /local/dashboard/dir +``` + +### `alertmanager_servers` + +`alertmanager_servers` specifies the machines to which the Alertmanager services are deployed. It also specifies the service configuration on each machine. `alertmanager_servers` is an array. Each array element contains the following fields: + +- `host`: Specifies the machine to which the Alertmanager services are deployed. The field value is an IP address and is mandatory. + +- `ssh_port`: Specifies the SSH port to connect to the target machine for operations. If it is not specified, the `ssh_port` of the `global` section is used. + +- `web_port`: Specifies the port used that Alertmanager uses to provide web services. The default value is `9093`. + +- `cluster_port`: Specifies the communication port between one Alertmanger and other Alertmanager. The default value is `9094`. + +- `deploy_dir`: Specifies the deployment directory. If it is not specified or specified as a relative directory, the directory is generated according to the `deploy_dir` directory configured in `global`. + +- `data_dir`: Specifies the data directory. If it is not specified or specified as a relative directory, the directory is generated according to the `data_dir` directory configured in `global`. + +- `log_dir`: Specifies the log directory. If it is not specified or specified as a relative directory, the log is generated according to the `log_dir` directory configured in `global`. + +- `numa_node`: Allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is the string type. The field value is the ID of the NUMA node, such as "0,1". + +- `config_file`: Specifies a local file that is transferred to the target machine during the initialization phase of the cluster configuration as the configuration of Alertmanager. + +- `os`: The operating system of the machine specified in `host`. If this field is not specified, the default value is the `os` value in `global`. + +- `arch`: The architecture of the machine specified in `host`. If this field is not specified, the default value is the `arch` value in `global`. + +- `resource_control`: Resource control for the service. If this field is configured, the field content is merged with the `resource_control` content in `global` (if the two fields overlap, the content of this field takes effect). Then, a systemd configuration file is generated and sent to the machine specified in `host`. The configuration rules of `resource_control` are the same as the `resource_control` content in `global`. + +For the above fields, you cannot modify these configured fields after the deployment: + +- `host` +- `web_port` +- `cluster_port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `alertmanager_servers` configuration example is as follows: + +```yaml +alertmanager_servers: + - host: 10.0.1.11 + config_file: /local/config/file + - host: 10.0.1.12 + config_file: /local/config/file +``` diff --git a/tiup/tiup-command-clean.md b/tiup/tiup-command-clean.md index e849216d3635d..fd61ec22b021f 100644 --- a/tiup/tiup-command-clean.md +++ b/tiup/tiup-command-clean.md @@ -8,7 +8,7 @@ The `tiup clean` command is used to clear the data generated during component op ## Syntax -```sh +```shell tiup clean [name] [flags] ``` diff --git a/tiup/tiup-command-completion.md b/tiup/tiup-command-completion.md index 22155f5dca0df..2157fabc1580a 100644 --- a/tiup/tiup-command-completion.md +++ b/tiup/tiup-command-completion.md @@ -13,7 +13,7 @@ If you want to complete `bash` commands, you need to install `bash-completion` f ## Syntax -```sh +```shell tiup completion ``` @@ -25,7 +25,7 @@ tiup completion Write the `tiup completion bash` command into a file and source the file in `.bash_profile`. See the following example: -```sh +```shell tiup completion bash > ~/.tiup.completion.bash printf " @@ -38,6 +38,6 @@ source $HOME/.bash_profile ### zsh -```sh +```shell tiup completion zsh > "${fpath[1]}/_tiup" ``` diff --git a/tiup/tiup-command-env.md b/tiup/tiup-command-env.md index 0e84aa3c17cda..6810f4ebac679 100644 --- a/tiup/tiup-command-env.md +++ b/tiup/tiup-command-env.md @@ -8,7 +8,7 @@ TiUP provides users with flexible and customized interfaces, some of which are i ## Syntax -```sh +```shell tiup env [name1...N] ``` diff --git a/tiup/tiup-command-help.md b/tiup/tiup-command-help.md index c67b9332eab9b..ea59b6bdd4733 100644 --- a/tiup/tiup-command-help.md +++ b/tiup/tiup-command-help.md @@ -8,7 +8,7 @@ The TiUP command-line interface provides users with a wealth of help information ## Syntax -```sh +```shell tiup help [command] ``` diff --git a/tiup/tiup-command-install.md b/tiup/tiup-command-install.md index acf5570708e6a..7d4fc71e060d0 100644 --- a/tiup/tiup-command-install.md +++ b/tiup/tiup-command-install.md @@ -8,7 +8,7 @@ The `tiup install` command is used for component installation. It downloads the ## Syntax -```sh +```shell tiup install [:version] [component2...N] [flags] ``` diff --git a/tiup/tiup-command-list.md b/tiup/tiup-command-list.md index 8cfece3d5d8a4..6d95280b8ab68 100644 --- a/tiup/tiup-command-list.md +++ b/tiup/tiup-command-list.md @@ -8,7 +8,7 @@ The command `tiup list` is used to get the list of available components of a mir ## Syntax -```sh +```shell tiup list [component] [flags] ``` diff --git a/tiup/tiup-command-mirror-clone.md b/tiup/tiup-command-mirror-clone.md index 7dc70a7a1d2e2..466c3eed7ccb2 100644 --- a/tiup/tiup-command-mirror-clone.md +++ b/tiup/tiup-command-mirror-clone.md @@ -8,7 +8,7 @@ The command `tiup mirror clone` is used to clone an existing mirror or clone som ## Syntax -```sh +```shell tiup mirror clone [global version] [flags] ``` diff --git a/tiup/tiup-command-mirror-init.md b/tiup/tiup-command-mirror-init.md new file mode 100644 index 0000000000000..a549e13ffabdc --- /dev/null +++ b/tiup/tiup-command-mirror-init.md @@ -0,0 +1,44 @@ +--- +title: tiup mirror init +--- + +# tiup mirror init + +The command `tiup mirror init` is used to initialize an empty mirror. The initialized mirror does not contain any components or component owners. The command only generates the following files for the initialized mirror: + +``` ++ # Mirror's root directory +|-- root.json # Mirror's root certificate +|-- 1.index.json # Component/user index +|-- snapshot.json # Mirror's latest snapshot +|-- timestamp.json # Mirror's latest timestamp +|--+ keys # Mirror's private key (can be moved to other locations) + |-- {hash1..hashN}-root.json # Private key of the root certificate + |-- {hash}-index.json # Private key of the indexes + |-- {hash}-snapshot.json # Private key of the snapshots + |-- {hash}-timestamp.json # Private key of the timestamps +``` + +For the specific usage and content format of the above files, refer to [TiUP Mirror Reference Guide](/tiup/tiup-mirror-reference.md). + +## Syntax + +```shell +tiup mirror init [flags] +``` + +`` is used to specify a local directory where TiUP generates and stores mirror files. The local directory can be a relative path. If the specified directory already exists, it must be empty; if it does not exist, TiUP creates it automatically. + +## Options + +### -k, --key-dir + +- Specifies the directory where TiUP generates private key files. If the specified directory does not exist, TiUP automatically creates it. +- Data type: `STRING` +- If this option is not specified in the command, TiUP generates private key files in `{path}/keys` by default. + +### Outputs + +- If the command is executed successfully, there is no output. +- If the specified `` is not empty, TiUP reports the error `Error: the target path '%s' is not an empty directory`. +- If the specified `` is not a directory, TiUP reports the error `Error: fdopendir: not a directory`. diff --git a/tiup/tiup-command-mirror-merge.md b/tiup/tiup-command-mirror-merge.md new file mode 100644 index 0000000000000..6e93d3a2330b5 --- /dev/null +++ b/tiup/tiup-command-mirror-merge.md @@ -0,0 +1,30 @@ +--- +title: tiup mirror merge +--- + +# tiup mirror merge + +The `tiup mirror merge` command is used to merge one or more mirrors to the current mirror. + +To execute this command, the following conditions must be met: + +- The owner IDs of all components of the target mirror exist in the current mirror. +- The `${TIUP_HOME}/keys` directory of the user who executes this command contains all the private keys corresponding to the above owner IDs in the current mirror (you can use the command [`tiup mirror set`](/tiup/tiup-command-mirror-set.md) to switch the current mirror to the mirror that is currently authorized to modify). + +## Syntax + +```shell +tiup mirror merge [mirror-dir-N] [flags] +``` + +- ``: the first mirror to be merged into the current mirror +- `[mirror-dir-N]`: the Nth mirror to be merged into the current mirror + +## Option + +None + +## Outputs + +- If the command is executed successfully, there is no output. +- If the current mirror does not have a component owner of the target mirror, or if `${TIUP_HOME}/keys` does not have the owner's private key, TiUP reports the `Error: missing owner keys for owner %s on component %s` error. diff --git a/tiup/tiup-command-mirror-modify.md b/tiup/tiup-command-mirror-modify.md new file mode 100644 index 0000000000000..9e5fe1e9b5da3 --- /dev/null +++ b/tiup/tiup-command-mirror-modify.md @@ -0,0 +1,62 @@ +--- +title: tiup mirror modify +--- + +# tiup mirror modify + +The `tiup mirror modify` command is used to modify published components. Only valid component owners can modify the components that they have published on their own. For the method to publish a component, refer to the [`publish` command](/tiup/tiup-command-mirror-publish.md). + +## Syntax + +```shell +tiup mirror modify [:version] [flags] +``` + +Each parameter is explained as follows: + +- ``: the component name +- `[version]`: the component version to modify. If it is not specified, the entire component is modified. + +## Options + +### -k, --key + +- Specifies the component owner's private key used for signing the component information (`{component}.json`). +- Data type: `STRING` +- If this option is not specified in the command, `"${TIUP_HOME}/keys/private.json"` is used by default to sign the component information. + +### --yank + +Marks a specified component or version as unavailable. + +- After the component is marked as unavailable, you can neither see it in the result list of `tiup list` nor install the new version of the component. +- After a component version is marked as unavailable, you can neither see it in the result list of `tiup list ` nor install this version. +- Data type: `BOOLEAN` +- This option is disabled by default and its default value is `false`. To enable this option, you can add this option to the command, and pass the `true` value or do not pass any value. + +### --hide + +- Specifies whether to hide the component. If a component is hidden, you cannot see it in the result list of `tiup list`. To see the hidden component, you can use `tiup list --all`. +- Data type: `BOOLEAN` +- This option is disabled by default and its default value is `false`. To enable this option, you can add this option to the command, and pass the `true` value or do not pass any value. + +> **Note:** +> +> This option can only be applied to the component, not to the component version. + +### --standalone + +- Controls whether the component can run standalone. This option is currently **NOT available**. +- Data type: `BOOLEAN` +- This option is disabled by default and its default value is `false`. To enable this option, you can add this option to the command, and pass the `true` value or do not pass any value. + +> **Note:** +> +> This option can only be applied to the component, not to the component version. + +## Outputs + +- If the command is executed successfully, there is no output. +- If the component owner is not authorized to modify the target component: + - If the mirror is a remote mirror, TiUP reports the error `Error: The server refused, make sure you have access to this component`. + - If the mirror is a local mirror, TiUP reports the error `Error: the signature is not correct`. diff --git a/tiup/tiup-command-mirror-rotate.md b/tiup/tiup-command-mirror-rotate.md new file mode 100644 index 0000000000000..212516e86d03d --- /dev/null +++ b/tiup/tiup-command-mirror-rotate.md @@ -0,0 +1,60 @@ +--- +title: tiup mirror rotate +--- + +# tiup mirror rotate + +`root.json` is an important file in a TiUP mirror. It stores the public keys needed for the entire system and is the basis of the chain of trust in TiUP. It mainly contains the following parts: + +- Signatures of mirror administrators. For the official mirror, there are five signatures. For an initialized mirror, there are three signatures by default. +- The public keys used to verify the following files: + - root.json + - index.json + - snapshot.json + - timestamp.json +- Expiration date of `root.json`. For the official mirror, the expiration date is one year later than the creation date of `root.json`. + +For detailed description of TiUP mirror, see [TiUP Mirror Reference](/tiup/tiup-mirror-reference.md). + +You need to update `root.json` in the following cases: + +- Replace the key of the mirror. +- Update the expiration date of certificate files. + +After the content of `root.json` is updated, the file must be re-signed by all administrators; otherwise, the client rejects the file. The update process is as follows: + +1. The user (client) updates the content of `root.json`. +2. All administrators sign the new `root.json` file. +3. tiup-server updates `snapshot.json` to record the version of the new `root.json` file. +4. tiup-server signs the new `snapshot.json` file. +5. tiup-server updates `timestamp.json` to record the hash value of the new `snapshot.json` file. +6. tiup-server signs the new `timestamp.json` file. + +TiUP uses the command `tiup mirror rotate` to automate the above process. + +> **Note:** +> +> + For TiUP versions earlier than v1.3.0, running this command does not returns a correct new `root.json` file. See [#983](https://github.com/pingcap/tiup/issues/983). +> + Before using this command, make sure that all TiUP clients are upgraded to v1.3.0 or a later version. + +## Syntax + +```shell +tiup mirror rotate [flags] +``` + +After executing this command, TiUP starts an editor for the user to modify the file content to the target value, such as changing the value of the `expires` field to a later date. Then, TiUP changes the `version` field from `N` to `N+1` and saves the file. After the file is saved, TiUP starts a temporary HTTP server and waits for all mirror administrators to sign the file. + +For how mirror administrators sign files, refer to the `sign` command. + +## Options + +### --addr + +- Specifies the listening address of the temporary server. You need to make sure that the address is accessible to other mirror administrators so that they can use the `sign` command to sign the file. +- Data type: `STRING` +- If this option is not specified in the command, TiUP listens on `0.0.0.0:8080` by default. + +## Outputs + +The current signature status of each mirror administrator. diff --git a/tiup/tiup-command-mirror-set.md b/tiup/tiup-command-mirror-set.md index 9e570af1133fe..241fb4db86e18 100644 --- a/tiup/tiup-command-mirror-set.md +++ b/tiup/tiup-command-mirror-set.md @@ -10,7 +10,7 @@ The address of the official mirror is `https://tiup-mirrors.pingcap.com`. ## Syntax -```sh +```shell tiup mirror set [flags] ``` diff --git a/tiup/tiup-command-mirror-sign.md b/tiup/tiup-command-mirror-sign.md new file mode 100644 index 0000000000000..397d07cefe660 --- /dev/null +++ b/tiup/tiup-command-mirror-sign.md @@ -0,0 +1,47 @@ +--- +title: tiup mirror sign +--- + +# tiup mirror sign + +The `tiup mirror sign` command is used to sign the metadata files (*.json）defined in TiUP [mirror](/tiup/tiup-mirror-reference.md). These metadata files might be stored on the local file system or remotely stored using the HTTP protocol to provide a signature entry. + +## Syntax + +```shell +tiup mirror sign [flags] +``` + +`` is the address of the file to be signed, which has two forms: + +- Network address, which starts with HTTP or HTTPS, such as `http://172.16.5.5:8080/rotate/root.json` +- Local file path, which is a relative path or an absolute path + +If it is a network address, this address must provide the following features: + +- Supports the access via `http get` that returns the complete content of the signed file (including the `signatures` field). +- Supports the access via `http post`. The client adds the signature to the `signatures` field of the content that is returned by `http get` and posts to this network address. + +## Options + +### -k, --key + +- Specifies the location of the private key used for signing the `{component}.json` file. +- Data type: `STRING` +- - If this option is not specified in the command, `"${TIUP_HOME}/keys/private.json"` is used by default. + +### --timeout + +- Specifies the access timeout time for signing through the network. The unit is in seconds. +- Data type: `INT` +- Default: 10 + +> **Note:** +> +> This option is valid only when `` is a network address. + +## Output + +- If the command is executed successfully, there is no output. +- If the file has been signed by the specified key, TiUP reports the error `Error: this manifest file has already been signed by specified key`. +- If the file is not a valid manifest, TiUP reports the error `Error: unmarshal manifest: %s`. diff --git a/tiup/tiup-command-mirror.md b/tiup/tiup-command-mirror.md new file mode 100644 index 0000000000000..61394d6e6b4c1 --- /dev/null +++ b/tiup/tiup-command-mirror.md @@ -0,0 +1,37 @@ +--- +title: tiup mirror +--- + +# tiup mirror + +In TiUP, [mirror](/tiup/tiup-mirror-reference.md) is an important concept. TiUP currently supports two forms of mirroring: + +- Local mirror: the TiUP client and the mirror are on the same machine, and the client accesses the mirror through the file system. +- Remote mirror: the TiUP client and the mirror are not on the same machine, and the client accesses the mirror through network. + +The `tiup mirror` command is used to manage mirrors and provides ways to create mirrors, distribute components, and manage keys. + +## Syntax + +```shell +tiup mirror [flags] +``` + +`` stands for sub-commands. For the list of supported sub-commands, refer to the [command list](#command-list) below. + +## Option + +None + +## Command list + +- [genkey](/tiup/tiup-command-mirror-genkey.md): generates the private key file +- [sign](/tiup/tiup-command-mirror-sign.md): signs a specific file using a private key file +- init: initiates an empty mirror +- [set](/tiup/tiup-command-mirror-set.md): sets the current mirror +- [grant](/tiup/tiup-command-mirror-grant.md): grants a new component owner for the current mirror +- [publish](/tiup/tiup-command-mirror-publish.md): publishes new components to the current mirror +- [modify](/tiup/tiup-command-mirror-modify.md): modifies the attributes of the components in the current mirror +- rotate: updates the root certificate in the current mirror +- [clone](/tiup/tiup-command-mirror-clone.md): clones a new mirror from an existing one +- [merge](/tiup/tiup-command-mirror-merge.md): merges mirrors diff --git a/tiup/tiup-command-status.md b/tiup/tiup-command-status.md index bdddcdb8b9b3d..78791031b4c46 100644 --- a/tiup/tiup-command-status.md +++ b/tiup/tiup-command-status.md @@ -15,7 +15,7 @@ The `tiup status` command is used to view the operation information of the compo ## Syntax -```sh +```shell tiup status [flags] ``` diff --git a/tiup/tiup-component-cluster-check.md b/tiup/tiup-component-cluster-check.md new file mode 100644 index 0000000000000..cd3a78f88fb81 --- /dev/null +++ b/tiup/tiup-component-cluster-check.md @@ -0,0 +1,216 @@ +--- +title: tiup cluster check +--- + +# tiup cluster check + +For a formal production environment, before the environment goes live, you need to perform a series of checks to ensure the clusters are in their best performance. To simplify the manual check steps, TiUP Cluster provides the `check` command to check whether the hardware and software environments of the target machines of a specified cluster meet the requirements to work normally. + +## List of check items + +### Operating system version + +Check the operating system distribution and version of the deployed machines. Currently, only CentOS 7 is supported for deployment. More system versions may be supported in later releases for compatibility improvement. + +### CPU EPOLLEXCLUSIVE + +Check whether the CPU of the target machine supports EPOLLEXCLUSIVE. + +### numactl + +Check whether numactl is installed on the target machine. If tied cores are configured on the target machine, you must install numactl. + +### System time + +Check whether the system time of the target machine is synchronized. Compare the system time of the target machine with that of the central control machine, and report an error if the deviation exceeds a certain threshold (500ms). + +### Time synchronization service + +Check whether the time synchronization service is configured on the target machine. Namely, check whether ntpd is running. + +### Swap partitioning + +Check whether swap partitioning is enabled on the target machine. It is recommended to disable swap partitioning. + +### Kernel parameters + +Check the values of the following kernel parameters: + +- `net.ipv4.tcp_tw_recycle`: 0 +- `net.ipv4.tcp_syncookies`: 0 +- `net.core.somaxconn`: 32768 +- `vm.swappiness`: 0 +- `vm.overcommit_memory`: 0 or 1 +- `fs.file-max`: 1000000 + +### Transparent Huge Pages (THP) + +Check whether THP is enabled on the target machine. It is recommended to disable THP. + +### System limits + +Check the limit values in the `/etc/security/limits.conf` file: + +``` + soft nofile 1000000 + hard nofile 1000000 + soft stack 10240 +``` + +`` is the user who deploys and runs the TiDB cluster, and the last column is the minimum value required for the system. + +### SELinux + +Check whether SELinux is enabled. It is recommended to disable SELinux. + +### Firewall + +Check whether the FirewallD service is enabled. It is recommended to either disable the FirewallD service or add permission rules for each service in the TiDB cluster. + +### irqbalance + +Check whether the irqbalance service is enabled. It is recommended to enable the irqbalance service. + +### Disk mount options + +Check the mount options for ext4 partitions. Make sure the mount options include the nodelalloc option and the noatime option. + +### Port usage + +Check if the ports defined in the topology (including the auto-completion default ports) are already used by the processes on the target machine. + +> **Note:** +> +> The port usage check assumes that a cluster is not started yet. If a cluster is already deployed and started, the port usage check on the cluster fails because the ports must be in use in this case. + +### CPU core number + +Check the CPU information of the target machine. For a production cluster, it is recommended that the number of the CPU logical core is greater than or equal to 16. + +> **Note:** +> +> CPU core number is not checked by default. To enable the check, you need to add the `-enable-cpu` option to the command. + +### Memory size + +Check the memory size of the target machine. For a production cluster, it is recommended that the total memory capacity is greater than or equal to 32GB. + +> **Note:** +> +> Memory size is not checked by default. To enable the check, you need to add the `-enable-mem` option to the command. + +### Fio disk performance test + +Use flexible I/O tester (fio) to test the performance of the disk where `data_dir` is located, including the following three test items: + +- fio_randread_write_latency +- fio_randread_write +- fio_randread + +> **Note:** +> +> The fio disk performance test is not performed by default. To perform the test, you need to add the `-enable-disk` option to the command. + +## Syntax + +```shell +tiup cluster check [flags] +``` + +- If a cluster is not deployed yet, you need to pass the topology.yml file that is used to deploy the cluster. According to the content in this file, tiup-cluster connects to the corresponding machine to perform the check. +- If a cluster is already deployed, you can use the `` as the check object. + +> **Note:** +> +> If `` is used for the check, you need to add the `--cluster` option in the command. + +## Options + +### --apply + +- Attempts to automatically repair the failed check items. Currently, tiup-cluster only attempts to repair the following check items: + - SELinux + - firewall + - irqbalance + - kernel parameters + - System limits + - THP (Transparent Huge Pages) +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --cluster + +- Indicates that the check is for the deployed clusters. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +> **Note:** +> +> tiup-cluster supports checking both un-deployed clusters and deployed clusters with the following command format: +> +> ```shell +> tiup cluster check [flags] +> ``` +> +> If the `tiup cluster check ` command is used, you must add the `--cluster` option: `tiup cluster check --cluster`. + +### --enable-cpu + +- Enables the check of CPU core number. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --enable-disk + +- Enables the fio disk performance test. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --enable-mem + +- Enables the memory size check. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --u, --user + +- Specifies the user name to connect to the target machine. The specified user needs to have the password-free sudo root privileges on the target machine. +- Data type: `STRING` +- If this option is not specified in the command, the user who executes the command is used as the default value. + +> **Note:** +> +> This option is valid only if the `-cluster` option is false. Otherwise, the value of this option is fixed to the username specified in the topology file for the cluster deployment. + +### -i, --identity_file + +- Specifies the key file to connect to the target machine. +- Data type: `STRING` +- The option is enabled by default with `~/.ssh/id_rsa` (the default value) passed in. + +> **Note:** +> +> This option is valid only if the `--cluster` option is false. Otherwise, the value of this option is fixed to `${TIUP_HOME}/storage/cluster/clusters//ssh/id_rsa`. + +### -p, --password + +- Logs in with a password when connecting to the target machine. + - If the `--cluster` option is added for a cluster, the password is the password of the user specified in the topology file when the cluster was deployed. + - If the `--cluster` option is not added for a cluster, the password is the password of the user specified in the `-u/--user` option. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -h, --help + +- Prints the help information of the related commands. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +A table containing the following fields: + +- `Node`: the target node +- `Check`: the check item +- `Result`: the check result (Pass, Warn, or Fail) +- `Message`: the result description diff --git a/tiup/tiup-component-cluster-deploy.md b/tiup/tiup-component-cluster-deploy.md index 2f6026642ada9..9da927733bfc2 100644 --- a/tiup/tiup-component-cluster-deploy.md +++ b/tiup/tiup-component-cluster-deploy.md @@ -8,7 +8,7 @@ The `tiup cluster deploy` command is used to deploy a new cluster. ## Syntax -```sh +```shell tiup cluster deploy [flags] ``` @@ -28,11 +28,11 @@ tiup cluster deploy [flags] - Specifies the key file used to connect to the target machine. - Data type: `STRING` -- Default: `~/.ssh/id_rsa` +- If this option is not specified in the command, the `~/.ssh/id_rsa` file is used to connect to the target machine by default. ### -p, --password -- Specifies the password used to connect to the target machine. Do not use it with `-i/--identity_file` at the same time. +- Specifies the password used to connect to the target machine. Do not use this option with `-i/--identity_file` at the same time. - Data type: `BOOLEAN` - This option is disabled by default and its default value is `false`. To enable this option, you can add this option to the command, and pass the `true` value or do not pass any value. diff --git a/tiup/tiup-component-cluster-destroy.md b/tiup/tiup-component-cluster-destroy.md index 68d42263aabe1..d09004f81a489 100644 --- a/tiup/tiup-component-cluster-destroy.md +++ b/tiup/tiup-component-cluster-destroy.md @@ -12,7 +12,7 @@ After an application goes offline, if you want to release the machines occupied ## Syntax -```sh +```shell tiup cluster destroy [flags] ``` diff --git a/tiup/tiup-component-cluster-disable.md b/tiup/tiup-component-cluster-disable.md index 48ddd1c816a77..14142c8c771ec 100644 --- a/tiup/tiup-component-cluster-disable.md +++ b/tiup/tiup-component-cluster-disable.md @@ -18,15 +18,15 @@ tiup cluster disable [flags] ### -N, --node +- Specifies the nodes whose auto-enabling service is to be disabled. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. - Data type: `strings` - Default: `[]`, which means no node is selected. -- Specifies the nodes whose auto-enabling service is to be disabled. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. > **Note:** > > If the `-R, --role` option is specified at the same time, the auto-enabling of services in their intersection is disabled. -### -R, --role strings +### -R, --role - Data type: `strings` - Default: `[]`, which means no role is selected. @@ -38,7 +38,7 @@ tiup cluster disable [flags] ### -h, --help -- Outputs the help information. +- Prints the help information. - Data type: `BOOLEAN` - This option is disabled by default and its default value is `false`. To enable this option, you can add this option to the command, and pass the `true` value or do not pass any value. diff --git a/tiup/tiup-component-cluster-display.md b/tiup/tiup-component-cluster-display.md index 98afffe1a7023..3eed2a1e91c1e 100644 --- a/tiup/tiup-component-cluster-display.md +++ b/tiup/tiup-component-cluster-display.md @@ -24,9 +24,9 @@ tiup cluster display [flags] ### -N, --node +- Specifies the node to query. If not specified, all nodes are queried. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. - Data type: `strings` - Default: `[]`, which means all nodes. -- Specifies the node to query. If not specified, all nodes are queried. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. > **Note:** > @@ -34,9 +34,9 @@ tiup cluster display [flags] ### -R, --role strings +- Specifies the role to query. If not specified, all roles are queried. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. - Data type: `strings` - Default: `[]`, which means all roles. -- Specifies the role to query. If not specified, all roles are queried. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. > **Note:** > diff --git a/tiup/tiup-component-cluster-edit-config.md b/tiup/tiup-component-cluster-edit-config.md index e7543b0a5e87f..a2654c16b9a5c 100644 --- a/tiup/tiup-component-cluster-edit-config.md +++ b/tiup/tiup-component-cluster-edit-config.md @@ -13,7 +13,7 @@ If you need to modify the cluster configuration after the cluster is deployed, y ## Syntax -```sh +```shell tiup cluster edit-config [flags] ``` diff --git a/tiup/tiup-component-cluster-enable.md b/tiup/tiup-component-cluster-enable.md new file mode 100644 index 0000000000000..7431182c4a440 --- /dev/null +++ b/tiup/tiup-component-cluster-enable.md @@ -0,0 +1,51 @@ +--- +title: tiup cluster enable +--- + +# tiup cluster enable + +The `tiup cluster enable` command is used to set the self-enabling of the cluster service after a machine restarts. To enable the self-enabling of the service, this command executes `systemctl enable ` at the specified node. + +> **Note:** +> +> When all clusters are shut down and restarted, the order of service startup is determined by the node's operating system startup order. When the restart order is incorrect, in some cases, the restarted cluster still cannot provide services. For example, if TiKV is started first but PD is not started, systemd gives up if TiKV is restarted multiple times while PD is not found). + +## Syntax + +```shell +tiup cluster enable [flags] +``` + +``: the cluster whose auto-enabling service is to be enabled. + +## Options + +### -N, --node + +- Specifies the nodes whose auto-enabling service is to be enabled. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. +- Data type: `strings` +- Default: `[]`, which means no node is selected. + +> **Note:** +> +> If the `-R, --role` option is specified at the same time, the auto-enabling of services in their intersection is enabled. + +### -R, --role + +- Specifies the roles whose auto-enabling service is to be enabled. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. +- Data type: `strings` +- Default: `[]`, which means no role is selected. + +> **Note:** +> +> If the `-N, --node` option is specified at the same time, the auto-enabling of services in their intersection is enabled. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The execution log of the tiup-cluster. diff --git a/tiup/tiup-component-cluster-help.md b/tiup/tiup-component-cluster-help.md new file mode 100644 index 0000000000000..1b11b35b01b47 --- /dev/null +++ b/tiup/tiup-component-cluster-help.md @@ -0,0 +1,25 @@ +--- +title: tiup cluster help +--- + +# tiup cluster help + +tiup-cluster provides a wealth of help information for users in the command line interface. You can obtain it via the `help` command or the `--help` option. `tiup cluster help ` is basically equivalent to `tiup cluster --help`. + +## Syntax + +```shell +tiup cluster help [command] [flags] +``` + +`[command]` is used to specify the help information of which command that users need to view. If it is not specified, the help information of tiup-cluster is viewed. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The help information of the `[command]` or tiup-cluster. \ No newline at end of file diff --git a/tiup/tiup-component-cluster-import.md b/tiup/tiup-component-cluster-import.md new file mode 100644 index 0000000000000..e49c318cb0a46 --- /dev/null +++ b/tiup/tiup-component-cluster-import.md @@ -0,0 +1,67 @@ +--- +title: tiup cluster import +--- + +# tiup cluster import + +Before TiDB v4.0, TiDB clusters were mainly deployed using TiDB Ansible. For TiDB v4.0 and later releases, TiUP Cluster provides the `import` command to transfer the clusters to the tiup-cluster component for management. + +> **Note:** +> +> + After importing the TiDB Ansible configuration to TiUP for management, **DO NOT** use TiDB Ansible for cluster operations anymore. Otherwise, conflicts might be caused due to inconsistent meta information. +> + If the clusters deployed using TiDB Ansible are in any of the following situations, do not use the `import` command. +> + Clusters with TLS encryption enabled +> + Pure KV clusters (clusters without TiDB instances) +> + Clusters with Kafka enabled +> + Clusters with Spark enabled +> + Clusters with TiDB Lightning/TiKV Importer enabled +> + Clusters still using the old `push` mode to collect monitoring metrics (if you keep the default mode `pull` unchanged, using the `import` command is supported) +> + Clusters in which the non-default ports (the ports configured in the `group_vars` directory are compatible) are separately configured in the `inventory.ini` configuration file using `node_exporter_port` / `blackbox_exporter_port` + +## Syntax + +```shell +tiup cluster import [flags] +``` + +## Options + +### -d, --dir + +- Specifies the directory where TiDB Ansible is located. +- Data type: `STRING` +- The option is enabled by default with the current directory (the default value) passed in. + +### --ansible-config + +- Specifies the path of the Ansible configuration file. +- Data type: `STRING` +- The option is enabled by default with `. /ansible.cfg` (the default value) passed in. + +### --inventory + +- Specifies the name of the Ansible inventory file. +- Data type: `STRING` +- The option is enabled by default with `inventory.ini` (the default value) passed in. + +### --no-backup + +- Controls whether to disable the backup of files in the directory where TiDB Ansible is located. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. After a successful import, everything in the directory specified by the `-dir` option is backed up to the `${TIUP_HOME}/.tiup/storage/cluster/clusters/{cluster-name}/ansible-backup` directory. If there are multiple inventory files (when multiple clusters are deployed) in this directory, it is recommended to enable this option. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --rename + +- Renames the imported cluster. +- Data type: `STRING` +- Default: NULL. If this option is not specified in the command, the cluster_name specified in inventory is used as the cluster name. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +Shows the logs of the import process. diff --git a/tiup/tiup-component-cluster-patch.md b/tiup/tiup-component-cluster-patch.md new file mode 100644 index 0000000000000..5b329c64fbe24 --- /dev/null +++ b/tiup/tiup-component-cluster-patch.md @@ -0,0 +1,89 @@ +--- +title: tiup cluster patch +--- + +# tiup cluster patch + +If you need to dynamically replace the binaries of a service while the cluster is running (namely, keep the cluster available during the replacement process), you can use the `tiup cluster patch` command. After the command is executed, TiUP does the following things: + +- Uploads the binary package for replacement to the target machine. +- If the target service is a storage service such as TiKV, TiFlash, or TiDB Binlog, TiUP first takes the related nodes offline via the API. +- Stops the target service. +- Unpacks the binary package and replace the service. +- Starts the target service. + +## Syntax + +```shell +tiup cluster patch [flags] +``` + +- ``: The name of the cluster to be operated. +- ``: The path to the binary package used for replacement. + +### Preparation + +You need to pack the binary package required for this command in advance according to the following steps: + +- Determine the name `${component}` of the component to be replaced (tidb, tikv, pd...), the `${version}` of the component (v4.0.0, v4.0.1 ...), and the operating system `${os}` (`linux`) and platform `${arch}` on which the component runs. +- Download the current component package using the command `wget https://tiup-mirrors.pingcap.com/${component}-${version}-${os}-${arch}.tar.gz -O /tmp/${component}-${version}-${os}-${arch}.tar.gz`. +- Run `mkdir -p /tmp/package && cd /tmp/package` to create a temporary directory to pack files. +- Run `tar xf /tmp/${component}-${version}-${os}-${arch}.tar.gz` to unpack the original binary package. +- Run `find .` to view the file structure in the temporary package directory. +- Copy the binary files or configuration files to the corresponding locations in the temporary directory. +- Run `tar czf /tmp/${component}-hotfix-${os}-${arch}.tar.gz *` to pack the files in the temporary directory. +- Finally, you can use `/tmp/${component}-hotfix-${os}-${arch}.tar.gz` as the `` in the `tiup cluster patch` command. + +## Options + +### --overwrite + +- After you patch a certain component (such as TiDB or TiKV), when the tiup cluster scales out the component, TiUP uses the original component version by default. To use the version that you patch when the cluster scales out in the future, you need to specified the option `--overwrite` in the command. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### --transfer-timeout + +- When restarting the PD or TiKV service, TiKV/PD first transfers the leader of the node to be restarted to another node. Because the transfer process takes some time, you can use the option `--transfer-timeout` to set the maximum waiting time (in seconds). After the timeout, TiUP directly restarts the service. +- Data type: `UINT` +- If this option is not specified, TiUP directly restarts the service after waiting for `300` seconds. + +> **Note:** +> +> If TiUP directly restarts the service after the timeout, the service performance might jitter. + +### -N, --node + +- Specifies nodes to be replaced. The value of this option is a comma-separated list of node IDs. You can get the node ID from the first column of the cluster status table returned by the `tiup cluster display` command. +- Data type: `STRINGS` +- If this option is not specified, TiUP does not select any nodes to replace by default. + +> **Note:** +> +> If the option `-R, --role` is specified at the same time, TiUP then replaces service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -R, --role + +- Specifies the roles to be replaced. The value of this option is a comma-separated list of the roles of the nodes. You can get the role deployed on a node from the second column of the cluster status table returned by the `tiup cluster display` command. +- Data type: `STRINGS` +- If this option is not specified, TiUP does not select any roles to replace by default. + +> **Note:** +> +> If the option `-N, --node` is specified at the same time, TiUP then replaces service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +## --offline + +- Declares that the current cluster is not running. When the option is specified, TiUP does not migrate the service leader to another node or restart the service, but only replaces the binary files of cluster components. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The execution log of the tiup-cluster. diff --git a/tiup/tiup-component-cluster-prune.md b/tiup/tiup-component-cluster-prune.md new file mode 100644 index 0000000000000..cebbb93cb3b2d --- /dev/null +++ b/tiup/tiup-component-cluster-prune.md @@ -0,0 +1,25 @@ +--- +title: tiup cluster prune +--- + +# tiup cluster prune + +When [scaling in the cluster], for some components, TiUP does not immediately stop their services or delete their data. You need to wait for the data scheduling to complete and then manually execute the `tiup cluster prune` command to clean up. + +## Syntax + +```shell +tiup cluster prune [flags] +``` + +## Option + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of the cleanup process. diff --git a/tiup/tiup-component-cluster-reload.md b/tiup/tiup-component-cluster-reload.md new file mode 100644 index 0000000000000..5cacc37185565 --- /dev/null +++ b/tiup/tiup-component-cluster-reload.md @@ -0,0 +1,83 @@ +--- +title: tiup cluster reload +--- + +# tiup cluster reload + +After [modifying the cluster configuration](/tiup/tiup-component-cluster-edit-config.md), the cluster needs to be reloaded using the `tiup cluster reload` command for the configuration to take effect. This command publishes the configuration of the control machine to the remote machine where the service is running, and follows the upgrade process to restart the services in order according to the upgrade process. The cluster is still available during the restart process. + +## Syntax + +```shell +tiup cluster reload [flags] +``` + +``: the cluster name to operate on. + +## Options + +### --force + +- Ignores errors in the reloading process and forces reload. +- Data type: `BOOLEAN` +- Default: false + +### --transfer-timeout + +- When restarting PD or TiKV, the leader of the restarted node is migrated to other nodes first, and the migration process takes some time. You can set the maximum wait time (in seconds) by setting `-transfer-timeout`. After the timeout, the service can be restarted directly without waiting. +- Data type: `uint` +- Default: 300 + +> **Note:** +> +> In the case of skipping the waiting and restarting directly, the service performance might jitter. + +### --ignore-config-check + +- After the binary files of components are deployed, the configurations of TiDB, TiKV, and PD components are checked using ` --config-check `. `` is the path of the deployed binary file. `` is the configuration file generated based on the user configuration. If you want to skip this check, you can use this option. +- Data type: `BOOLEAN` +- Default: false + +### -N, --node + +- Specifies the nodes to be restarted. If not specified, all nodes are restarted. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. +- Data type: `strings` +- Default: `[]`, which means all nodes are selected. + +> **Note:** +> +> + If the `-R, --role` option is specified at the same time, then the service status in their intersection is restarted. +> + If the `--skip-restart` option is specified, the `-N, --node` option is invalid. + +### -R, --role + +- Specifies the roles to be restarted. If not specified, all roles are restarted. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status](/tiup/tiup-component-cluster-display.md) table. +- Data type: `strings` +- Default: `[]`, which means all roles are selected. + +> **Note:** +> +> 1. If the `-N, --node` option is specified at the same time, the services in their intersection is restarted. +> 2. If the `--skip-restart` option is specified, the `-R, --role` option is invalid. + +### --skip-restart + +The `tiup cluster reload` command performs two operations: + +- Refreshes all node configurations +- Restarts the specified node + +After you specify the `--skip-restart` option, it only refreshes the configuration without restarting any nodes, so that the refreshed configuration is not applied and does not take effect until the next restart of the corresponding service. + +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The execution log of the tiup-cluster. diff --git a/tiup/tiup-component-cluster-rename.md b/tiup/tiup-component-cluster-rename.md new file mode 100644 index 0000000000000..6042bbced0c00 --- /dev/null +++ b/tiup/tiup-component-cluster-rename.md @@ -0,0 +1,35 @@ +--- +title: tiup cluster rename +--- + +# tiup cluster rename + +The cluster name is specified when [the cluster is deployed](/tiup/tiup-component-cluster-deploy.md). If you want to change the cluster name after the cluster is deployed, you can use the command `tiup cluster rename`. + +> **Note:** +> +> If the `dashboard_dir` field of `grafana_servers` is configured for the TiUP cluster, after you execute the command `tiup cluster rename` to rename the cluster, the following additional steps are required: +> +> + For the `*.json` files in the local dashboards directory, update the `datasource` field of each file to the new cluster name, because the value of `datasource` must be the name of the cluster. +> + Execute the command `tiup cluster reload -R grafana`. + +## Syntax + +```shell +tiup cluster rename [flags] +``` + +- ``: The old cluster name. +- ``: The new cluster name. + +## Options + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The execution log of the tiup-cluster. diff --git a/tiup/tiup-component-cluster-restart.md b/tiup/tiup-component-cluster-restart.md new file mode 100644 index 0000000000000..0da657da69415 --- /dev/null +++ b/tiup/tiup-component-cluster-restart.md @@ -0,0 +1,51 @@ +--- +title: tiup cluster restart +--- + +# tiup cluster restart + +The command `tiup cluster restart` is used to restart all or some of the services of the specified cluster. + +> **Note:** +> +> During the restart process, the related services are unavailable for a period of time. + +## Syntax + +```shell +tiup cluster restart [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list](/tiup/tiup-component-cluster-list.md) command. + +## Options + +### -N, --node + +- Specifies the nodes to be restarted. The value of this option is a comma-separated list of node IDs. You can get the node IDs from the first column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- If this option is not specified, TiUP restarts all nodes by default. + +> **Note:** +> +> If the option `-R, --role` is specified at the same time, TiUP restarts service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -R, --role + +- Specified the roles of nodes to be restarted. The value of this option is a comma-separated list of the roles of the nodes. You can get the roles of the nodes from the second column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- If this option is not specified, TiUP restarts nodes of all roles by default. + +> **Note:** +> +> If the option `-N, --node` is specified at the same time, TiUP restarts service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The log of the service restart process. diff --git a/tiup/tiup-component-cluster-scale-in.md b/tiup/tiup-component-cluster-scale-in.md new file mode 100644 index 0000000000000..aaa96dfa1934a --- /dev/null +++ b/tiup/tiup-component-cluster-scale-in.md @@ -0,0 +1,70 @@ +--- +title: tiup cluster scale-in +--- + +# tiup cluster scale-in + +The `tiup cluster scale-in` command is used to scale in the cluster, which takes the services of the specified nodes offline, removes the specified nodes from the cluster, and deletes the remaining files from those nodes. + +Because the TiKV, TiFlash, and TiDB Binlog components are taken offline asynchronously (which requires TiUP to remove the node through API first) and the stopping process takes a long time (which requires TiUP to continuously check whether the node is successfully taken offline), the TiKV, TiFlash, and TiDB Binlog components are handled particularly as follows: + +- For TiKV, TiFlash and, TiDB Binlog components: + + 1. TiUP Cluster takes the node offline through API and directly exits without waiting for the process to be completed. + 2. To check the status of the nodes being scaled in, you need to execute the `tiup cluster display` command and wait for the status to become `Tombstone`. + 3. To clean up the nodes in the `Tombstone` status, you need to execute the `tiup cluster prune` command. The `tiup cluster prune` command performs the following operations: + + - Stops the services of the nodes that have been taken offline. + - Cleans up the data files of the nodes that have been taken offline. + - Updates the cluster topology and removes the nodes that have been taken offline. + +For other components: + +- When taking the PD components offline, TiUP Cluster quickly deletes the specified nodes from the cluster through API, stops the service of the specified PD nodes, and then deletes the related data files from the nodes. +- When taking other components down, TiUP Cluster directly stops the node services and deletes the related data files from the specified nodes. + +## Syntax + +```shell +tiup cluster scale-in [flags] +``` + +`` is the name of the cluster to scale in. If you forget the cluster name, you can check it using the [`tiup cluster list`](/tiup/tiup-component-cluster-list.md) command. + +## Options + +### -N, --node + +- Specifies the nodes to take down. Multiple nodes are separated by commas. +- Data type: `STRING` +- There is no default value. This option is mandatory and the value must be not null. + +### --force + +- Controls whether to forcibly remove the specified nodes from the cluster. Sometimes, the host of the node to take offline might be down, which makes it impossible to connect to the node via SSH for operations, so you can forcibly remove the node from the cluster using the `-force` option. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +> **Note:** +> +> Because the forced removal of a TiKV node does not wait for data to be scheduled, removing more than one serving TiKV node is at the risk of data loss. + +### --transfer-timeout + +- When a PD or TiKV node is to be removed, the Region leader on the node will be transferred to another node first. Because the transferring process takes some time, you can set the maximum waiting time (in seconds) by configuring `--transfer-timeout`. After the timeout, the `tiup cluster scale-in` command skips waiting and starts the scaling-in directly. +- Data type: `UINT` +- The option is enabled by default with `300` seconds (the default value) passed in. + +> **Note:** +> +> If a PD or TiKV node is taken offline directly without waiting for the leader transfer to be completed, the service performance might jitter. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +Shows the logs of the scaling-in process. diff --git a/tiup/tiup-component-cluster-scale-out.md b/tiup/tiup-component-cluster-scale-out.md new file mode 100644 index 0000000000000..e297b75ad4395 --- /dev/null +++ b/tiup/tiup-component-cluster-scale-out.md @@ -0,0 +1,63 @@ +--- +title: tiup cluster scale-out +--- + +# tiup cluster scale-out + +The `tiup cluster scale-out` command is used for scaling out the cluster. The internal logic of scaling out the cluster is similar to the cluster deployment. The tiup-cluster component first establishes an SSH connection to the new node, creates the necessary directories on the target node, then performs the deployment and starts the service. + +When PD is scaled out, new PD nodes are added to the cluster by the join operation, and the configuration of the services associated with PD is updated; other services are directly started and added to the cluster. + +## Syntax + +```shell +tiup cluster scale-out [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the `cluster list`command. + +``: the prepared topology file. This topology file should only contain the new nodes that are to be added to the current cluster. + +## Options + +### -u, --user + +- Specifies the user name used to connect to the target machine. This user must have the secret-free sudo root permission on the target machine. +- Data type: `STRING` +- Default: the current user who executes the command. + +### -i, --identity_file + +- Specifies the key file used to connect to the target machine. +- Data type: `STRING` +- If this option is not specified in the command, the `~/.ssh/id_rsa` file is used to connect to the target machine by default. + +### -p, --password + +- Specifies the password used to connect to the target machine. Do not use this option and `-i/--identity_file` at the same time. +- Data type: `BOOLEAN` +- Default: false + +### --no-labels + +- This option is used to skip the label check. +- When two or more TiKV nodes are deployed on the same physical machine, a risk exists: PD does not know the cluster topology, so it might schedule multiple replicas of a Region to different TiKV nodes on one physical machine, which makes this physical machine a single point of failure. To avoid this risk, you can use labels to tell PD not to schedule the same Region to the same machine. See [Schedule Replicas by Topology Labels](/schedule-replicas-by-topology-labels.md) for label configuration. +- For the test environment, this risk might not matter, and you can use `--no-labels` to skip the check. +- Data type: `BOOLEAN` +- Default: false + +### --skip-create-user + +- During the cluster deployment, tiup-cluster checks whether the specified user name in the topology file exists or not. If not, it creates one. To skip this check, you can use the `--skip-create-user` option. +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of scaling out. diff --git a/tiup/tiup-component-cluster-start.md b/tiup/tiup-component-cluster-start.md new file mode 100644 index 0000000000000..de41844f773f9 --- /dev/null +++ b/tiup/tiup-component-cluster-start.md @@ -0,0 +1,47 @@ +--- +title: tiup cluster start +--- + +# tiup cluster start + +The `tiup cluster start` command is used to start all services or some services of the specified cluster. + +## Syntax + +```shell +tiup cluster start [flags] +``` + +`` is the name of the cluster to operate on. If you forget the cluster name, check it using the [cluster list](/tiup/tiup-component-cluster-list.md) command. + +## Options + +### -N, --node + +- Specifies the nodes to be started. The value of this option is a comma-separated list of node IDs. You can get the node IDs from the first column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- Default: `[]`. If this option is not specified in the command, all nodes are started. + +> **Note:** +> +> If the `-R, --role` option is specified at the same time, only the service nodes that match both the specifications of `-N, --node` and `-R, --role` are restarted. + +### -R, --role + +- Specified the roles of nodes to be started. The value of this option is a comma-separated list of the roles of the nodes. You can get the roles of the nodes from the second column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- Default: `[]`. If this option is not specified in the command, all roles are started. + +> **Note:** +> +> If the `-N, --node` option is specified at the same time, only the service nodes that match both the specifications of `-N, --node` and `-R, --role` are restarted. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +The log of starting the service. diff --git a/tiup/tiup-component-cluster-stop.md b/tiup/tiup-component-cluster-stop.md new file mode 100644 index 0000000000000..a7bbe4510114a --- /dev/null +++ b/tiup/tiup-component-cluster-stop.md @@ -0,0 +1,51 @@ +--- +title: tiup cluster stop +--- + +# tiup cluster stop + +The `tiup cluster stop` command is used to stop all services or some services of the specified cluster. + +> **Note:** +> +> If the core services of a cluster are stopped, the cluster cannot provide services anymore. + +## Syntax + +```shell +tiup cluster stop [flags] +``` + +`` is the name of the cluster to operate on. If you forget the cluster name, check it with the [cluster list](/tiup/tiup-component-cluster-list.md) command. + +## Options + +### -N, --node + +- Specifies the nodes to be stoped. The value of this option is a comma-separated list of node IDs. You can get the node IDs from the first column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- Default: []. If this option is not specified in the command, the command stops all the nodes. + +> **Note:** +> +> If the `-R, --role` option is specified at the same time, only the service nodes that match both the specifications of `-N, --node` and `-R, --role` are restarted. + +### -R, --role + +- Specifies the roles of nodes to be stoped. The value of this option is a comma-separated list of the roles of the nodes. You can get the roles of the nodes from the second column of the [cluster status table](/tiup/tiup-component-cluster-display.md) returned by the `tiup cluster display` command. +- Data type: `STRING` +- Default: []. If this option is not specified in the command, the command stops all the roles. + +> **Note:** +> +> If the `-N, --node` option is specified at the same time, only the service nodes that match both the specifications of `-N, --node` and `-R, --role` are restarted. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +The log of stopping the service. diff --git a/tiup/tiup-component-cluster-upgrade.md b/tiup/tiup-component-cluster-upgrade.md new file mode 100644 index 0000000000000..2108f62be22a4 --- /dev/null +++ b/tiup/tiup-component-cluster-upgrade.md @@ -0,0 +1,54 @@ +--- +title: tiup cluster upgrade +--- + +# tiup cluster upgrade + +The `tiup cluster upgrade` command is used to upgrade the specified cluster to a specific version. + +## Syntax + +```shell +tiup cluster upgrade [flags] +``` + +- ``: the cluster name to operate on. If you forget the cluster name, you can check it with the [cluster list](/tiup/tiup-component-cluster-list.md) command. +- ``: the target version to upgrade to. Currently, it is only allowed to upgrade to a version higher than the current cluster, that is, no downgrade is allowed. It is also not allowed to upgrade to the nightly version. + +## Options + +### --force + +- To upgrade the cluster, you need to ensure that the cluster is currently started. In some cases, you might want to upgrade when the cluster is not started. At this time, you can use `--force` to ignore the error during the upgrade, forcibly replace the binary file and start the cluster. +- Data type: `BOOLEAN` +- Default: false + +> **Note:** +> +> Forcing an upgrade of the cluster that is providing services might result in service unavailability. Unstarted clusters are started automatically after a successful upgrade. + +### --transfer-timeout + +- When upgrading PD or TiKV, the leader of the upgraded node is migrated to other nodes first. The migration process takes some time, and you can set the maximum wait time (in seconds) by the `-transfer-timeout` option. After the timeout, the wait is skipped and the service is upgraded directly. +- Data type: `uint` +- Default: 300 + +> **Note:** +> +> If the wait is skipped and the service is upgraded directly, the service performance might jitter. + +### --ignore-config-check + +- After the binary is updated, a configuration check is performed on the TiDB, TiKV and PD components using ` --config-check `. `` is the path to the newly deployed binary and `` is the configuration file generated based on the user configuration. To skip this check, you can use the `--ignore-config-check` option. +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of the upgrading progress. diff --git a/tiup/tiup-component-cluster.md b/tiup/tiup-component-cluster.md new file mode 100644 index 0000000000000..bd46f9aa8eed6 --- /dev/null +++ b/tiup/tiup-component-cluster.md @@ -0,0 +1,87 @@ +--- +title: TiUP Cluster +--- + +# TiUP Cluster + +TiUP Cluster is a cluster management component of TiUP written in Golang. You can use the TiUP Cluster component to perform daily operations and maintenance, including deploying, starting, shutting down, destroying, elastic scaling, upgrading TiDB clusters, and managing TiDB cluster parameters. + +## Syntax + +```shell +tiup cluster [command] [flags] +``` + +`[command]` is the name of the command. For the supported commands, refer to the [command list](#command-list) below. + +## Options + +### --ssh + +- Specifies the SSH client to connect to the remote end (the machine where the TiDB service is deployed) for the command execution. +- Data type: `STRING` +- Supported values: + + - `builtin`: uses the easyssh client built in tiup-cluster as the SSH client. + - `system`: uses the default SSH client of the current operating system. + - `none`: The SSH client is not used. The deployment is only for the current machine. + +- If this option is not specified in the command, `builtin` is used as the default value. + +### --ssh-timeout + +- Specifies the SSH connection timeout in seconds. +- Data type: `UINT` +- If this option is not specified in the command, the default timeout is `5` seconds. + +### --wait-timeout + +- Specifies the maximum waiting time (in seconds) for each step in the operation process. The operation process consists of many steps, such as specifying systemctl to start or stop services, and waiting for ports to be online or offline. Each step may take several seconds. If the execution time of a step exceeds the specified timeout, the step exits with an error. +- Data type: `UINT` +- If this option is not specified in the command, the maximum waiting time for each step is `120` seconds. + +### -y, --yes + +- Skips the secondary confirmation of all risky operations. Using this option is not recommended, unless you are using a script to call TiUP. +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -v, --version + +- Prints the current version of TiUP Cluster. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -h, --help + +- Prints the help information of the related commands. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Command list + +- import: imports a cluster deployed by Ansible + +- check: checks a cluster before and after the deployment + +- [deploy](/tiup/tiup-component-cluster-deploy.md): deploys a cluster based on a specified topology +- [list](/tiup/tiup-component-cluster-list.md): queries the list of deployed clusters +- [display](/tiup/tiup-component-cluster-display.md): displays the status of a specified cluster +- [start](/tiup/tiup-component-cluster-start.md): starts a specified cluster +- [stop](/tiup/tiup-component-cluster-stop.md): stops a specified cluster +- restart: restarts a specified cluster + +- scale-in: scales in a specified cluster + +- [scale-out](/tiup/tiup-component-cluster-scale-out.md): scales out a specified cluster +- [upgrade](/tiup/tiup-component-cluster-upgrade.md): upgrades a specified cluster +- [prune](/tiup/tiup-component-cluster-prune.md): cleans up the instances in the Tombstone status for a specified cluster +- [edit-config](/tiup/tiup-component-cluster-edit-config.md): modifies the configuration of a specified cluster +- [reload](/tiup/tiup-component-cluster-reload.md): reloads the configuration of a specified cluster +- [patch](/tiup/tiup-component-cluster-patch.md): replaces a service in a deployed cluster + +- [clean](/tiup/tiup-component-cluster-clean.md): deletes data from the specified cluster +- [destroy](/tiup/tiup-component-cluster-destroy.md): destroys a specified cluster +- [audit](/tiup/tiup-component-cluster-audit.md): queries the operation audit log of a specified cluster +- [enable](/tiup/tiup-component-cluster-enable.md): enables a specified cluster or service to start on boot +- [disable](/tiup/tiup-component-cluster-disable.md): disables a specified cluster or service to start on boot +- [help](/tiup/tiup-component-cluster-help.md): prints the help information diff --git a/tiup/tiup-component-dm-audit.md b/tiup/tiup-component-dm-audit.md new file mode 100644 index 0000000000000..43d8badea3179 --- /dev/null +++ b/tiup/tiup-component-dm-audit.md @@ -0,0 +1,32 @@ +--- +title: tiup dm audit +--- + +# tiup dm audit + +The `tiup dm audit` command is used to view historical commands executed on all clusters and the execution log of each command. + +## Syntax + +```shell +tiup dm audit [audit-id] [flags] +``` + +- If you do not fill in the `[audit-id]`, the table of operation records is output in reverse chronological order. The first column is the `audit-id`. +- If you fill in the `[audit-id]`, the execution log of the specified `audit-id` is checked. + +## Option + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +- If `[audit-id]` is specified, the corresponding execution log is output. +- If `[audit-id]` is not specified, a table with the following fields is output: + - ID: the `audit-id` corresponding to this record + - Time: the execution time of the command corresponding to the record + - Command: the command corresponding to the record \ No newline at end of file diff --git a/tiup/tiup-component-dm-deploy.md b/tiup/tiup-component-dm-deploy.md new file mode 100644 index 0000000000000..0d21a7fdb4f6d --- /dev/null +++ b/tiup/tiup-component-dm-deploy.md @@ -0,0 +1,47 @@ +--- +title: tiup dm deploy +--- + +# tiup dm deploy + +The `tiup dm deploy` command is used to deploy a new cluster. + +## Syntax + +```shell +tiup dm deploy [flags] +``` + +- ``: the name of the new cluster, which cannot be the same as the existing cluster names. +- ``: the version number of the DM cluster to be deployed, such as `v2.0.0`. +- ``: the prepared [topology file]. + +## Options + +### -u, --user + +- Specifies the user name used to connect to the target machine. This user must have the secret-free sudo root permission on the target machine. +- Data type: `STRING` +- Default: the current user who executes the command. + +### -i, --identity_file + +- Specifies the key file used to connect to the target machine. +- Data type: `STRING` +- Default: `~/.ssh/id_rsa` + +### -p, --password + +- Specifies the password used to connect to the target machine. Do not use this option and `-i/--identity_file` at the same time. +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The deployment log. diff --git a/tiup/tiup-component-dm-destroy.md b/tiup/tiup-component-dm-destroy.md new file mode 100644 index 0000000000000..3cbfd1645a2e0 --- /dev/null +++ b/tiup/tiup-component-dm-destroy.md @@ -0,0 +1,31 @@ +--- +title: tiup dm destroy +--- + +# tiup dm destroy + +After an application goes offline, if you want to release the machines occupied by the cluster for use by other applications, you need to clean up the data on the cluster and the deployed binary files. To destroy the cluster, the `tiup dm destroy` command performs the following operations: + +- Stops the cluster. +- For each service, delete its log directory, deployment directory, and data directory. +- If the parent directory of the data directory or deployment directory of each service is created by `tiup-dm`, also delete the parent directory. + +## Syntax + +```shell +tiup dm destroy [flags] +``` + +``: the name of the cluster to be destroyed. + +## Option + +### -h, --help + +- Prints the help information. +- Data type: `Boolean` +- Default: false + +## Output + +The execution log of the tiup-dm. diff --git a/tiup/tiup-component-dm-display.md b/tiup/tiup-component-dm-display.md new file mode 100644 index 0000000000000..1945ae76a8476 --- /dev/null +++ b/tiup/tiup-component-dm-display.md @@ -0,0 +1,58 @@ +--- +title: tiup dm display +--- + +# tiup dm display + +If you want to check the operational status of each component in a DM cluster, it is inefficient to log in to each machine one by one. Therefore, tiup-dm provides the `tiup dm display` command to do this job efficiently. + +## Syntax + +```shell +tiup dm display [flags] +``` + +`` is the name of the cluster to be operated. If you forget the cluster name, you can use the `[tiup dm list](/tiup/tiup-component-dm-list.md)` command to check it. + +## Options + +### -N, --node + +- Specifies the IDs of the nodes to query, splitting by commas for multiple nodes. If you are not sure about the ID of a node, you can skip this option in the command to show the IDs and status of all nodes in the output. +- Data type: `STRING` +- This option is enabled by default with `[]` (which means all nodes) passed in. + +> **Note:** +> +> If `-R, --role` is also specified, only the services in the intersection of the specified nodes and roles is queried. + +### -R, --role + +- Specifies the roles to query, splitting by commas for multiple roles. If you are not sure about the role deployed on a node, you can skip this option in the command to show the roles and status of all nodes in the output. +- Data type: `STRING` +- This option is enabled by default with `[]` (which means all roles) passed in. + +> **Note:** +> +> If `-N, --node` is also specified, only the services in the intersection of the specified nodes and roles is queried. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +- Cluster name +- Cluster version +- SSH client type +- A table containing the following fields: + - `ID`: the node ID, consisting of IP:PORT. + - `Role`: the service role deployed on the node (for example, TiDB or TiKV). + - `Host`: the IP address of the machine corresponding to the node. + - `Ports`: the port number used by the service. + - `OS/Arch`: the operating system and machine architecture of the node. + - `Status`: the current status of the services on the node. + - `Data Dir`: the data directory of the service. `-` means that there is no data directory. + - `Deploy Dir`: the deployment directory of the service. diff --git a/tiup/tiup-component-dm-edit-config.md b/tiup/tiup-component-dm-edit-config.md new file mode 100644 index 0000000000000..a125478902a2e --- /dev/null +++ b/tiup/tiup-component-dm-edit-config.md @@ -0,0 +1,33 @@ +--- +title: tiup dm edit-config +--- + +# tiup dm edit-config + +If you need to modify the cluster service configuration after the cluster is deployed, you can use the `tiup dm edit-config` command that starts an editor for you to modify the [topology file]. of the specified cluster. This editor is specified in the `$EDITOR` environment variable by default. If the `$EDITOR` environment variable does not exist, the `vi` editor is used. + +> **Note:** +> +> + When you modify the configuration, you cannot add or delete machines. For how to add machines, see [Scale out a cluster](/tiup/tiup-component-dm-scale-out.md). For how to delete machines, see [Scale in a cluster](/tiup/tiup-component-dm-scale-in.md). +> + After you execute the `tiup dm edit-config` command, the configuration is modified only on the control machine. Then you need to execute the `tiup dm relaod` command to reload the configuration. + +## Syntax + +```shell +tiup dm edit-config [flags] +``` + +``: the cluster to operate on. + +## Option + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +- Normally, no output. +- If you have mistakenly modified the fields that cannot be modified, when you save the file, an error is reported, reminding you to edit the file again. For the fields that cannot be modified, see [the topology file]. diff --git a/tiup/tiup-component-dm-help.md b/tiup/tiup-component-dm-help.md new file mode 100644 index 0000000000000..da4867e0a40df --- /dev/null +++ b/tiup/tiup-component-dm-help.md @@ -0,0 +1,25 @@ +--- +title: tiup dm help +--- + +# tiup dm help + +tiup-dm command-line interface provides users with a wealth of help information. You can view it via the `help` command or the `--help` option. Basically, `tiup dm help ` is equivalent to `tiup dm --help`. + +## Syntax + +```shell +tiup dm help [command] [flags] +``` + +`[command]` is used to specify the help information of which command that users need to view. If it is not specified, the help information of `tiup-dm` is viewed. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The help information of `[command]` or `tiup-dm`. \ No newline at end of file diff --git a/tiup/tiup-component-dm-import.md b/tiup/tiup-component-dm-import.md new file mode 100644 index 0000000000000..fd5f023e10bf2 --- /dev/null +++ b/tiup/tiup-component-dm-import.md @@ -0,0 +1,60 @@ +--- +title: tiup dm import +--- + +# tiup dm import + +In DM v1.0, the cluster is basically deployed using TiDB Ansible. TiUP DM provides the `import` command to import v1.0 clusters and redeploy the clusters in DM v2.0. + +> **Note:** +> +> - The command does not support importing DM Portal components from DM v1.0 clusters. +> - Before importing the cluster, stop running the original cluster first. +> - For data migration tasks that need to be upgraded to v2.0, do not execute `stop-task` on these tasks. +> - The command only supports importing to DM v2.0.0-rc.2 and later versions. +> - The `import` command is used to import a DM v1.0 cluster to a new DM v2.0 cluster. If you need to import data migration tasks to an existing v2.0 cluster, refer to [Manually Upgrade TiDB Data Migration from v1.0.x to v2.0.x](https://docs.pingcap.com/tidb-data-migration/stable/manually-upgrade-dm-1.0-to-2.0) +> - The deployment directories of some components might be different from those in the original cluster. You can check it with the `display` command. +> - Before importing the cluster, run `tiup update --self && tiup update dm` to upgrade TiUP DM components to the latest version. +> - After the cluster is imported, there is only one DM-master node in the cluster. You can refer to [the `scale out` command](/tiup/tiup-component-dm-scale-out.md) to scale out the DM-master node. + +## Syntax + +```shell +tiup dm import [flags] +``` + +## Options + +### -v, --cluster-version + +- Specifies the version number for redeploying. You must use a version later than v2.0.0-rc.2 (including v2.0.0-rc.2). +- Data type: `STRING` +- This option is **required** to execute the command. + +### -d, --dir + +- Specifies the directory of TiDB Ansible. +- Data type: `STRING` +- If this option is not specified in the command, the current directory is the default directory. + +### --inventory + +- Specifies the name of the Ansible inventory file. +- Data type: `STRING` +- If this option is not specified in the command, the default file name is `"inventory.ini"`. + +### --rename + +- Renames the imported cluster. +- Data type: `STRING` +- If this option is not specified in the command, the default cluster name is the `cluster_name` specified in the inventory file. + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The log of the importing process. diff --git a/tiup/tiup-component-dm-list.md b/tiup/tiup-component-dm-list.md new file mode 100644 index 0000000000000..7339da8ef2927 --- /dev/null +++ b/tiup/tiup-component-dm-list.md @@ -0,0 +1,35 @@ +--- +title: tiup dm list +--- + +# tiup dm list + +`tiup-dm` supports deploying multiple clusters using the same control machine. You can use the `tiup dm list` command to check which clusters are deployed using the control machine by the currently logged-in user. + +> **Note:** +> +> By default, the data of the deployed clusters is stored in the `~/.tiup/storage/dm/clusters/` directory. The currently logged-in user cannot view the clusters deployed by other users on the same control machine. + +## Syntax + +```shell +tiup dm list [flags] +``` + +## Options + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +A table consisting of the following fields: + +- `Name`: the cluster name. +- `User`: the user who deployed the cluster. +- `Version`: the cluster version. +- `Path`: the path of the cluster deployment data on the control machine. +- `PrivateKey`: the path of the private key to the cluster. diff --git a/tiup/tiup-component-dm-patch.md b/tiup/tiup-component-dm-patch.md new file mode 100644 index 0000000000000..65d4641d0078b --- /dev/null +++ b/tiup/tiup-component-dm-patch.md @@ -0,0 +1,73 @@ +--- +title: tiup dm patch +--- + +# tiup dm patch + +If you need to dynamically replace the binaries of a service while the cluster is running (that is, to keep the cluster available during the replacement), you can use the `tiup dm patch` command. The command does the following: + +- Uploads the binary package for replacement to the target machine. +- Takes the related nodes offline using the API. +- Stops the target service. +- Unpacks the binary package and replaces the service. +- Starts the target service. + +## Syntax + +```shell +tiup dm patch [flags] +``` + +- ``: The name of the cluster to be operated. +- ``: The path to the binary package used for replacement. + +### Preparation + +You need to pack the binary package required for this command in advance according to the following steps: + +- Determine the name `${component}` of the component to be replaced (dm-master, dm-worker ...), the `${version}` of the component (v2.0.0, v2.0.1 ...), and the operating system `${os}` and platform `${arch}` on which the component runs. +- Download the current component package using the command `wget https://tiup-mirrors.pingcap.com/${component}-${version}-${os}-${arch}.tar.gz -O /tmp/${component}-${version}-${os}-${arch}.tar.gz`. +- Run `mkdir -p /tmp/package && cd /tmp/package` to create a temporary directory to pack files. +- Run `tar xf /tmp/${component}-${version}-${os}-${arch}.tar.gz` to unpack the original binary package. +- Run `find .` to view the file structure in the temporary package directory. +- Copy the binary files or configuration files to the corresponding locations in the temporary directory. +- Run `tar czf /tmp/${component}-hotfix-${os}-${arch}.tar.gz *` to pack the files in the temporary directory. +- Finally, you can use `/tmp/${component}-hotfix-${os}-${arch}.tar.gz` as the value of `` in the `tiup dm patch` command. + +## Options + +### --overwrite + +- After you patch a certain component (such as dm-worker), when the tiup-dm scales out the component, tiup-dm uses the original component version by default. To use the version that you patch when the cluster scales out in the future, you need to specify the option `--overwrite` in the command. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -N, --node + +- Specifies the nodes to be replaced. The value of this option is a comma-separated list of node IDs. You can get the node IDs from the first column of the cluster status table returned by the `[tiup dm display](/tiup/tiup-component-dm-display.md)` command. +- Data type: `STRING` +- If this option is not specified, TiUP selects all nodes to replace by default. + +> **Note:** +> +> If the option `-R, --role` is specified at the same time, TiUP then replaces service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -R, --role + +- Specifies the roles to be replaced. The value of this option is a comma-separated list of the roles of the nodes. You can get the roles of the nodes from the second column of the cluster status table returned by the `[tiup dm display](/tiup/tiup-component-dm-display.md)` command. +- Data type: `STRING` +- If this option is not specified, TiUP selects all roles to replace by default. + +> **Note:** +> +> If the option `-N, --node` is specified at the same time, TiUP then replaces service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The execution log of tiup-dm. diff --git a/tiup/tiup-component-dm-prune.md b/tiup/tiup-component-dm-prune.md new file mode 100644 index 0000000000000..12097c4a0079c --- /dev/null +++ b/tiup/tiup-component-dm-prune.md @@ -0,0 +1,25 @@ +--- +title: tiup dm prune +--- + +# tiup dm prune + +When you scale in the cluster(/tiup/tiup-component-dm-scale-in.md), a small amount of metadata in etcd is not cleaned up, which usually causes no problem. If you need to clean up the metadata, you can manually execute the `tiup dm prune` command. + +## Syntax + +```shell +tiup dm prune [flags] +``` + +## Option + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of the cleanup process. diff --git a/tiup/tiup-component-dm-reload.md b/tiup/tiup-component-dm-reload.md new file mode 100644 index 0000000000000..f0a24d8520ee6 --- /dev/null +++ b/tiup/tiup-component-dm-reload.md @@ -0,0 +1,61 @@ +--- +title: tiup dm reload +--- + +# tiup dm reload + +After [modifying the cluster configuration], the cluster needs to be reloaded using the `tiup dm reload` command for the configuration to take effect. This command publishes the configuration of the control machine to the remote machine where the service is running and restarts the service in order according to the upgrade process. The cluster remains available during the restart process. + +## Syntax + +```shell +tiup dm reload [flags] +``` + +``: the name of the cluster to operate on. + +## Options + +### -N, --node + +- Specifies the nodes to be restarted. If not specified, all nodes are restarted. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status]table. +- Data type: `strings` +- Default: `[]`, which means all nodes are selected. + +> **Note:** +> +> + If the `-R, --role` option is specified at the same time, then the service status in their intersection is restarted. +> + If the `--skip-restart` option is specified, the `-N, --node` option is invalid. + +### -R, --role + +- Specifies the roles to be restarted. If not specified, all roles are restarted. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status]table. +- Data type: `strings` +- Default: `[]`, which means all roles are selected. + +> **Note:** +> +> + If the `-N, --node` option is specified at the same time, the services in their intersection is restarted. +> + If the `--skip-restart` option is specified, the `-R, --role` option is invalid. + +### --skip-restart + +The `tiup dm reload` command performs two operations: + +- Refreshes all node configurations +- Restarts the specified node + +After you specify the `--skip-restart` option, it only refreshes the configuration without restarting any nodes, so that the refreshed configuration is not applied and does not take effect until the next restart of the corresponding service. + +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The execution log of the tiup-dm. \ No newline at end of file diff --git a/tiup/tiup-component-dm-restart.md b/tiup/tiup-component-dm-restart.md new file mode 100644 index 0000000000000..278dab2aef4e1 --- /dev/null +++ b/tiup/tiup-component-dm-restart.md @@ -0,0 +1,51 @@ +--- +title: tiup dm restart +--- + +# tiup dm restart + +The command `tiup dm restart` is used to restart all or some of the services of the specified cluster. + +> **Note:** +> +> During the restart process, the related services are unavailable for a period of time. + +## Syntax + +```shell +tiup dm restart [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list](/tiup/tiup-component-cluster-list.md) command. + +## Options + +### -N, --node + +- Specifies the nodes to be restarted. The value of this option is a comma-separated list of node IDs. You can get the node IDs from the first column of the cluster status table returned by the `[tiup dm display](/tiup/tiup-component-dm-display.md)` command. +- Data type: `STRING` +- If this option is not specified, TiUP restarts all nodes by default. + +> **Note:** +> +> If the option `-R, --role` is specified at the same time, TiUP restarts service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -R, --role + +- Specifies the roles of nodes to be restarted. The value of this option is a comma-separated list of the roles of the nodes. You can get the roles of the nodes from the second column of the cluster status table returned by the `[tiup dm display](/tiup/tiup-component-dm-display.md)` command. +- Data type: `STRING` +- If this option is not specified, TiUP restarts nodes of all roles by default. + +> **Note:** +> +> If the option `-N, --node` is specified at the same time, TiUP restarts service nodes that match both the requirements of `-N, --node` and `-R, --role`. + +### -h, --help + +- Prints help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Outputs + +The log of the service restart process. diff --git a/tiup/tiup-component-dm-scale-in.md b/tiup/tiup-component-dm-scale-in.md new file mode 100644 index 0000000000000..b52614b80fcb9 --- /dev/null +++ b/tiup/tiup-component-dm-scale-in.md @@ -0,0 +1,39 @@ +--- +title: tiup dm scale-in +--- + +# tiup dm scale-in + +The `tiup dm scale-in` command is used to scale in the cluster. Scaling in the cluster means taking the service offline, which eventually removes the specified node from the cluster and deletes the remaining related files. + +## Syntax + +```shell +tiup dm scale-in [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list] command. + +## Options + +### -N, --node + +- Specifies the nodes to be scaled in. If you need to scale in multiple nodes, split them by commas. +- Data type: `Strings` +- Default: no. This option is mandatory and the value must be not null. + +### --force + +- In some cases, some scale-in nodes in the cluster have been down, making it impossible to connect to the node through SSH for operation. At this time, you can use the `--force` option to remove these nodes from the cluster. +- Data type: `BOOLEAN` +- Default: false. If this option is not specified in the command, the specified nodes are not forcibly removed. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of scaling in. \ No newline at end of file diff --git a/tiup/tiup-component-dm-scale-out.md b/tiup/tiup-component-dm-scale-out.md new file mode 100644 index 0000000000000..64daca0e5d257 --- /dev/null +++ b/tiup/tiup-component-dm-scale-out.md @@ -0,0 +1,47 @@ +--- +title: tiup dm scale-out +--- + +# tiup dm scale-out + +The `tiup dm scale-out` command is used for scaling out the cluster. The internal logic of scaling out the cluster is similar to the cluster deployment. The `tiup-dm` components first establish an SSH connection to the new node, create the necessary directories on the target node, then perform the deployment and start the service. + +## Syntax + +```shell +tiup dm scale-out [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list]. + +``: the prepared [topology file]. This topology file should only contain the new nodes that are to be added to the current cluster. + +## Options + +### -u, --user + +- Specifies the user name used to connect to the target machine. This user must have the secret-free sudo root permission on the target machine. +- Data type: `STRING` +- Default: the current user who executes the command. + +### -i, --identity_file + +- Specifies the key file used to connect to the target machine. +- Data type: `STRING` +- If this option is not specified in the command, the `~/.ssh/id_rsa` file is used to connect to the target machine by default. + +### -p, --password + +- Specifies the password used to connect to the target machine. Do not use this option and `-i/--identity_file` at the same time. +- Data type: `BOOLEAN` +- Default: false + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of scaling out. diff --git a/tiup/tiup-component-dm-start.md b/tiup/tiup-component-dm-start.md new file mode 100644 index 0000000000000..551f61777a66d --- /dev/null +++ b/tiup/tiup-component-dm-start.md @@ -0,0 +1,47 @@ +--- +title: tiup dm start +--- + +# tiup dm start + +The `tiup dm start` command is used to start all or part of the services of the specified cluster. + +## Syntax + +```shell +tiup dm start [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list] command. + +## Options + +### -N, --node + +- Specifies the nodes to be started. If not specified, all nodes are started. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status]table. +- Data type: `strings` +- Default: `[]`. If this option is not specified in the command, all nodes are started. + +> **Note:** +> +> If the `-R, --role` option is specified at the same time, then the services in their intersection are started. + +### -R, --role + +- Specifies the roles to be started. If not specified, all roles are started. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status] table. +- Data type: `strings` +- Default: `[]`. If this option is not specified in the command, all roles are started. + +> **Note:** +> +> If the `-N, --node` option is specified at the same time, the services in their intersection are started. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of starting the service. \ No newline at end of file diff --git a/tiup/tiup-component-dm-stop.md b/tiup/tiup-component-dm-stop.md new file mode 100644 index 0000000000000..c8a73a788301f --- /dev/null +++ b/tiup/tiup-component-dm-stop.md @@ -0,0 +1,51 @@ +--- +title: tiup dm stop +--- + +# tiup dm stop + +The `tiup dm stop` command is used to stop all or part of the services of the specified cluster. + +> **Note:** +> +> The cluster cannot provide services after the core service is stopped. + +## Syntax + +```shell +tiup dm stop [flags] +``` + +``: the name of the cluster to operate on. If you forget the cluster name, you can check it with the [cluster list]command. + +## Options + +### -N, --node + +- Specifies the nodes to be stopped. If not specified, all nodes are stopped. The value of this option is a comma-separated list of node IDs. The node ID is the first column of the [cluster status] table. +- Data type: `strings` +- Default: `[]`, which means all nodes are selected. + +> **Note:** +> +> If the `-R, --role` option is specified at the same time, then the service status in their intersection is stopped. + +### -R, --role + +- Specifies the roles to be stopped. If not specified, all roles are stopped. The value of this option is a comma-separated list of node roles. The role is the second column of the [cluster status] table. +- Data type: `strings` +- Default: `[]`, which means all roles are selected. + +> **Note:** +> +> If the `-N, --node` option is specified at the same time, the services in their intersection is stopped. + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- Default: false + +## Output + +The log of stopping the service. \ No newline at end of file diff --git a/tiup/tiup-component-dm-upgrade.md b/tiup/tiup-component-dm-upgrade.md new file mode 100644 index 0000000000000..31bcb8aa2e90b --- /dev/null +++ b/tiup/tiup-component-dm-upgrade.md @@ -0,0 +1,28 @@ +--- +title: tiup dm upgrade +--- + +# tiup dm upgrade + +The `tiup dm upgrade` command is used to upgrade a specified cluster to a specific version. + +## Syntax + +```shell +tiup dm upgrade [flags] +``` + +- `` is the name of the cluster to be operated on. If you forget the cluster name, you can use the `[tiup dm list](/tiup/tiup-component-dm-list.md)` command to check it. +- `` is the target version to be upgraded to. Currently, only upgrading to a later version is allowed, and upgrading to an earlier version is not allowed, which means the downgrade is not allowed. Upgrading to a nightly version is not allowed either. + +## Options + +### -h, --help + +- Prints the help information. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## Output + +Log of the service upgrade process. diff --git a/tiup/tiup-component-dm.md b/tiup/tiup-component-dm.md new file mode 100644 index 0000000000000..c988ea80c9ec9 --- /dev/null +++ b/tiup/tiup-component-dm.md @@ -0,0 +1,79 @@ +--- +title: TiUP DM +--- + +# TiUP DM + +Similar to [TiUP Cluster](/tiup/tiup-component-cluster.md) which is used to manage TiDB clusters, TiUP DM is used to manage DM clusters. You can use the TiUP DM component to perform daily operations and maintenance tasks of DM clusters, including deploying, starting, stopping, destroying, elastic scaling, upgrading DM clusters, and managing the configuration parameters of DM clusters. + +## Syntax + +```shell +tiup dm [command] [flags] +``` + +`[command]` is used to pass the name of the command. See the [command list](#list-of-commands) for supported commands. + +## Options + +### --ssh + +- Specifies the SSH client to connect to the remote end (the machine where the TiDB service is deployed) for the command execution. +- Data type:`STRING` +- Support values: + + - `builtin`: Uses the built-in easyssh client of tiup-cluster as the SSH client. + - `system`: Uses the default SSH client of the current operating system. + - `none`: No SSH client is used. The deployment is only for the current machine. + +- If this option is not specified in the command, `builtin` is used as the default value. + +### --ssh-timeout + +- Specifies the SSH connection timeout in seconds. +- Data type: `UINT` +- If this option is not specified in the command, the default timeout is `5` seconds. + +### --wait-timeout + +- Specifies the maximum waiting time (in seconds) for each step in the operation process. The operation process consists of many steps, such as specifying systemctl to start or stop services, and waiting for ports to be online or offline. Each step may take several seconds. If the execution time of a step exceeds the specified timeout, the step exits with an error. +- Data type:`UINT` +- If this option is not specified in the command, the maximum waiting time for each steps is `120` seconds. + +### -y, --yes + +- Skips the secondary confirmation of all risky operations. It is not recommended to use this option unless you use a script to call TiUP. +- Data type:`BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -v, --version + +- Prints the current version of TiUP DM. +- Data type:`BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +### -h, --help + +- Prints help information about the specified command. +- Data type: `BOOLEAN` +- This option is disabled by default with the `false` value. To enable this option, add this option to the command, and either pass the `true` value or do not pass any value. + +## List of commands + +- [import](/tiup/tiup-component-dm-import.md): Imports a DM v1.0 cluster deployed by DM-Ansible. +- [deploy](/tiup/tiup-component-dm-deploy.md): Deploys a cluster based on a specified topology. +- [list](/tiup/tiup-component-dm-list.md): Queries the list of deployed clusters. +- [display](/tiup/tiup-component-dm-display.md): Displays the status of a specified cluster. +- [start](/tiup/tiup-component-dm-start.md): Starts a specified cluster. +- [stop](/tiup/tiup-component-dm-stop.md): Stops a specified cluster. +- [restart](/tiup/tiup-component-dm-restart.md): Restarts a specified cluster. +- [scale-in](/tiup/tiup-component-dm-scale-in.md): Scales in a specified cluster. +- [scale-out](/tiup/tiup-component-dm-scale-out.md): Scales out a specified cluster. +- [upgrade](/tiup/tiup-component-dm-upgrade.md): Upgrades a specified cluster. +- [prune](/tiup/tiup-component-dm-prune.md): Cleans up instances in the Tombstone status for a specified cluster. +- [edit-config](/tiup/tiup-component-dm-edit-config.md): Modifies the configuration of a specified cluster. +- [reload](/tiup/tiup-component-dm-reload.md): Reloads the configuration of a specified cluster. +- [patch](/tiup/tiup-component-dm-patch.md): Replaces a specified service in a deployed cluster. +- [destroy](/tiup/tiup-component-dm-destroy.md): Destroys a specified cluster. +- [audit](/tiup/tiup-component-dm-audit.md): Queries the operation audit log of a specified cluster. +- [help](/tiup/tiup-component-dm-help.md): Prints help information. diff --git a/tiup/tiup-dm-topology-reference.md b/tiup/tiup-dm-topology-reference.md new file mode 100644 index 0000000000000..a1e951a3aa232 --- /dev/null +++ b/tiup/tiup-dm-topology-reference.md @@ -0,0 +1,293 @@ +--- +title: Topology Configuration File for DM Cluster Deployment Using TiUP +--- + +# Topology Configuration File for DM Cluster Deployment Using TiUP + +To deploy or scale a TiDB Data Migration (DM) cluster, you need to provide a topology file ([sample](https://github.com/pingcap/tiup/blob/master/embed/templates/examples/dm/topology.example.yaml)) to describe the cluster topology. + +Similarly, to modify the cluster topology, you need to modify the topology file. The difference is that, after the cluster is deployed, you can only modify a part of the fields in the topology file. This document introduces each section of the topology file and each field in each section. + +## File structure + +A topology configuration file for DM cluster deployment using TiUP might contain the following sections: + +- [global](#global): the cluster's global configuration. Some of the configuration items use the default values of the cluster, and you can configure them separately in each instance. +- [server_configs](#server_configs): the components' global configuration. You can configure each component separately. If an instance has a configuration item with the same key, the instance's configuration item will take effect. +- [master_servers](#master_servers): the configuration of the DM-master instance. The configuration specifies the machines to which the master service of the DM component is deployed. +- [worker_servers](#worker_servers): the configuration of the DM-worker instance. The configuration specifies the machines to which the worker service of the DM component is deployed. +- [monitoring_servers](#monitoring_servers): specifies the machines to which the Prometheus instances are deployed. TiUP supports deploying multiple Prometheus instances but only the first instance is used. +- [grafana_servers](#grafana_servers): the configuration of the Grafana instances. The configuration specifies the machines to which the Grafana instances are deployed. +- [alertmanager_servers](#alertmanager_servers): the configuration of the Alertemanager instances. The configuration specifies the machines to which the Alertmanager instances are deployed. + +### `global` + +The `global` section corresponds to the cluster's global configuration and has the following fields: + +- `user`: the user to start the deployed cluster. The default value is "tidb". If the user specified in the `` field does not exist on the target machine, TiUP will automatically try to create the user. +- `group`: the user group to which a user belongs when the user is automatically created. The default value is the same as the `` field. If the specified group does not exist, it will be created automatically. +- `ssh_port`: the SSH port to connect to the target machine for operations. The default value is "22". +- `deploy_dir`: the deployment directory for each component. The default value is "deploy". The construction rules are as follows: + - If the absolute path `deploy_dir` is configured at the instance level, the actual deployment directory is the `deploy_dir` configured for the instance. + - For each instance, if you do not configure `deploy_dir`, the default value is the relative path `-`. + - If `global.deploy_dir` is set to an absolute path, the component is deployed to the `/` directory. + - If `global.deploy_dir` is set to a relative path, the component is deployed to the `/home///` directory. +- `data_dir`: the data directory. The default value is "data". The construction rules are as follows. + - If the absolute path `data_dir` is configured at the instance level, the actual data directory is the `data_dir` configured for the instance. + - For each instance, if `data_dir` is not configured, the default value is ``. + - If `data_dir` is set to a relative path, the component data is stored in `/`. For the construction rules of ``, see the construction rules of the `deploy_dir` field. +- `log_dir`: the data directory. The default value is "log". The construction rules are as follows. + - If the absolute path of `log_dir` is configured at the instance level, the actual log directory is the `log_dir` configured for the instance. + - For each instance, if `log_dir` is not configured by the user, the default value is ``. + - If `log_dir` is a relative path, the component logs will be stored in `/`. For the construction rules of ``, see the construction rules of the `deploy_dir` field. +- `os`: the operating system of the target machine. The field controls which operating system to adapt to for the components pushed to the target machine. The default value is "linux". +- `arch`: the CPU architecture of the target machine. The field controls which platform to adapt to for the binary packages pushed to the target machine. The supported values are "amd64" and "arm64". The default value is "amd64". +- `resource_control`: runtime resource control. All configurations in this field are written to the service file of systemd. There is no limit by default. The resources that can be controlled are as follows: + - `memory_limit`: limits the maximum memory at runtime. For example, "2G" means that the maximum memory of 2 GB can be used. + - `cpu_quota`: limits the maximum CPU usage at runtime. For example, "200%". + - `io_read_bandwidth_max`: limits the maximum I/O bandwidth for disk reads. For example, `"/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0:0 100M"`. + - `io_write_bandwidth_max`: limits the maximum I/O bandwidth for disk writes. For example, `"/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0:0 100M"`. + - `limit_core`: controls the size of core dump. + +A `global` configuration example: + +```yaml +global: + user: "tidb" + resource_control: + memory_limit: "2G" +``` + +In the example, the configuration specifies that the `tidb` user is used to start the cluster, and that each component is limited to a maximum of 2 GB of memory when it is running. + +### `server_configs` + +`server_configs` is used to configure services and to generate configuration files for each component. Similar to the `global` section, the configurations in the `server_configs` section can be overwritten by the configurations with the same keys in an instance. `server_configs` mainly contains the following fields: + +- `master`: the configuration related to the DM-master service. For all the supported configuration items, see [DM-master Configuration File](https://docs.pingcap.com/tidb-data-migration/stable/dm-master-configuration-file). +- `worker`: the configuration related to the DM-worker service, For all the supported configuration items, see [DM-worker Configuration File](https://docs.pingcap.com/tidb-data-migration/stable/dm-worker-configuration-file). + +A `server_configs` configuration example is as follows: + +```yaml +server_configs: + master: + log-level: info + rpc-timeout: "30s" + rpc-rate-limit: 10.0 + rpc-rate-burst: 40 + worker: + log-level: info +``` + +## `master_servers` + +`master_servers` specifies the machines to which the master node of the DM component is deployed. You can also specify the service configuration on each machine. `master_servers` is an array. Each array element contains the following fields: + +- `host`: specifies the machine to deploy to. The field value is an IP address and is mandatory. +- `ssh_port`: specifies the SSH port to connect to the target machine for operations. If the field is not specified, the `ssh_port` in the `global` section is used. +- `name`: specifies the name of the DM-master instance. The name must be unique for different instances. Otherwise, the cluster cannot be deployed. +- `port`: specifies the port on which DM-master provides services. The default value is "8261". +- `peer_port`: specifies the port for communication between DM-masters. The default value is "8291". +- `deploy_dir`: specifies the deployment directory. If the field is not specified, or specified as a relative directory, the deployment directory is generated according to the `deploy_dir` configuration in the `global` section. +- `data_dir`: specifies the data directory. If the field is not specified, or specified as a relative directory, the data directory is generated according to the `data_dir` configuration in the `global` section. +- `log_dir`: specifies the log directory. If the field is not specified, or specified as a relative directory, the log directory is generated according to the `log_dir` configuration in the `global` section. +- `numa_node`: allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is a string type. The field value is the ID of the NUMA node, such as "0,1". +- `config`: the configuration rules of this field are the same as that of `master` in the `server_configs` section. If `config` is specified, the configuration of `config` will be merged with the configuration of `master` in `server_configs` (if the two fields overlap, the configuration of this field takes effect), and then the configuration file is generated and distributed to the machine specified in the `host` field. +- `os`: the operating system of the machine specified in the `host` field. If the field is not specified, the default value is the `os` value configured in the `global` section. +- `arch`: the architecture of the machine specified in the `host` field. If the field is not specified, the default value is the `arch` value configured in the `global` section. +- `resource_control`: resource control on this service. If this field is specified, the configuration of this field will be merged with the configuration of `resource_control` in the `global` section (if the two fields overlap, the configuration of this field takes effect), and then the configuration file of systemd is generated and distributed to the machine specified in the `host` field. The configuration rules of this field are the same as that of `resource_control` in the `global` section. +- `v1_source_path`: when upgrading from v1.0.x, you can specify the directory where the configuration file of the V1 source is located in this field. + +In the `master_servers` section, the following fields cannot be modified after the deployment is completed: + +- `host` +- `name` +- `port` +- `peer_port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` +- `v1_source_path` + +A `master_servers` configuration example is as follows: + +```yaml +master_servers: + - host: 10.0.1.11 + name: master1 + ssh_port: 22 + port: 8261 + peer_port: 8291 + deploy_dir: "/dm-deploy/dm-master-8261" + data_dir: "/dm-data/dm-master-8261" + log_dir: "/dm-deploy/dm-master-8261/log" + numa_node: "0,1" + # The following configs are used to overwrite the `server_configs.master` values. + config: + log-level: info + rpc-timeout: "30s" + rpc-rate-limit: 10.0 + rpc-rate-burst: 40 + - host: 10.0.1.18 + name: master2 + - host: 10.0.1.19 + name: master3 +``` + +## `worker_servers` + +`worker_servers` specifies the machines to which the master node of the DM component is deployed. You can also specify the service configuration on each machine. `worker_servers` is an array. Each array element contains the following fields: + +- `host`: specifies the machine to deploy to. The field value is an IP address and is mandatory. +- `ssh_port`: specifies the SSH port to connect to the target machine for operations. If the field is not specified, the `ssh_port` in the `global` section is used. +- `name`: specifies the name of the DM-worker instance. The name must be unique for different instances. Otherwise, the cluster cannot be deployed. +- `port`: specifies the port on which DM-worker provides services. The default value is "8262". +- `deploy_dir`: specifies the deployment directory. If the field is not specified, or specified as a relative directory, the deployment directory is generated according to the `deploy_dir` configuration in the `global` section. +- `data_dir`: specifies the data directory. If the field is not specified, or specified as a relative directory, the data directory is generated according to the `data_dir` configuration in the `global` section. +- `log_dir`: specifies the log directory. If the field is not specified, or specified as a relative directory, the log directory is generated according to the `log_dir` configuration in the `global` section. +- `numa_node`: allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is a string type. The field value is the ID of the NUMA node, such as "0,1". +- `config`: the configuration rules of this field are the same as that of `worker` in the `server_configs` section. If `config` is specified, the configuration of `config` will be merged with the configuration of `worker` in `server_configs` (if the two fields overlap, the configuration of this field takes effect), and then the configuration file is generated and distributed to the machine specified in the `host` field. +- `os`: the operating system of the machine specified in the `host` field. If the field is not specified, the default value is the `os` value configured in the `global` section. +- `arch`: the architecture of the machine specified in the `host` field. If the field is not specified, the default value is the `arch` value configured in the `global` section. +- `resource_control`: resource control on this service. If this field is specified, the configuration of this field will be merged with the configuration of `resource_control` in the `global` section (if the two fields overlap, the configuration of this field takes effect), and then the configuration file of systemd is generated and distributed to the machine specified in the `host` field. The configuration rules of this field are the same as that of `resource_control` in the `global` section. + +In the `worker_servers` section, the following fields cannot be modified after the deployment is completed: + +- `host` +- `name` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `worker_servers` configuration example is as follows: + +```yaml +worker_servers: + - host: 10.0.1.12 + ssh_port: 22 + port: 8262 + deploy_dir: "/dm-deploy/dm-worker-8262" + log_dir: "/dm-deploy/dm-worker-8262/log" + numa_node: "0,1" + # config is used to overwrite the `server_configs.worker` values + config: + log-level: info + - host: 10.0.1.19 +``` + +### `monitoring_servers` + +`monitoring_servers` specifies the machines to which the Prometheus service is deployed. You can also specify the service configuration on the machine. `monitoring_servers` is an array. Each array element contains the following fields: + +- `host`: specifies the machine to deploy to. The field value is an IP address and is mandatory. +- `ssh_port`: specifies the SSH port to connect to the target machine for operations. If the field is not specified, the `ssh_port` in the `global` section is used. +- `port`: specifies the port on which Prometheus provides services. The default value is "9090". +- `deploy_dir`: specifies the deployment directory. If the field is not specified, or specified as a relative directory, the deployment directory is generated according to the `deploy_dir` configuration in the `global` section. +- `data_dir`: specifies the data directory. If the field is not specified, or specified as a relative directory, the data directory is generated according to the `data_dir` configuration in the `global` section. +- `log_dir`: specifies the log directory. If the field is not specified, or specified as a relative directory, the log directory is generated according to the `log_dir` configuration in the `global` section. +- `numa_node`: allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is a string type. The field value is the ID of the NUMA node, such as "0,1" +- `storage_retention`: specifies the retention time of the Prometheus monitoring data. The default value is "15d". +- `rule_dir`: specifies a local directory where the complete `*.rules.yml` files are located. The files in the specified directory will be sent to the target machine as the Prometheus rules during the initialization phase of the cluster configuration. +- `os`: the operating system of the machine specified in the `host` field. If the field is not specified, the default value is the `os` value configured in the `global` section. +- `arch`: the architecture of the machine specified in the `host` field. If the field is not specified, the default value is the `arch` value configured in the `global` section. +- `resource_control`: resource control on this service. If this field is specified, the configuration of this field will be merged with the configuration of `resource_control` in the `global` section (if the two fields overlap, the configuration of this field takes effect), and then the configuration file of systemd is generated and distributed to the machine specified in the `host` field. The configuration rules of this field are the same as that of `resource_control` in the `global` section. + +In the `monitoring_servers` section, the following fields cannot be modified after the deployment is completed: + +- `host` +- `port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +A `monitoring_servers` configuration example is as follows: + +```yaml +monitoring_servers: + - host: 10.0.1.11 + rule_dir: /local/rule/dir +``` + +### `grafana_servers` + +`grafana_servers` specifies the machines to which the Grafana service is deployed. You can also specify the service configuration on the machine. `grafana_servers` is an array. Each array element contains the following fields: + +- `host`: specifies the machine to deploy to. The field value is an IP address and is mandatory. +- `ssh_port`: specifies the SSH port to connect to the target machine for operations. If the field is not specified, the `ssh_port` in the `global` section is used. +- `port`: specifies the port on which Grafana provides services. The default value is "3000". +- `deploy_dir`: specifies the deployment directory. If the field is not specified, or specified as a relative directory, the deployment directory is generated according to the `deploy_dir` configuration in the `global` section. +- `os`: the operating system of the machine specified in the `host` field. If the field is not specified, the default value is the `os` value configured in the `global` section. +- `arch`: the architecture of the machine specified in the `host` field. If the field is not specified, the default value is the `arch` value configured in the `global` section. +- `username`: specifies the username of the Grafana login screen. +- `password`: specifies the corresponding password of Grafana. +- `dashboard_dir`: specifies a local directory where the complete `dashboard(*.json)` files are located. The files in the specified directory will be sent to the target machine as Grafana dashboards during the initialization phase of the cluster configuration. +- `resource_control`: resource control on this service. If this field is specified, the configuration of this field will be merged with the configuration of `resource_control` in the `global` section (if the two fields overlap, the configuration of this field takes effect), and then the configuration file of systemd is generated and distributed to the machine specified in the `host` field. The configuration rules of this field are the same as that of `resource_control` in the `global` section. + +> **Note:** +> +> If the `dashboard_dir` field of `grafana_servers` is configured, after executing the `tiup cluster rename` command to rename the cluster, you need to perform the following operations: +> +> 1. In the local `dashboards` directory, update the value of the `datasource` field to the new cluster name (the `datasource` is named after the cluster name). +> 2. Execute the `tiup cluster reload -R grafana` command. + +In `grafana_servers`, the following fields cannot be modified after the deployment is completed: + +- `host` +- `port` +- `deploy_dir` +- `arch` +- `os` + +A `grafana_servers` configuration example is as follows: + +```yaml +grafana_servers: + - host: 10.0.1.11 + dashboard_dir: /local/dashboard/dir +``` + +### `alertmanager_servers` + +`alertmanager_servers` specifies the machines to which the Alertmanager service is deployed. You can also specify the service configuration on each machine. `alertmanager_servers` is an array. Each array element contains the following fields: + +- `host`: specifies the machine to deploy to. The field value is an IP address and is mandatory. +- `ssh_port`: specifies the SSH port to connect to the target machine for operations. If the field is not specified, the `ssh_port` in the `global` section is used. +- `web_port`: specify the port on which Alertmanager provides web services. The default value is "9093". +- `cluster_port`: Specify the communication port between one Alertmanger and other Alertmanager. The default value is "9094". +- `deploy_dir`: specifies the deployment directory. If the field is not specified, or specified as a relative directory, the deployment directory is generated according to the `deploy_dir` configuration in the `global` section. +- `data_dir`: specifies the data directory. If the field is not specified, or specified as a relative directory, the data directory is generated according to the `data_dir` configuration in the `global` section. +- `log_dir`: specifies the log directory. If the field is not specified, or specified as a relative directory, the log directory is generated according to the `log_dir` configuration in the `global` section. +- `numa_node`: allocates the NUMA policy to the instance. Before specifying this field, you need to make sure that the target machine has [numactl](https://linux.die.net/man/8/numactl) installed. If this field is specified, cpubind and membind policies are allocated using [numactl](https://linux.die.net/man/8/numactl). This field is a string type. The field value is the ID of the NUMA node, such as "0,1" +- `config_file`: specifies a local file. The specified file will be sent to the target machine as the configuration for Alertmanager during the initialization phase of the cluster configuration. +- `os`: the operating system of the machine specified in the `host` field. If the field is not specified, the default value is the `os` value configured in the `global` section. +- `arch`: the architecture of the machine specified in the `host` field. If the field is not specified, the default value is the `arch` value configured in the `global` section. +- `resource_control`: resource control on this service. If this field is specified, the configuration of this field will be merged with the configuration of `resource_control` in the `global` section (if the two fields overlap, the configuration of this field takes effect), and then the configuration file of systemd is generated and distributed to the machine specified in the `host` field. The configuration rules of this field are the same as that of `resource_control` in the `global` section. + +In `alertmanager_servers`, the following fields cannot be modified after the deployment is completed: + +- `host` +- `web_port` +- `cluster_port` +- `deploy_dir` +- `data_dir` +- `log_dir` +- `arch` +- `os` + +An `alertmanager_servers` configuration example is as follows: + +```yaml +alertmanager_servers: + - host: 10.0.1.11 + config_file: /local/config/file + - host: 10.0.1.12 + config_file: /local/config/file +```