diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index dda16cb26a4e9..522afa658bc43 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -254,6 +254,7 @@ - [`ADMIN SHOW DDL [JOBS|JOB QUERIES]`](/sql-statements/sql-statement-admin-show-ddl.md) - [`ALTER DATABASE`](/sql-statements/sql-statement-alter-database.md) - [`ALTER INDEX`](/sql-statements/sql-statement-alter-index.md) + - [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md) - [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md) - [`ALTER TABLE COMPACT`](/sql-statements/sql-statement-alter-table-compact.md) - [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) @@ -267,6 +268,7 @@ - [`CREATE [GLOBAL|SESSION] BINDING`](/sql-statements/sql-statement-create-binding.md) - [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md) - [`CREATE INDEX`](/sql-statements/sql-statement-create-index.md) + - [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) - [`CREATE ROLE`](/sql-statements/sql-statement-create-role.md) - [`CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) - [`CREATE TABLE LIKE`](/sql-statements/sql-statement-create-table-like.md) @@ -282,6 +284,7 @@ - [`DROP COLUMN`](/sql-statements/sql-statement-drop-column.md) - [`DROP DATABASE`](/sql-statements/sql-statement-drop-database.md) - [`DROP INDEX`](/sql-statements/sql-statement-drop-index.md) + - [`DROP RESOURCE GROUP`](/sql-statements/sql-statement-drop-resource-group.md) - [`DROP ROLE`](/sql-statements/sql-statement-drop-role.md) - [`DROP SEQUENCE`](/sql-statements/sql-statement-drop-sequence.md) - [`DROP STATS`](/sql-statements/sql-statement-drop-stats.md) @@ -327,6 +330,7 @@ - [`SHOW COLLATION`](/sql-statements/sql-statement-show-collation.md) - [`SHOW [FULL] COLUMNS FROM`](/sql-statements/sql-statement-show-columns-from.md) - [`SHOW CREATE DATABASE`](/sql-statements/sql-statement-show-create-database.md) + - [`SHOW CREATE RESOURCE GROUP`](/sql-statements/sql-statement-show-create-resource-group.md) - [`SHOW CREATE SEQUENCE`](/sql-statements/sql-statement-show-create-sequence.md) - [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) - [`SHOW CREATE USER`](/sql-statements/sql-statement-show-create-user.md) @@ -444,6 +448,7 @@ - [`PARTITIONS`](/information-schema/information-schema-partitions.md) - [`PROCESSLIST`](/information-schema/information-schema-processlist.md) - [`REFERENTIAL_CONSTRAINTS`](/information-schema/information-schema-referential-constraints.md) + - [`RESOURCE_GROUPS`](/information-schema/information-schema-resource-groups.md) - [`SCHEMATA`](/information-schema/information-schema-schemata.md) - [`SEQUENCES`](/information-schema/information-schema-sequences.md) - [`SESSION_VARIABLES`](/information-schema/information-schema-session-variables.md) @@ -477,6 +482,7 @@ - [Table Filter](/table-filter.md) - [Troubleshoot Inconsistency Between Data and Indexes](/troubleshoot-data-inconsistency-errors.md) - [Serverless Tier Limitations](/tidb-cloud/serverless-tier-limitations.md) + - [Resource Control](/tidb-resource-control.md) - FAQs - [TiDB Cloud FAQs](/tidb-cloud/tidb-cloud-faq.md) - [Serverless Tier FAQs](/tidb-cloud/serverless-tier-faqs.md) diff --git a/TOC.md b/TOC.md index dfcd7b372af7b..25f67af6e744c 100644 --- a/TOC.md +++ b/TOC.md @@ -159,6 +159,7 @@ - [Back up and Restore Data Using Dumpling and TiDB Lightning](/backup-and-restore-using-dumpling-lightning.md) - [Back Up and Restore RawKV](/br/rawkv-backup-and-restore.md) - [Incremental Backup and Restore](/br/br-incremental-guide.md) + - [Resource Control](/tidb-resource-control.md) - [Configure Time Zone](/configure-time-zone.md) - [Daily Checklist](/daily-check.md) - [Maintain TiFlash](/tiflash/maintain-tiflash.md) @@ -608,6 +609,7 @@ - [TiKV](/grafana-tikv-dashboard.md) - [TiFlash](/tiflash/monitor-tiflash.md) - [TiCDC](/ticdc/monitor-ticdc.md) + - [Resource Control](/grafana-resource-control-dashboard.md) - Secure - [Enable TLS Between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md) - [Enable TLS Between TiDB Components](/enable-tls-between-components.md) @@ -647,6 +649,7 @@ - [`ALTER INDEX`](/sql-statements/sql-statement-alter-index.md) - [`ALTER INSTANCE`](/sql-statements/sql-statement-alter-instance.md) - [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) + - [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md) - [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md) - [`ALTER TABLE COMPACT`](/sql-statements/sql-statement-alter-table-compact.md) - [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) @@ -662,6 +665,7 @@ - [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md) - [`CREATE INDEX`](/sql-statements/sql-statement-create-index.md) - [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md) + - [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) - [`CREATE ROLE`](/sql-statements/sql-statement-create-role.md) - [`CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) - [`CREATE TABLE LIKE`](/sql-statements/sql-statement-create-table-like.md) @@ -678,6 +682,7 @@ - [`DROP DATABASE`](/sql-statements/sql-statement-drop-database.md) - [`DROP INDEX`](/sql-statements/sql-statement-drop-index.md) - [`DROP PLACEMENT POLICY`](/sql-statements/sql-statement-drop-placement-policy.md) + - [`DROP RESOURCE GROUP`](/sql-statements/sql-statement-drop-resource-group.md) - [`DROP ROLE`](/sql-statements/sql-statement-drop-role.md) - [`DROP SEQUENCE`](/sql-statements/sql-statement-drop-sequence.md) - [`DROP STATS`](/sql-statements/sql-statement-drop-stats.md) @@ -729,6 +734,7 @@ - [`SHOW CONFIG`](/sql-statements/sql-statement-show-config.md) - [`SHOW CREATE DATABASE`](/sql-statements/sql-statement-show-create-database.md) - [`SHOW CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-show-create-placement-policy.md) + - [`SHOW CREATE RESOURCE GROUP`](/sql-statements/sql-statement-show-create-resource-group.md) - [`SHOW CREATE SEQUENCE`](/sql-statements/sql-statement-show-create-sequence.md) - [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) - [`SHOW CREATE USER`](/sql-statements/sql-statement-show-create-user.md) @@ -855,6 +861,7 @@ - [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) - [`PROCESSLIST`](/information-schema/information-schema-processlist.md) - [`REFERENTIAL_CONSTRAINTS`](/information-schema/information-schema-referential-constraints.md) + - [`RESOURCE_GROUPS`](/information-schema/information-schema-resource-groups.md) - [`SCHEMATA`](/information-schema/information-schema-schemata.md) - [`SEQUENCES`](/information-schema/information-schema-sequences.md) - [`SESSION_VARIABLES`](/information-schema/information-schema-session-variables.md) diff --git a/grafana-resource-control-dashboard.md b/grafana-resource-control-dashboard.md new file mode 100644 index 0000000000000..c75719342b2b3 --- /dev/null +++ b/grafana-resource-control-dashboard.md @@ -0,0 +1,36 @@ +--- +title: Key Monitoring Metrics of Resource Control +summary: Learn some key metrics displayed on the Grafana Resource Control dashboard. +--- + +# Key Monitoring Metrics of Resource Control + +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). + +The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and Performance\_overview. + +If your cluster has used the [Resource Control](/tidb-resource-control.md) feature, you can get an overview of the resource consumption status from the Resource Control dashboard. + +This document describes some key monitoring metrics displayed on the Resource Control dashboard. + +## Metrics about Request Unit + +- RU: the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) consumption information of each resource group, calculated in real time. `total` is the sum of the Request Units consumed by all Resource Groups. The Request Unit consumption of each resource group should be equal to the sum of its read consumption (Read Request Unit) and write consumption (Write Request Unit). +- RU Per Query: the average number of Request Units consumed by each SQL statement per second. It is obtained by dividing the above RU metric by the number of SQL statements executed per second. +- RRU: the Read Request Unit consumption information of each resource group, calculated in real time. `total` is the sum of the Read Request Units consumed by all Resource Groups. +- RRU Per Query: the average number of Read Request Units consumed by each SQL statement per second. It is obtained by dividing the above RRU metric by the number of SQL statements executed per second. +- WRU: the Write Request Unit consumption information of each resource group, calculated in real time. `total` is the sum of the Write Request Units consumed by all Resource Groups. +- WRU Per Query: the average number of Write Request Units consumed by each SQL statement per second. It is obtained by dividing the above WRU metric by the number of SQL statements executed per second. + +## Metrics about resources + +- KV Request Count: the number of KV requests for each resource group, calculated per second. The requests are divided into read and write types. `total` is the sum of the KV requests for all Resource Groups. +- KV Request Count Per Query: the average number of read and write KV requests by each SQL statement per second. It is obtained by dividing the above KV Request Count metric by the number of SQL statements executed per second. +- Bytes Read: the amount of data read by each Resource Group, calculated per second. `total` is the sum of the data read by all Resource Groups. +- Bytes Read Per Query: the average amount of data read by each SQL statement per second. It is obtained by dividing the above Bytes Read metric by the number of SQL statements executed per second. +- Bytes Written: the amount of data written by each Resource Group, calculated in real time. `total` is the sum of the data written by all Resource Groups. +- Bytes Written Per Query: the average amount of data written by each SQL statement per second. It is obtained by dividing the above Bytes Written metric by the number of SQL statements executed per second. +- KV CPU Time: the KV layer CPU time consumed by each Resource Group, calculated in real time . `total` is the sum of the KV layer CPU time consumed by all Resource Groups. +- KV CPU Time Per Query: the average KV layer CPU time consumed by each SQL statement per second. It is obtained by dividing the above KV CPU Time metric by the number of SQL statements executed per second. +- SQL CPU Time: the SQL layer CPU time consumed by each Resource Group, calculated in real time. `total` is the sum of the SQL layer CPU time consumed by all Resource Groups. +- SQL CPU Time Per Query: the average SQL layer CPU time consumed by each SQL statement per second. It is obtained by dividing the above SQL CPU Time metric by the number of SQL statements executed per second. diff --git a/grafana-tikv-dashboard.md b/grafana-tikv-dashboard.md index e2b42fae80b97..20eb34b4f4e21 100644 --- a/grafana-tikv-dashboard.md +++ b/grafana-tikv-dashboard.md @@ -69,6 +69,9 @@ This section provides a detailed description of these key metrics on the **TiKV- - Average gRPC message duration: The average execution time of gRPC messages - gRPC batch size: The batch size of gRPC messages between TiDB and TiKV - Raft message batch size: The batch size of Raft messages between TiKV instances +- gRPC request sources QPS: The QPS of gRPC request sources +- gRPC request sources duration: The execution time of gRPC request sources +- gRPC resource group QPS: The QPS of gRPC request sources by resource groups ### Thread CPU diff --git a/information-schema/information-schema-processlist.md b/information-schema/information-schema-processlist.md index 2200c5944ecb7..919ac63b9da20 100644 --- a/information-schema/information-schema-processlist.md +++ b/information-schema/information-schema-processlist.md @@ -12,7 +12,8 @@ The `PROCESSLIST` table has additional columns not present in `SHOW PROCESSLIST` * A `DIGEST` column to show the digest of the SQL statement. * A `MEM` column to show the memory used by the request that is being processed, in bytes. * A `DISK` column to show the disk usage in bytes. -* A `TxnStart` column to show the start time of the transaction +* A `TxnStart` column to show the start time of the transaction. +* A `RESOURCE_GROUP` column to show the resource group name. {{< copyable "sql" >}} @@ -21,24 +22,25 @@ USE information_schema; DESC processlist; ``` -``` -+----------+---------------------+------+------+---------+-------+ -| Field | Type | Null | Key | Default | Extra | -+----------+---------------------+------+------+---------+-------+ -| ID | bigint(21) unsigned | NO | | 0 | | -| USER | varchar(16) | NO | | | | -| HOST | varchar(64) | NO | | | | -| DB | varchar(64) | YES | | NULL | | -| COMMAND | varchar(16) | NO | | | | -| TIME | int(7) | NO | | 0 | | -| STATE | varchar(7) | YES | | NULL | | -| INFO | longtext | YES | | NULL | | -| DIGEST | varchar(64) | YES | | | | -| MEM | bigint(21) unsigned | YES | | NULL | | -| DISK | bigint(21) unsigned | YES | | NULL | | -| TxnStart | varchar(64) | NO | | | | -+----------+---------------------+------+------+---------+-------+ -12 rows in set (0.00 sec) +```sql ++---------------------+---------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++---------------------+---------------------+------+------+---------+-------+ +| ID | bigint(21) unsigned | NO | | 0 | | +| USER | varchar(16) | NO | | | | +| HOST | varchar(64) | NO | | | | +| DB | varchar(64) | YES | | NULL | | +| COMMAND | varchar(16) | NO | | | | +| TIME | int(7) | NO | | 0 | | +| STATE | varchar(7) | YES | | NULL | | +| INFO | longtext | YES | | NULL | | +| DIGEST | varchar(64) | YES | | | | +| MEM | bigint(21) unsigned | YES | | NULL | | +| DISK | bigint(21) unsigned | YES | | NULL | | +| TxnStart | varchar(64) | NO | | | | +| RESOURCE_GROUP | varchar(32) | NO | | | | ++---------------------+---------------------+------+------+---------+-------+ +13 rows in set (0.00 sec) ``` {{< copyable "sql" >}} @@ -47,18 +49,21 @@ DESC processlist; SELECT * FROM processlist\G ``` -``` +```sql *************************** 1. row *************************** - ID: 16 - USER: root - HOST: 127.0.0.1 - DB: information_schema - COMMAND: Query - TIME: 0 - STATE: autocommit - INFO: SELECT * FROM processlist - MEM: 0 -TxnStart: + ID: 2300033189772525975 + USER: root + HOST: 127.0.0.1:51289 + DB: NULL + COMMAND: Query + TIME: 0 + STATE: autocommit + INFO: SELECT * FROM processlist + DIGEST: dbfaa16980ec628011029f0aaf0d160f4b040885240dfc567bf760d96d374f7e + MEM: 0 + DISK: 0 + TxnStart: + RESOURCE_GROUP: rg1 1 row in set (0.00 sec) ``` @@ -76,6 +81,7 @@ Fields in the `PROCESSLIST` table are described as follows: * MEM: The memory used by the request that is being processed, in bytes. * DISK: The disk usage in bytes. * TxnStart: The start time of the transaction. +* RESOURCE_GROUP: The resource group name. ## CLUSTER_PROCESSLIST @@ -88,13 +94,14 @@ SELECT * FROM information_schema.cluster_processlist; ``` ```sql -+-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+ -| INSTANCE | ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO | MEM | TxnStart | -+-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+ -| 10.0.1.22:10080 | 150 | u1 | 10.0.1.1 | test | Query | 0 | autocommit | select count(*) from usertable | 372 | 05-28 03:54:21.230(416976223923077223) | -| 10.0.1.22:10080 | 138 | root | 10.0.1.1 | test | Query | 0 | autocommit | SELECT * FROM information_schema.cluster_processlist | 0 | 05-28 03:54:21.230(416976223923077220) | -| 10.0.1.22:10080 | 151 | u1 | 10.0.1.1 | test | Query | 0 | autocommit | select count(*) from usertable | 372 | 05-28 03:54:21.230(416976223923077224) | -| 10.0.1.21:10080 | 15 | u2 | 10.0.1.1 | test | Query | 0 | autocommit | select max(field0) from usertable | 496 | 05-28 03:54:21.230(416976223923077222) | -| 10.0.1.21:10080 | 14 | u2 | 10.0.1.1 | test | Query | 0 | autocommit | select max(field0) from usertable | 496 | 05-28 03:54:21.230(416976223923077225) | -+-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+ ++-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+----------------+ +| INSTANCE | ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO | MEM | TxnStart | RESOURCE_GROUP | ++-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+----------------+ + +| 10.0.1.22:10080 | 150 | u1 | 10.0.1.1 | test | Query | 0 | autocommit | select count(*) from usertable | 372 | 05-28 03:54:21.230(416976223923077223) | default | +| 10.0.1.22:10080 | 138 | root | 10.0.1.1 | test | Query | 0 | autocommit | SELECT * FROM information_schema.cluster_processlist | 0 | 05-28 03:54:21.230(416976223923077220) | rg1 | +| 10.0.1.22:10080 | 151 | u1 | 10.0.1.1 | test | Query | 0 | autocommit | select count(*) from usertable | 372 | 05-28 03:54:21.230(416976223923077224) | rg2 | +| 10.0.1.21:10080 | 15 | u2 | 10.0.1.1 | test | Query | 0 | autocommit | select max(field0) from usertable | 496 | 05-28 03:54:21.230(416976223923077222) | default | +| 10.0.1.21:10080 | 14 | u2 | 10.0.1.1 | test | Query | 0 | autocommit | select max(field0) from usertable | 496 | 05-28 03:54:21.230(416976223923077225) | default | ++-----------------+-----+------+----------+------+---------+------+------------+------------------------------------------------------+-----+----------------------------------------+----------------+ ``` diff --git a/information-schema/information-schema-resource-groups.md b/information-schema/information-schema-resource-groups.md new file mode 100644 index 0000000000000..8b094de03661e --- /dev/null +++ b/information-schema/information-schema-resource-groups.md @@ -0,0 +1,63 @@ +--- +title: RESOURCE_GROUPS +summary: Learn the `RESOURCE_GROUPS` information_schema table. +--- + +# RESOURCE_GROUPS + +> **Warning:** +> +> This feature is experimental and its form and usage might change in subsequent versions. + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md). + +```sql +USE information_schema; +DESC resource_groups; +``` + +```sql ++------------+-------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++------------+-------------+------+------+---------+-------+ +| NAME | varchar(32) | NO | | NULL | | +| RU_PER_SEC | bigint(21) | YES | | NULL | | +| BURSTABLE | varchar(3) | YES | | NULL | | ++------------+-------------+------+------+---------+-------+ +3 rows in set (0.00 sec) +``` + +## Examples + +```sql +mysql> CREATE RESOURCE GROUP rg1 RU_PER_SEC=1000; -- Create the resource group rg1 +Query OK, 0 rows affected (0.34 sec) +mysql> SHOW CREATE RESOURCE GROUP rg1; -- Display the definition of the rg1 resource group ++----------------+---------------------------------------------+ +| Resource_Group | Create Resource Group | ++----------------+---------------------------------------------+ +| rg1 | CREATE RESOURCE GROUP `rg1` RU_PER_SEC=1000 | ++----------------+---------------------------------------------+ +1 row in set (0.00 sec) +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME = 'rg1'; ++------+------------+-----------+ +| NAME | RU_PER_SEC | BURSTABLE | ++------+------------+-----------+ +| rg1 | 1000 | NO | ++------+------------+-----------+ +1 row in set (0.00 sec) +``` + +The descriptions of the columns in the `RESOURCE_GROUPS` table are as follows: + +* `NAME`: the name of the resource group. +* `RU_PER_SEC`：the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control.md#what-is-request-unit-ru). +* `BURSTABLE`: whether to allow the resource group to overuse the available system resources. diff --git a/mysql-compatibility.md b/mysql-compatibility.md index 0e3f006fada8f..2fe3480278848 100644 --- a/mysql-compatibility.md +++ b/mysql-compatibility.md @@ -256,3 +256,11 @@ TiDB does not implement certain features that have been marked as deprecated in * Specifying precision for floating point types. MySQL 8.0 [deprecates](https://dev.mysql.com/doc/refman/8.0/en/floating-point-types.html) this feature, and it is recommended to use the `DECIMAL` type instead. * The `ZEROFILL` attribute. MySQL 8.0 [deprecates](https://dev.mysql.com/doc/refman/8.0/en/numeric-type-attributes.html) this feature, and it is recommended to instead pad numeric values in your application. + +### `CREATE RESOURCE GROUP`, `DROP RESOURCE GROUP`, and `ALTER RESOURCE GROUP` statements + +For the statements of creating, modifying, and dropping resource groups, the supported parameters are different from that of MySQL. See the following documents for details: + +- [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) +- [`DROP RESOURCE GROUP`](/sql-statements/sql-statement-drop-resource-group.md) +- [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md) diff --git a/privilege-management.md b/privilege-management.md index 0d62cc731af05..d7adedb933bf8 100644 --- a/privilege-management.md +++ b/privilege-management.md @@ -364,6 +364,18 @@ Requires `SUPER` privilege. Requires `SUPER` or `CONNECTION_ADMIN` privilege to kill other user sessions. +### CREATE RESOURCE GROUP + +Requires `SUPER` or `RESOURCE_GROUP_ADMIN` privilege. + +### ALTER RESOURCE GROUP + +Requires `SUPER` or `RESOURCE_GROUP_ADMIN` privilege. + +### DROP RESOURCE GROUP + +Requires `SUPER` or `RESOURCE_GROUP_ADMIN` privilege. + ## Implementation of the privilege system ### Privilege table diff --git a/sql-statements/sql-statement-alter-resource-group.md b/sql-statements/sql-statement-alter-resource-group.md new file mode 100644 index 0000000000000..088f2b5759abe --- /dev/null +++ b/sql-statements/sql-statement-alter-resource-group.md @@ -0,0 +1,91 @@ +--- +title: ALTER RESOURCE GROUP +summary: Learn the usage of ALTER RESOURCE GROUP in TiDB. +--- + +# ALTER RESOURCE GROUP + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +The `ALTER RESOURCE GROUP` statement is used to modify a resource group in a database. + +## Synopsis + +```ebnf+diagram +AlterResourceGroupStmt: + "ALTER" "RESOURCE" "GROUP" IfExists ResourceGroupName ResourceGroupOptionList + +IfExists ::= + ('IF' 'EXISTS')? + +ResourceGroupName: + Identifier + +ResourceGroupOptionList: + DirectResourceGroupOption +| ResourceGroupOptionList DirectResourceGroupOption +| ResourceGroupOptionList ',' DirectResourceGroupOption + +DirectResourceGroupOption: + "RU_PER_SEC" EqOpt stringLit +| "BURSTABLE" + +``` + +TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. + +| Option | Description | Example | +|---------------|-------------------------------------|------------------------| +| `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second | + +If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. + +> **Note:** +> +> The `ALTER RESOURCE GROUP` statement can only be executed when the global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) is set to `ON`. + +## Examples + +Create a resource group named `rg1` and modify its properties. + +```sql +mysql> DROP RESOURCE GROUP IF EXISTS rg1; +Query OK, 0 rows affected (0.22 sec) +mysql> CREATE RESOURCE GROUP IF NOT EXISTS rg1 + -> RU_PER_SEC = 100 + -> BURSTABLE; +Query OK, 0 rows affected (0.08 sec) +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1'; ++------+------------+-----------+ +| NAME | RU_PER_SEC | BURSTABLE | ++------+------------+-----------+ +| rg1 | 100 | YES | ++------+------------+-----------+ +1 rows in set (1.30 sec) +mysql> ALTER RESOURCE GROUP rg1 + -> RU_PER_SEC = 200; +Query OK, 0 rows affected (0.08 sec) +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1'; ++------+------------+-----------+ +| NAME | RU_PER_SEC | BURSTABLE | ++------+------------+-----------+ +| rg1 | 200 | NO | ++------+------------+-----------+ +1 rows in set (1.30 sec) +``` + +## MySQL compatibility + +MySQL also supports [ALTER RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/en/alter-resource-group.html). However, the acceptable parameters are different from that of TiDB so that they are not compatible. + +## See also + +* [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) +* [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) +* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) diff --git a/sql-statements/sql-statement-alter-user.md b/sql-statements/sql-statement-alter-user.md index c437c2de633d5..18ef49ada0b92 100644 --- a/sql-statements/sql-statement-alter-user.md +++ b/sql-statements/sql-statement-alter-user.md @@ -12,7 +12,7 @@ This statement changes an existing user inside the TiDB privilege system. In the ```ebnf+diagram AlterUserStmt ::= - 'ALTER' 'USER' IfExists (UserSpecList RequireClauseOpt ConnectionOptions PasswordOption LockOption AttributeOption | 'USER' '(' ')' 'IDENTIFIED' 'BY' AuthString) + 'ALTER' 'USER' IfExists (UserSpecList RequireClauseOpt ConnectionOptions PasswordOption LockOption AttributeOption | 'USER' '(' ')' 'IDENTIFIED' 'BY' AuthString) ResourceGroupNameOption UserSpecList ::= UserSpec ( ',' UserSpec )* @@ -31,6 +31,8 @@ PasswordOption ::= ( 'PASSWORD' 'EXPIRE' ( 'DEFAULT' | 'NEVER' | 'INTERVAL' N 'D LockOption ::= ( 'ACCOUNT' 'LOCK' | 'ACCOUNT' 'UNLOCK' )? AttributeOption ::= ( 'COMMENT' CommentString | 'ATTRIBUTE' AttributeString )? + +ResourceGroupNameOption::= ( 'RESOURCE' 'GROUP' Identifier)? ``` ## Examples @@ -135,6 +137,16 @@ ALTER USER 'newuser' PASSWORD REUSE INTERVAL 90 DAY; Query OK, 0 rows affected (0.02 sec) ``` +Use `ALTER USER ... RESOURCE GROUP` to modify the resource group of the user `newuser` to `rg1`. + +```sql +ALTER USER 'newuser' RESOURCE GROUP rg1; +``` + +``` +Query OK, 0 rows affected (0.02 sec) +``` + ## See also diff --git a/sql-statements/sql-statement-create-resource-group.md b/sql-statements/sql-statement-create-resource-group.md new file mode 100644 index 0000000000000..39ab52ac72e16 --- /dev/null +++ b/sql-statements/sql-statement-create-resource-group.md @@ -0,0 +1,87 @@ +--- +title: CREATE RESOURCE GROUP +summary: Learn the usage of CREATE RESOURCE GROUP in TiDB. +--- + +# CREATE RESOURCE GROUP + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +You can use the `CREATE RESOURCE GROUP` statement to create a resource group. + +## Synopsis + +```ebnf+diagram +CreateResourceGroupStmt: + "CREATE" "RESOURCE" "GROUP" IfNotExists ResourceGroupName ResourceGroupOptionList + +IfNotExists ::= + ('IF' 'NOT' 'EXISTS')? + +ResourceGroupName: + Identifier + +ResourceGroupOptionList: + DirectResourceGroupOption +| ResourceGroupOptionList DirectResourceGroupOption +| ResourceGroupOptionList ',' DirectResourceGroupOption + +DirectResourceGroupOption: + "RU_PER_SEC" EqOpt stringLit +| "BURSTABLE" + +``` + +The resource group name parameter (`ResourceGroupName`) must be globally unique. + +TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. + +| Option | Description | Example | +|---------------|-------------------------------------|------------------------| +| `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second | + +If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. + +> **Note:** +> +> The `CREATE RESOURCE GROUP` statement can only be executed when the global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) is set to `ON`. + +## Examples + +Create two resource groups `rg1` and `rg2`. + +```sql +mysql> DROP RESOURCE GROUP IF EXISTS rg1; +Query OK, 0 rows affected (0.22 sec) +mysql> CREATE RESOURCE GROUP IF NOT EXISTS rg1 + -> RU_PER_SEC = 100 + -> BURSTABLE; +Query OK, 0 rows affected (0.08 sec) +mysql> CREATE RESOURCE GROUP IF NOT EXISTS rg2 + -> RU_PER_SEC = 200; +Query OK, 0 rows affected (0.08 sec) +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1' or NAME = 'rg2'; ++------+-------------+-----------+ +| NAME | RU_PER_SEC | BURSTABLE | ++------+-------------+-----------+ +| rg1 | 100 | YES | +| rg2 | 200 | NO | ++------+-------------+-----------+ +2 rows in set (1.30 sec) +``` + +## MySQL compatibility + +MySQL also supports [CREATE RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/en/create-resource-group.html). However, the acceptable parameters are different from that of TiDB so that they are not compatible. + +## See also + +* [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) +* [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) +* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) diff --git a/sql-statements/sql-statement-create-user.md b/sql-statements/sql-statement-create-user.md index c6f96317adc78..db460762dbaea 100644 --- a/sql-statements/sql-statement-create-user.md +++ b/sql-statements/sql-statement-create-user.md @@ -12,7 +12,7 @@ This statement creates a new user, specified with a password. In the MySQL privi ```ebnf+diagram CreateUserStmt ::= - 'CREATE' 'USER' IfNotExists UserSpecList RequireClauseOpt ConnectionOptions PasswordOption LockOption AttributeOption + 'CREATE' 'USER' IfNotExists UserSpecList RequireClauseOpt ConnectionOptions PasswordOption LockOption AttributeOption ResourceGroupNameOption IfNotExists ::= ('IF' 'NOT' 'EXISTS')? @@ -35,6 +35,8 @@ PasswordOption ::= ( 'PASSWORD' 'EXPIRE' ( 'DEFAULT' | 'NEVER' | 'INTERVAL' N 'D LockOption ::= ( 'ACCOUNT' 'LOCK' | 'ACCOUNT' 'UNLOCK' )? AttributeOption ::= ( 'COMMENT' CommentString | 'ATTRIBUTE' AttributeString )? + +ResourceGroupNameOption::= ( 'RESOURCE' 'GROUP' Identifier)? ``` ## Examples @@ -129,6 +131,22 @@ CREATE USER 'newuser9'@'%' PASSWORD EXPIRE; Query OK, 1 row affected (0.02 sec) ``` +Create a user that uses the resource group `rg1`. + +```sql +CREATE USER 'newuser7'@'%' RESOURCE GROUP rg1; +SELECT USER, HOST, USER_ATTRIBUTES FROM MYSQL.USER WHERE USER='newuser7'; +``` + +```sql ++----------+------+---------------------------+ +| USER | HOST | USER_ATTRIBUTES | ++----------+------+---------------------------+ +| newuser7 | % | {"resource_group": "rg1"} | ++----------+------+---------------------------+ +1 rows in set (0.00 sec) +``` + ## MySQL compatibility The following `CREATE USER` options are not yet supported by TiDB, and will be parsed but ignored: diff --git a/sql-statements/sql-statement-drop-resource-group.md b/sql-statements/sql-statement-drop-resource-group.md new file mode 100644 index 0000000000000..2edd398c59be8 --- /dev/null +++ b/sql-statements/sql-statement-drop-resource-group.md @@ -0,0 +1,67 @@ +--- +title: DROP RESOURCE GROUP +summary: Learn the usage of DROP RESOURCE GROUP in TiDB. +--- + +# DROP RESOURCE GROUP + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +You can use the `DROP RESOURCE GROUP` statement to drop a resource group. + +## Synopsis + +```ebnf+diagram +DropResourceGroupStmt: + "DROP" "RESOURCE" "GROUP" IfExists ResourceGroupName + +IfExists ::= + ('IF' 'EXISTS')? + +ResourceGroupName: + Identifier +``` + +> **Note:** +> +> The `DROP RESOURCE GROUP` statement can only be executed when the global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) is set to `ON`. + +## Examples + +Drop a resource group named `rg1`. + +```sql +mysql> DROP RESOURCE GROUP IF EXISTS rg1; +Query OK, 0 rows affected (0.22 sec) +mysql> CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 BURSTABLE; +Query OK, 0 rows affected (0.08 sec) +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1'; ++------+------------+-----------+ +| NAME | RU_PER_SEC | BURSTABLE | ++------+------------+-----------+ +| rg1 | 500 | YES | ++------+------------+-----------+ +1 row in set (0.01 sec) + +mysql> DROP RESOURCE GROUP IF EXISTS rg1; +Query OK, 1 rows affected (0.09 sec) + +mysql> SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1'; +Empty set (0.00 sec) +``` + +## MySQL compatibility + +MySQL also supports [DROP RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/en/drop-resource-group.html), but TiDB does not support the `FORCE` parameter. + +## See also + +* [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) +* [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) +* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-create-resource-group.md b/sql-statements/sql-statement-show-create-resource-group.md new file mode 100644 index 0000000000000..ab3523a14df24 --- /dev/null +++ b/sql-statements/sql-statement-show-create-resource-group.md @@ -0,0 +1,59 @@ +--- +title: SHOW CREATE RESOURCE GROUP +summary: Learn the usage of SHOW CREATE RESOURCE GROUP in TiDB. +--- + +# SHOW CREATE RESOURCE GROUP + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +You can use the `SHOW CREATE RESOURCE GROUP` statement to view the current definition of a resource group. + +## Synopsis + +```ebnf+diagram +ShowCreateResourceGroupStmt ::= + "SHOW" "CREATE" "RESOURCE" "GROUP" ResourceGroupName + +ResourceGroupName ::= + Identifier +``` + +## Examples + +Create a resource group `rg1`. + +```sql +CREATE RESOURCE GROUP rg1 RU_PER_SEC=100; +Query OK, 0 rows affected (0.10 sec) +``` + +View the definition of `rg1`. + +```sql +SHOW CREATE RESOURCE GROUP rg1; +***************************[ 1. row ]*************************** ++----------------+--------------------------------------------+ +| Resource_Group | Create Resource Group | ++----------------+--------------------------------------------+ +| rg1 | CREATE RESOURCE GROUP `rg1` RU_PER_SEC=100 | ++----------------+--------------------------------------------+ +1 row in set (0.01 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension for MySQL. + +## See also + +* [TiDB RESOURCE CONTROL](/tidb-resource-control.md) +* [CREATE RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) +* [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) +* [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) diff --git a/sql-statements/sql-statement-show-privileges.md b/sql-statements/sql-statement-show-privileges.md index 83abfc6d52c3c..986e1260b71fe 100644 --- a/sql-statements/sql-statement-show-privileges.md +++ b/sql-statements/sql-statement-show-privileges.md @@ -18,43 +18,60 @@ This statement shows a list of assignable privileges in TiDB. It is a static lis ```sql mysql> show privileges; -+-------------------------+---------------------------------------+-------------------------------------------------------+ -| Privilege | Context | Comment | -+-------------------------+---------------------------------------+-------------------------------------------------------+ -| Alter | Tables | To alter the table | -| Alter | Tables | To alter the table | -| Alter routine | Functions,Procedures | To alter or drop stored functions/procedures | -| Create | Databases,Tables,Indexes | To create new databases and tables | -| Create routine | Databases | To use CREATE FUNCTION/PROCEDURE | -| Create temporary tables | Databases | To use CREATE TEMPORARY TABLE | -| Create view | Tables | To create new views | -| Create user | Server Admin | To create new users | -| Delete | Tables | To delete existing rows | -| Drop | Databases,Tables | To drop databases, tables, and views | -| Event | Server Admin | To create, alter, drop and execute events | -| Execute | Functions,Procedures | To execute stored routines | -| File | File access on server | To read and write files on the server | -| Grant option | Databases,Tables,Functions,Procedures | To give to other users those privileges you possess | -| Index | Tables | To create or drop indexes | -| Insert | Tables | To insert data into tables | -| Lock tables | Databases | To use LOCK TABLES (together with SELECT privilege) | -| Process | Server Admin | To view the plain text of currently executing queries | -| Proxy | Server Admin | To make proxy user possible | -| References | Databases,Tables | To have references on tables | -| Reload | Server Admin | To reload or refresh tables, logs and privileges | -| Replication client | Server Admin | To ask where the slave or master servers are | -| Replication slave | Server Admin | To read binary log events from the master | -| Select | Tables | To retrieve rows from table | -| Show databases | Server Admin | To see all databases with SHOW DATABASES | -| Show view | Tables | To see views with SHOW CREATE VIEW | -| Shutdown | Server Admin | To shut down the server | -| Super | Server Admin | To use KILL thread, SET GLOBAL, CHANGE MASTER, etc. | -| Trigger | Tables | To use triggers | -| Create tablespace | Server Admin | To create/alter/drop tablespaces | -| Update | Tables | To update existing rows | -| Usage | Server Admin | No privileges - allow connect only | -+-------------------------+---------------------------------------+-------------------------------------------------------+ -32 rows in set (0.00 sec) ++---------------------------------+---------------------------------------+-------------------------------------------------------+ +| Privilege | Context | Comment | ++---------------------------------+---------------------------------------+-------------------------------------------------------+ +| Alter | Tables | To alter the table | +| Alter routine | Functions,Procedures | To alter or drop stored functions/procedures | +| Config | Server Admin | To use SHOW CONFIG and SET CONFIG statements | +| Create | Databases,Tables,Indexes | To create new databases and tables | +| Create routine | Databases | To use CREATE FUNCTION/PROCEDURE | +| Create role | Server Admin | To create new roles | +| Create temporary tables | Databases | To use CREATE TEMPORARY TABLE | +| Create view | Tables | To create new views | +| Create user | Server Admin | To create new users | +| Delete | Tables | To delete existing rows | +| Drop | Databases,Tables | To drop databases, tables, and views | +| Drop role | Server Admin | To drop roles | +| Event | Server Admin | To create, alter, drop and execute events | +| Execute | Functions,Procedures | To execute stored routines | +| File | File access on server | To read and write files on the server | +| Grant option | Databases,Tables,Functions,Procedures | To give to other users those privileges you possess | +| Index | Tables | To create or drop indexes | +| Insert | Tables | To insert data into tables | +| Lock tables | Databases | To use LOCK TABLES (together with SELECT privilege) | +| Process | Server Admin | To view the plain text of currently executing queries | +| Proxy | Server Admin | To make proxy user possible | +| References | Databases,Tables | To have references on tables | +| Reload | Server Admin | To reload or refresh tables, logs and privileges | +| Replication client | Server Admin | To ask where the slave or master servers are | +| Replication slave | Server Admin | To read binary log events from the master | +| Select | Tables | To retrieve rows from table | +| Show databases | Server Admin | To see all databases with SHOW DATABASES | +| Show view | Tables | To see views with SHOW CREATE VIEW | +| Shutdown | Server Admin | To shut down the server | +| Super | Server Admin | To use KILL thread, SET GLOBAL, CHANGE MASTER, etc. | +| Trigger | Tables | To use triggers | +| Create tablespace | Server Admin | To create/alter/drop tablespaces | +| Update | Tables | To update existing rows | +| Usage | Server Admin | No privileges - allow connect only | +| BACKUP_ADMIN | Server Admin | | +| RESTORE_ADMIN | Server Admin | | +| SYSTEM_USER | Server Admin | | +| SYSTEM_VARIABLES_ADMIN | Server Admin | | +| ROLE_ADMIN | Server Admin | | +| CONNECTION_ADMIN | Server Admin | | +| PLACEMENT_ADMIN | Server Admin | | +| DASHBOARD_CLIENT | Server Admin | | +| RESTRICTED_TABLES_ADMIN | Server Admin | | +| RESTRICTED_STATUS_ADMIN | Server Admin | | +| RESTRICTED_VARIABLES_ADMIN | Server Admin | | +| RESTRICTED_USER_ADMIN | Server Admin | | +| RESTRICTED_CONNECTION_ADMIN | Server Admin | | +| RESTRICTED_REPLICA_WRITER_ADMIN | Server Admin | | +| RESOURCE_GROUP_ADMIN | Server Admin | | ++---------------------------------+---------------------------------------+-------------------------------------------------------+ +49 rows in set (0.00 sec) ``` ## MySQL compatibility diff --git a/system-variables.md b/system-variables.md index 31fe36ea35fad..308ff9a5aafbb 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1825,6 +1825,18 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a +### `tidb_enable_resource_control` New in v6.6.0 + +> **Warning:** +> +> This feature is experimental and its form and usage might change in subsequent versions. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `OFF` +- Type: Boolean +- This variable is a switch for [the resource control feature](/tidb-resource-control.md). When this variable is set to `ON`, the TiDB cluster can isolate application resources based on resource groups. + ### tidb_enable_reuse_chunk New in v6.4.0 - Scope: SESSION | GLOBAL diff --git a/tidb-resource-control.md b/tidb-resource-control.md new file mode 100644 index 0000000000000..fb3af5fb7f6fa --- /dev/null +++ b/tidb-resource-control.md @@ -0,0 +1,197 @@ +--- +title: Use Resource Control to Achieve Resource Isolation +summary: Learn how to use the resource control feature to control and schedule application resources. +--- + +# Use Resource Control to Achieve Resource Isolation + +> **Warning:** +> +> This feature is experimental and its form and usage might change in subsequent versions. + + + +> **Note:** +> +> This feature is not available on [Serverless Tier clusters](/tidb-cloud/select-cluster-tier.md#serverless-tier-beta). + + + +As a cluster administrator, you can use the resource control feature to create resource groups, set read and write quotas for resource groups, and bind users to those groups. This allows the TiDB layer to control the flow of user read and write requests based on the quotas set for the resource groups, and allows the TiKV layer to schedule the requests based on the priority mapped to the read and write quota. By doing this, you can ensure resource isolation for your applications and meet quality of service (QoS) requirements. + +The TiDB resource control feature provides two layers of resource management capabilities: the flow control capability at the TiDB layer and the priority scheduling capability at the TiKV layer. The two capabilities can be enabled separately or simultaneously. See the [Parameters for resource control](#parameters-for-resource-control) for details. + +- TiDB flow control: TiDB flow control uses the [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket). If there are not enough tokens in a bucket, and the resource group does not specify the `BURSTABLE` option, the requests to the resource group will wait for the token bucket to backfill the tokens and retry. The retry might fail due to timeout. + + + +- TiKV scheduling: if [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) is enabled, TiKV uses the value of `RU_PER_SEC` of each resource group to determine the priority of the read and write requests for each resource group. Based on the priorities, the storage layer uses the priority queue to schedule and process requests. + + + + + +- TiKV scheduling: for on-premises TiDB, if the `resource-control.enabled` parameter is enabled, TiKV uses the value of `RU_PER_SEC` of each resource group to determine the priority of the read and write requests for each resource group. Based on the priorities, the storage layer uses the priority queue to schedule and process requests. For TiDB Cloud, the value of the `resource-control.enabled` parameter is `false` by default and does not support dynamic modification. If you need to enable it for TiDB Cloud Dedicated Tier clusters, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). + + + +## Scenarios for resource control + +The introduction of the resource control feature is a milestone for TiDB. It can divide a distributed database cluster into multiple logical units. Even if an individual unit overuses resources, it does not crowd out the resources needed by other units. + +With this feature, you can: + +- Combine multiple small and medium-sized applications from different systems into a single TiDB cluster. When the workload of an application grows larger, it does not affect the normal operation of other applications. When the system workload is low, busy applications can still be allocated the required system resources even if they exceed the set read and write quotas, so as to achieve the maximum utilization of resources. +- Choose to combine all test environments into a single TiDB cluster, or group the batch tasks that consume more resources into a single resource group. It can improve hardware utilization and reduce operating costs while ensuring that critical applications can always get the necessary resources. + +In addition, the rational use of the resource control feature can reduce the number of clusters, ease the difficulty of operation and maintenance, and save management costs. + +## What is Request Unit (RU) + +Request Unit (RU) is a unified abstraction unit in TiDB for system resources, which currently includes CPU, IOPS, and IO bandwidth metrics. The consumption of these three metrics is represented by RU according to a certain ratio. + +The following table shows the consumption of TiKV storage layer CPU and IO resources by user requests and the corresponding RU weights. + +| Resource | RU Weight | +|:----------------|:-----------------| +| CPU | 1/3 RU per millisecond | +| Read IO | 1/64 RU per KB | +| Write IO | 1 RU/KB | +| Basic overhead of a read request | 0.25 RU | +| Basic overhead of a write request | 1.5 RU | + +Based on the above table, assuming that the TiKV time consumed by a resource group is `c` milliseconds, `r1` times of requests read `r2` KB data, `w1` times of write requests write `w2` KB data, and the number of non-witness TiKV nodes in the cluster is `n`. Then, the formula for the total RUs consumed by the resource group is as follows: + +`c`\* 1/3 + (`r1` \* 0.25 + `r2` \* 1/64) + (1.5 \* `w1` + `w2` \* 1 \* `n`) + +## Parameters for resource control + +The resource control feature introduces two new global variables. + +* TiDB: you can use the [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) system variable to control whether to enable flow control for resource groups. + + + +* TiKV: you can use the [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) parameter to control whether to use request scheduling based on resource groups. + + + + + +* TiKV: For on-premises TiDB, you can use the `resource-control.enabled` parameter to control whether to use request scheduling based on resource group quotas. For TiDB Cloud, the value of the `resource-control.enabled` parameter is `false` by default and does not support dynamic modification. If you need to enable it for TiDB Cloud Dedicated Tier clusters, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). + + + +The results of the combinations of these two parameters are shown in the following table. + +| `resource-control.enabled` | `tidb_enable_resource_control`= ON | `tidb_enable_resource_control`= OFF | +|:----------------------------|:-------------------------------------|:-------------------------------------| +| `resource-control.enabled`= true | Flow control and scheduling (recommended) | Invalid combination | +| `resource-control.enabled`= false | Only flow control (not recommended) | The feature is disabled. | + +For more information about the resource control mechanism and parameters, see [RFC: Global Resource Control in TiDB](https://docs.google.com/document/d/1sV5EVv8Cdpc6aBCDihc2akpE0iuantPf/edit?n=RFC_Global_Resource_Control_in_TiDB.docx#heading=h.sjp9cmjfszlu). + +## How to use resource control + +To create, modify, or delete a resource group, you need to have the `SUPER` or `RESOURCE_GROUP_ADMIN` privilege. + +You can create a resource group in the cluster by using [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md), and then bind users to a specific resource group by using [`CREATE USER`](/sql-statements/ sql-statement-create-user.md) or [`ALTER USER`](/sql-statements/sql-statement-alter-user.md). + +For an existing resource group, you can modify the `RU_PER_SEC` option (the rate of RU backfilling per second) of the resource group by using [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). The changes to the resource group take effect immediately. + +You can delete a resource group by using [`DROP RESOURCE GROUP`](/sql-statements/sql-statement-drop-resource-group.md). + +> **Note:** +> +> - When you bind a user to a resource group by using `CREATE USER` or `ALTER USER`, it will not take effect for the user's existing sessions, but only for the user's new sessions. +> - If a user is not bound to a resource group or is bound to a `default` resource group, the user's requests are not subject to the flow control restrictions of TiDB. The `default` resource group is currently not visible to the user and cannot be created or modified. You cannot view it with `SHOW CREATE RESOURCE GROUP` or `SELECT * FROM information_schema.resource_groups`. But you can view it through the `mysql.user` table. + +### Step 1. Enable the resource control feature + +Enable the resource control feature. + +```sql +SET GLOBAL tidb_enable_resource_control = 'ON'; +``` + + + +Set the TiKV [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) parameter to `true`. + + + + + +For on-premises TiDB, set the TiKV `resource-control.enabled` parameter to `true`. For TiDB Cloud, the value of the `resource-control.enabled` parameter is `false` by default and does not support dynamic modification. If you need to enable it for TiDB Cloud Dedicated Tier clusters, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). + + + +### Step 2. Create a resource group, and then bind users to it + +The following is an example of how to create a resource group and bind users to it. + +1. Create a resource group `rg1`. The RU backfill rate is 500 RUs per second and allows applications in this resource group to overrun resources. + + ```sql + CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 BURSTABLE; + ``` + +2. Create a resource group `rg2`. The RU backfill rate is 600 RUs per second and does not allow applications in this resource group to overrun resources. + + ```sql + CREATE RESOURCE GROUP IF NOT EXISTS rg2 RU_PER_SEC = 600; + ``` + +3. Bind users `usr1` and `usr2` to resource groups `rg1` and `rg2` respectively. + + ```sql + ALTER USER usr1 RESOURCE GROUP rg1; + ``` + + ```sql + ALTER USER usr2 RESOURCE GROUP rg2; + ``` + +After you complete the above operations of creating resource groups and binding users, the resource consumption of newly created sessions will be controlled by the specified quota. If the system workload is relatively high and there is no spare capacity, the resource consumption rate of `usr2` will be strictly controlled not to exceed the quota. Because `usr1` is bound by `rg1` with `BURSTABLE` configured, the consumption rate of `usr1` is allowed to exceed the quota. + +If there are too many requests that result in insufficient resources for the resource group, the client's requests will wait. If the wait time is too long, the requests will report an error. + +## Monitoring metrics and charts + + + +TiDB regularly collects runtime information about resource control and provides visual charts of the metrics in Grafana's **TiDB** > **Resource Control** dashboard. The metrics are detailed in the **Resource Control** section of [TiDB Important Monitoring Metrics](/grafana-tidb-dashboard.md). + +TiKV also records the request QPS from different resource groups. For more details, see [TiKV Monitoring Metrics Detail](/grafana-tikv-dashboard.md#grpc). + + + + + +> **Note:** +> +> This section is only applicable to on-premises TiDB. Currently, TiDB Cloud does not provide resource control metrics. + +TiDB regularly collects runtime information about resource control and provides visual charts of the metrics in Grafana's **TiDB** > **Resource Control** dashboard. + +TiKV also records the request QPS from different resource groups in Grafana's **TiKV** dashboard. + + + +## Tool compatibility + +The resource control feature is still in its experimental stage and does not impact the regular usage of data import, export, and other replication tools. BR, TiDB Lightning, and TiCDC do not currently support processing DDL operations related to resource control, and their resource consumption is not limited by resource control. + +## Limitations + +Currently, the resource control feature has the following limitations: + +* This feature only supports flow control and scheduling of read and write requests initiated by foreground clients. It does not support flow control and scheduling of background tasks such as DDL operations and auto analyze. +* Resource control incurs additional scheduling overhead. Therefore, there might be a slight performance degradation when this feature is enabled. + +## See also + +* [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) +* [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) +* [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) +* [RESOURCE GROUP RFC](https://docs.google.com/document/d/1sV5EVv8Cdpc6aBCDihc2akpE0iuantPf/) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 213284cf53f51..d1915f2cebef5 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -2154,3 +2154,13 @@ To reduce write latency, TiKV periodically fetches and caches a batch of timesta + The maximum number of TSOs in a timestamp request. + In a default TSO physical time update interval (`50ms`), PD provides at most 262144 TSOs. When requested TSOs exceed this number, PD provides no more TSOs. This configuration item is used to avoid exhausting TSOs and the reverse impact of TSO exhaustion on other businesses. If you increase the value of this configuration item to improve high availability, you need to decrease the value of [`tso-update-physical-interval`](/pd-configuration-file.md#tso-update-physical-interval) at the same time to get enough TSOs. + Default value: `8192` + +## resource-control + +Configuration items related to resource control of the TiKV storage layer. + +### `enabled` New in v6.6.0 + ++ Controls whether to enable scheduling for user foreground read/write requests according to [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) of the corresponding resource groups. For information about TiDB resource groups and resource control, see [TiDB resource control](/tidb-resource-control.md). ++ Enabling this configuration item only works when [`tidb_enable_resource_control](/system-variables.md#tidb_enable_resource_control-new-in-v660) is enabled on TiDB. When this configuration item is enabled, TiKV will use the priority queue to schedule the queued read/write requests from foreground users. The scheduling priority of a request is inversely related to the amount of resources already consumed by the resource group that receives this request, and positively related to the quota of the corresponding resource group. ++ Default value: `false`, which means to disable scheduling based on the RU of the resource group. \ No newline at end of file