Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: refine placement #8706

Merged
merged 26 commits into from
Mar 23, 2022
Merged

*: refine placement #8706

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
ce19e0d
*: refine placement
xhebox Mar 18, 2022
9aa7e26
*: reword, update compatibility
xhebox Mar 18, 2022
b7e4363
fix wrong & dead link in relnote
xhebox Mar 18, 2022
689f98f
address comments
xhebox Mar 18, 2022
7fad8c0
Update placement-rules-in-sql.md
xhebox Mar 18, 2022
4a6d8e0
Update placement-rules-in-sql.md
xhebox Mar 18, 2022
185fc66
Update information-schema/information-schema-placement-policies.md
xhebox Mar 18, 2022
ef58220
Update placement-rules-in-sql.md
xhebox Mar 18, 2022
9512b72
Update information-schema/information-schema-placement-policies.md
xhebox Mar 21, 2022
0b58dc2
reword
xhebox Mar 21, 2022
938988f
Update placement-rules-in-sql.md
xhebox Mar 21, 2022
7730315
reword
xhebox Mar 21, 2022
2e05131
Update information-schema/information-schema-placement-policies.md
xhebox Mar 21, 2022
00dc9af
Update information-schema/information-schema-placement-policies.md
xhebox Mar 22, 2022
13aeb2d
Update placement-rules-in-sql.md
xhebox Mar 22, 2022
738f689
Update placement-rules-in-sql.md
xhebox Mar 22, 2022
d68e2c7
variable update
xhebox Mar 22, 2022
c3e3bbc
Update system-variables.md
xhebox Mar 22, 2022
0b3452d
compatibility refine
xhebox Mar 22, 2022
7abf2fd
Update br/backup-and-restore-tool.md
xhebox Mar 22, 2022
75f4c06
Update placement-rules-in-sql.md
xhebox Mar 22, 2022
7b59040
Update br/backup-and-restore-tool.md
xhebox Mar 22, 2022
e5491f0
reword
xhebox Mar 22, 2022
2db3ae6
Update placement-rules-in-sql.md
xhebox Mar 23, 2022
649add2
Update information-schema/information-schema.md
xhebox Mar 23, 2022
56fe1a8
reword
xhebox Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion TOC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,7 @@
- [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md)
- [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md)
- [`PARTITIONS`](/information-schema/information-schema-partitions.md)
- [`PLACEMENT_RULES`](/information-schema/information-schema-placement-rules.md)
- [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md)
- [`PROCESSLIST`](/information-schema/information-schema-processlist.md)
- [`REFERENTIAL_CONSTRAINTS`](/information-schema/information-schema-referential-constraints.md)
- [`SCHEMATA`](/information-schema/information-schema-schemata.md)
Expand Down
6 changes: 4 additions & 2 deletions br/backup-and-restore-tool.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ SST 文件以 `storeID_regionID_regionEpoch_keyHash_cf` 的格式命名。格式

### 兼容性

BR 和 TiDB 集群的兼容性问题分为以下两方面：
BR 和 TiDB 集群的兼容性问题有以下几方面：

+ BR 部分版本和 TiDB 集群的接口不兼容

BR 在 v5.4.0 之前不支持恢复 `charset=GBK` 的表。并且，任何版本的 BR 都不支持恢复 `charset=GBK` 的表到 5.4.0 之前的 TiDB 集群。
+ BR 在 v5.4.0 之前不支持恢复 `charset=GBK` 的表。并且，任何版本的 BR 都不支持恢复 `charset=GBK` 的表到 5.4.0 之前的 TiDB 集群。

+ BR 在 v6.0.0 之前不支持[放置规则](/placement-rules-in-sql.md)。 BR v6.0.0 及以上版本开始支持并提供了命令行选项 `--with-tidb-placement-mode=strict/ignore` 来控制放置规则的导入模式。 默认值为 `strict` 代表导入并检查放置规则，否则当其设置为 `ignore` 时忽略所有的放置规则。

+ 某些功能在开启或关闭状态下，会导致 KV 格式发生变化，因此备份和恢复期间如果没有统一开启或关闭，就会带来不兼容的问题

Expand Down
73 changes: 73 additions & 0 deletions information-schema/information-schema-placement-policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: PLACEMENT_POLICIES
summary: 了解 information_schema 表 `PLACEMENT_POLICIES`。
aliases: ['/zh/tidb/dev/information-schema-placement-rules']
---
xhebox marked this conversation as resolved.
Show resolved Hide resolved

# PLACEMENT_POLICIES

`PLACEMENT_POLICIES` 表展示所有放置策略 (placement policy) 的信息，见 [Placement Rules in SQL](/placement-rules-in-sql.md)。

{{< copyable "sql" >}}

```sql
USE information_schema;
DESC placement_policies;
```

```sql
+----------------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+---------------+------+-----+---------+-------+
| POLICY_ID | bigint(64) | NO | | <null> | |
| CATALOG_NAME | varchar(512) | NO | | <null> | |
| POLICY_NAME | varchar(64) | NO | | <null> | |
| PRIMARY_REGION | varchar(1024) | YES | | <null> | |
| REGIONS | varchar(1024) | YES | | <null> | |
| CONSTRAINTS | varchar(1024) | YES | | <null> | |
| LEADER_CONSTRAINTS | varchar(1024) | YES | | <null> | |
| FOLLOWER_CONSTRAINTS | varchar(1024) | YES | | <null> | |
| LEARNER_CONSTRAINTS | varchar(1024) | YES | | <null> | |
| SCHEDULE | varchar(20) | YES | | <null> | |
| FOLLOWERS | bigint(64) | YES | | <null> | |
| LEARNERS | bigint(64) | YES | | <null> | |
+----------------------+---------------+------+-----+---------+-------+
12 rows in set (0.00 sec)
```

## 示例

`PLACEMENT_POLICIES` 表只展示放置策略 (placement policy) 的信息。如需查看所有信息的规范版本（放置策略，以及绑定放置策略的对象），请改用 `SHOW PLACEMENT` 语句：

{{< copyable "sql" >}}

```sql
CREATE TABLE t1 (a INT);
CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1";
CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1;
SHOW PLACEMENT; -- 显示所有信息，包含 t3。
SELECT * FROM information_schema.placement_policies; -- 只显示放置策略，不包含 t3。
```

```sql
Query OK, 0 rows affected (0.09 sec)

Query OK, 0 rows affected (0.11 sec)

Query OK, 0 rows affected (0.08 sec)

+---------------+------------------------------------------------+------------------+
| Target | Placement | Scheduling_State |
+---------------+------------------------------------------------+------------------+
| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | NULL |
| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | SCHEDULED |
+---------------+------------------------------------------------+------------------+
2 rows in set (0.00 sec)

+-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+
| POLICY_ID | CATALOG_NAME | POLICY_NAME | PRIMARY_REGION | REGIONS | CONSTRAINTS | LEADER_CONSTRAINTS | FOLLOWER_CONSTRAINTS | LEARNER_CONSTRAINTS | SCHEDULE | FOLLOWERS | LEARNERS |
+-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+
| 1 | def | p1 | us-east-1 | us-east-1 | | | | | | 2 | 0 |
+-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+
1 rows in set (0.00 sec)
```
77 changes: 0 additions & 77 deletions information-schema/information-schema-placement-rules.md
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion information-schema/information-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Information Schema 提供了一种查看系统元数据的 ANSI 标准方法。
| [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | 从 Prometheus 获取的指标的摘要。 |
| `METRICS_SUMMARY_BY_LABEL` | 参见 `METRICS_SUMMARY` 表。 |
| [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | 为 `METRICS_SCHEMA` 中的表提供 PromQL 定义。 |
| [`PLACEMENT_RULES`](/information-schema/information-schema-placement-rules.md) | 对于所有被显式分配了放置规则的对象，该扩展表提供与这些对象相关的所有信息。 |
| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | 提供所有放置策略的定义信息。 |
| [`SEQUENCES`](/information-schema/information-schema-sequences.md) | 描述了基于 MariaDB 实现的 TiDB 序列。 |
| [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | 提供当前 TiDB 服务器上慢查询的信息。 |
| [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | 类似于 MySQL 中的 performance_schema 语句摘要。 |
Expand Down
68 changes: 57 additions & 11 deletions placement-rules-in-sql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Placement Rules in SQL 特性用于通过 SQL 接口配置数据在 TiKV 集群

## 指定放置规则

指定放置规则，首先需要创建放置策略 (placement policy)。
指定放置规则，首先需要通过 [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md) 语句创建**放置策略 (placement policy)**。

```sql
CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1";
Expand All @@ -34,16 +34,28 @@ CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east-1" REGIONS="us
然后可以使用 `CREATE TABLE` 或者 `ALTER TABLE` 将规则绑定至表或分区表，这样就在表或分区上指定了放置规则：

```sql
CREATE TABLE t1 (a INT) PLACEMENT POLICY myplacementpolicy;
CREATE TABLE t1 (a INT) PLACEMENT POLICY=myplacementpolicy;
CREATE TABLE t2 (a INT);
ALTER TABLE t2 PLACEMENT POLICY myplacementpolicy;
ALTER TABLE t2 PLACEMENT POLICY=myplacementpolicy;
```

`PLACEMENT POLICY` 为全局作用域，不与任何数据库表结构相关联。因此，通过 `CREATE TABLE` 指定放置规则时，无需任何额外的权限。

要修改放置策略，你可以使用 [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) 语句。修改将传播到所有绑定此放置策略的对象。

```sql
ALTER PLACEMENT POLICY myplacementpolicy FOLLOWERS=5;
```

要删除没有绑定任何分区或表的放置策略，你可以使用 [`DROP PLACEMENT POLICY`](/sql-statements/sql-statement-drop-placement-policy.md)：

```sql
DROP PLACEMENT POLICY myplacementpolicy;
```

## 查看放置规则

如果一张表绑定了放置规则，你可以用 `SHOW CREATE TABLE` 来查看。还可以用 `SHOW CREATE PLACEMENT POLICY` 来查看已经创建的规则。
如果一张表绑定了放置规则，你可以用 [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) 来查看。还可以用 [`SHOW CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-show-create-placement-policy.md) 来查看已经创建的放置策略。

```sql
tidb> SHOW CREATE TABLE t1\G
Expand All @@ -60,6 +72,26 @@ Create Policy: CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east
1 row in set (0.00 sec)
```

你也可以用 [`INFORMATION_SCHEMA.PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) 系统表查看所有放置策略的定义。

```sql
tidb> select * from information_schema.placement_policies\G
***************************[ 1. row ]***************************
POLICY_ID | 1
CATALOG_NAME | def
POLICY_NAME | p1
PRIMARY_REGION | us-east-1
REGIONS | us-east-1,us-west-1
CONSTRAINTS |
LEADER_CONSTRAINTS |
FOLLOWER_CONSTRAINTS |
LEARNER_CONSTRAINTS |
SCHEDULE |
FOLLOWERS | 4
LEARNERS | 0
1 row in set
```

`information_schema.tables` 表和 `information_schema.partitions` 表也有一列 `tidb_placement_policy_name`，用于展示所有绑定了放置规则的对象：

```sql
Expand Down Expand Up @@ -98,8 +130,11 @@ SELECT * FROM information_schema.partitions WHERE tidb_placement_policy_name IS

| 选项名 | 描述 |
|----------------------------|------------------------------------------------------------------------------------------------|
| `CONSTRAINTS` | 适用于所有角色 (role) 的约束列表。例如，`CONSTRAINTS="[+disk=ssd]`。 |
| `FOLLOWER_CONSTRAINTS` | 仅适用于 follower 的约束列表。 |
| `CONSTRAINTS` | 适用于所有角色 (role) 的约束列表。例如，`CONSTRAINTS="[+disk=ssd]`。 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个 ] 后面少了 "?

| `LEADER_CONSTRAINTS` | 仅适用于 leader 的约束列表。 |
| `FOLLOWER_CONSTRAINTS` | 仅适用于 follower 的约束列表。 |
| `LEARNER_CONSTRAINTS` | 仅适用于 learner 的约束列表。 |
| `LEARNERS` | 指定 learner 的数量。 |

## 示例

Expand Down Expand Up @@ -134,19 +169,23 @@ CREATE TABLE t1 (a INT) PLACEMENT POLICY=eastandwest;
除了给表绑定放置策略之外，你还可以给表分区绑定放置策略。示例如下：

```sql
CREATE PLACEMENT POLICY p1 FOLLOWERS=5;
CREATE PLACEMENT POLICY europe PRIMARY_REGION="eu-central-1" REGIONS="eu-central-1,eu-west-1";
CREATE PLACEMENT POLICY northamerica PRIMARY_REGION="us-east-1" REGIONS="us-east-1";

SET tidb_enable_list_partition = 1;
CREATE TABLE t1 (
country VARCHAR(10) NOT NULL,
userdata VARCHAR(100) NOT NULL
) PARTITION BY LIST COLUMNS (country) (
) PLACEMENT POLICY=p1 PARTITION BY LIST COLUMNS (country) (
PARTITION pEurope VALUES IN ('DE', 'FR', 'GB') PLACEMENT POLICY=europe,
PARTITION pNorthAmerica VALUES IN ('US', 'CA', 'MX') PLACEMENT POLICY=northamerica
PARTITION pAsia VALUES IN ('CN', 'KR', 'JP')
);
```

如果分区没有绑定任何放置策略，分区将尝试继承表上可能存在的策略。比如，`pEurope` 分区将会应用 `europe` 策略，而 `pAsia` 分区将会应用表 `t1` 的放置策略 `p1`。如果 `t1` 没有绑定任何策略，`pAsia` 就不会应用任何策略。

### 为数据库配置默认的放置规则

你可以为某个数据库指定默认的放置策略，类似于为数据库设置默认字符集或排序规则。如果没有指定其他选项，就会使用数据库上指定的配置。示例如下：
Expand All @@ -173,7 +212,7 @@ CREATE TABLE t4 (a INT); -- 创建表 t4，默认的放置策略 p3 生效。
ALTER PLACEMENT POLICY p3 FOLLOWERS=3; -- 绑定策略 p3 的表，也就是 t4，会采用 FOLLOWERS=3。
```

用户可以通过使用 [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) 改变放置策略，从而改变已从数据库继承放置规则的表。
注意分区与表之间的继承和这里的继承不同。改变表的放置策略，也会让分区应用新的策略。但是只有建表时没有指定放置策略的时候，表才会从数据库继承放置策略，且之后再改变数据库也不影响已经继承的表。

### 高级放置选项

Expand Down Expand Up @@ -207,12 +246,19 @@ PARTITION BY RANGE( YEAR(purchased) ) (
>
> 字典和列表格式都基于YAML解析，但 YAML 语法有些时候不能被正常解析。例如 YAML 会把 "{+disk=ssd:1,+disk=hdd:2}" 错误地解析成 '{"+disk=ssd:1": null, "+disk=hdd:1": null}'，不符合预期。但 "{+disk=ssd: 1,+disk=hdd: 1}" 能被正确解析成 '{"+disk=ssd": 1, "+disk=hdd": 1}'。

## 工具兼容性

| 工具名称 | 最低兼容版本 | 说明 |
| --- | --- | --- |
| Backup & Restore (BR) | 6.0 | 支持放置规则的导入与导出，见 [BR 兼容性](/br/backup-and-restore-tool.md#兼容性] |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

br 兼容性链接后面缺 )

| TiDB Lightning | 暂时不兼容 | |
| TiCDC | 6.0 | 忽略放置规则，不同步规则到下游集群 |
| TiDB Binlog | 6.0 | 忽略放置规则，不同步规则到下游集群 |

## 使用限制

目前已知 Placement Rules in SQL 实验特性存在以下限制：
目前已知 Placement Rules in SQL 特性存在以下限制：

* Dumpling 不支持导出放置策略，见 [issue #29371](https://github.com/pingcap/tidb/issues/29371)。
* TiDB 生态工具，包括 Backup & Restore (BR)、TiCDC、TiDB Lightning 和 TiDB Data Migration (DM)，不支持放置规则。
lcwangchao marked this conversation as resolved.
Show resolved Hide resolved
* 临时表不支持放置规则。
* 设置 `PRIMARY_REGION` 和 `REGIONS` 时允许存在语法糖。但在未来版本中，我们计划为 `PRIMARY_RACK`、`PRIMARY_ZONE` 和 `PRIMARY_HOST` 添加变体支持，见 [issue #18030](https://github.com/pingcap/tidb/issues/18030)。
* 不能通过放置规则语法配置 TiFlash 副本。
Expand Down
2 changes: 1 addition & 1 deletion releases/release-5.3.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ TiDB 版本：5.3.0
- 把热点数据的 leader 放到高性能的 TiKV 实例上。
- 将冷数据分离到不同的存储中以提高可用性。

[用户文档](/information-schema/information-schema-placement-rules.md)，[#18030](https://github.com/pingcap/tidb/issues/18030)
[用户文档](/placement-rules-in-sql.md)，[#18030](https://github.com/pingcap/tidb/issues/18030)

- **临时表**

Expand Down
Loading