Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add system variable tidb_opt_range_max_size #11239

Merged
merged 26 commits into from
Nov 7, 2022
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
3aab818
add system variable tidb_opt_range_mem_quota
xuyifangreeneyes Sep 7, 2022
74501d1
add example
xuyifangreeneyes Sep 8, 2022
affc6cf
fix code format
shichun-0415 Sep 14, 2022
ef3622f
fix more code format
shichun-0415 Sep 14, 2022
d887f01
Update system-variables.md
xuyifangreeneyes Sep 15, 2022
ebf47f5
Update system-variables.md
xuyifangreeneyes Oct 10, 2022
538f26c
Update system-variables.md
xuyifangreeneyes Oct 10, 2022
a5044e3
Update system-variables.md
xuyifangreeneyes Oct 10, 2022
e4ac153
Update system-variables.md
xuyifangreeneyes Oct 10, 2022
3b57982
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
a573b98
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
0f88dd1
upd example
xuyifangreeneyes Oct 18, 2022
7b56c55
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
01c6537
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
d62e07b
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
ef666e5
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
c446a49
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
662fc38
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
d22a611
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
c5a4a3f
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
52bacfc
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
efb3571
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
42ed6ba
Update system-variables.md
xuyifangreeneyes Oct 18, 2022
e5cab0b
Update system-variables.md
TomShawn Oct 18, 2022
0261d76
add toggle display
TomShawn Oct 18, 2022
35cf09a
Update system-variables.md
TomShawn Oct 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 130 additions & 0 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -1807,6 +1807,136 @@ explain select * from t where age=5;
- 默认值:`OFF`
- 指定是否允许优化器将 `Projection` 算子下推到 TiKV 或者 TiFlash。

### `tidb_opt_range_max_size` <span class="version-mark">从 v6.4.0 版本开始引入</span>

- 作用域:SESSION | GLOBAL
- 是否持久化到集群:是
- 默认值:`67108864` (64 MiB)
- 取值范围:`[0, 9223372036854775807]`
- 单位:字节
- 该变量用于指定优化器构造扫描范围的内存用量上限。当该变量为 `0` 时,表示对扫描范围没有内存限制。如果构造精确的扫描范围会超出内存用量限制,优化器会使用更宽松的扫描范围(例如 `[[NULL,+inf]]`)。如果执行计划中未使用精确的扫描范围,可以调大该变量的值让优化器构造精确的扫描范围。

该变量的使用示例如下:

<details>
<summary><code>tidb_opt_range_max_size</code> 使用示例</summary>

查看该变量的默认值,即优化器构造扫描范围最多使用 64 MiB 内存。

xuyifangreeneyes marked this conversation as resolved.
Show resolved Hide resolved
```sql
SELECT @@tidb_opt_range_max_size;
```

```sql
+----------------------------+
| @@tidb_opt_range_max_size |
+----------------------------+
| 67108864 |
+----------------------------+
1 row in set (0.01 sec)
```

```sql
EXPLAIN SELECT * FROM t use index (idx) WHERE a IN (10,20,30) AND b IN (40,50,60);
TomShawn marked this conversation as resolved.
Show resolved Hide resolved
```

在 64 MiB 的内存最大限制约束下,优化器构造出精确的扫描范围 `[10 40,10 40], [10 50,10 50], [10 60,10 60], [20 40,20 40], [20 50,20 50], [20 60,20 60], [30 40,30 40], [30 50,30 50], [30 60,30 60]`,见如下执行计划返回结果。

```sql
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | estRows | task | access object | operator info |
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| IndexLookUp_7 | 0.90 | root | | |
| ├─IndexRangeScan_5(Build) | 0.90 | cop[tikv] | table:t, index:idx(a, b) | range:[10 40,10 40], [10 50,10 50], [10 60,10 60], [20 40,20 40], [20 50,20 50], [20 60,20 60], [30 40,30 40], [30 50,30 50], [30 60,30 60], keep order:false, stats:pseudo |
| └─TableRowIDScan_6(Probe) | 0.90 | cop[tikv] | table:t | keep order:false, stats:pseudo |
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
TomShawn marked this conversation as resolved.
Show resolved Hide resolved
3 rows in set (0.00 sec)
```

xuyifangreeneyes marked this conversation as resolved.
Show resolved Hide resolved
现将优化器构造扫描范围的内存用量上限设为 1500 字节。

```sql
SET @@tidb_opt_range_max_size = 1500;
```

```sql
Query OK, 0 rows affected (0.00 sec)
```

```sql
EXPLAIN SELECT * FROM t USE INDEX (idx) WHERE a IN (10,20,30) AND b IN (40,50,60);
```

在 1500 字节内存的最大限制约束下,优化器构造出了更宽松的扫描范围 `[10,10], [20,20], [30,30]`,并用 warning 提示用户构造精确的扫描范围所需的内存用量超出了 `tidb_opt_range_max_size` 的限制。

```sql
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------+
| id | estRows | task | access object | operator info |
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------+
| IndexLookUp_8 | 0.09 | root | | |
| ├─Selection_7(Build) | 0.09 | cop[tikv] | | in(test.t.b, 40, 50, 60) |
| │ └─IndexRangeScan_5 | 30.00 | cop[tikv] | table:t, index:idx(a, b) | range:[10,10], [20,20], [30,30], keep order:false, stats:pseudo |
| └─TableRowIDScan_6(Probe) | 0.09 | cop[tikv] | table:t | keep order:false, stats:pseudo |
+-------------------------------+---------+-----------+--------------------------+-----------------------------------------------------------------+
4 rows in set, 1 warning (0.00 sec)
```

TomShawn marked this conversation as resolved.
Show resolved Hide resolved
```sql
SHOW WARNINGS;
```

```sql
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | Memory capacity of 1500 bytes for 'tidb_opt_range_max_size' exceeded when building ranges. Less accurate ranges such as full range are chosen |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
```

xuyifangreeneyes marked this conversation as resolved.
Show resolved Hide resolved
再将优化器构造扫描范围的内存用量上限设为 100 字节。

```sql
set @@tidb_opt_range_max_size = 100;
```

```sql
Query OK, 0 rows affected (0.00 sec)
```

```sql
EXPLAIN SELECT * FROM t USE INDEX (idx) WHERE a IN (10,20,30) AND b IN (40,50,60);
```

在 100 字节的内存最大限制约束下,优化器选择了 `IndexFullScan`,并用 warning 提示用户构造精确的扫描范围所需的内存超出了 `tidb_opt_range_max_size` 的限制。

```sql
+-------------------------------+----------+-----------+--------------------------+----------------------------------------------------+
| id | estRows | task | access object | operator info |
+-------------------------------+----------+-----------+--------------------------+----------------------------------------------------+
| IndexLookUp_8 | 8000.00 | root | | |
| ├─Selection_7(Build) | 8000.00 | cop[tikv] | | in(test.t.a, 10, 20, 30), in(test.t.b, 40, 50, 60) |
| │ └─IndexFullScan_5 | 10000.00 | cop[tikv] | table:t, index:idx(a, b) | keep order:false, stats:pseudo |
| └─TableRowIDScan_6(Probe) | 8000.00 | cop[tikv] | table:t | keep order:false, stats:pseudo |
+-------------------------------+----------+-----------+--------------------------+----------------------------------------------------+
4 rows in set, 1 warning (0.00 sec)
```

```sql
SHOW WARNINGS;
```

```sql
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | Memory capacity of 100 bytes for 'tidb_opt_range_max_size' exceeded when building ranges. Less accurate ranges such as full range are chosen |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
```

</details>

### `tidb_opt_scan_factor`

- 作用域:SESSION | GLOBAL
Expand Down