bootstrap: aggregate the unused index across multiple nodes

[YangKeao]

What problem does this PR solve?

Issue Number: close #51269

What changed and how does it work?

Aggregate the unused index across multiple nodes. The original SQL didn't aggregate them 🤦 , so an index will appear multiple times. A silly bug.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Create a cluster with two tidb nodes

tiup playground --db 2 --db.binpath ./bin/tidb-server

Run the following SQLs

create table t (id_1 int, id_2 int, unique key idx_1(id_1), unique key idx_2(id_2));
insert into t with recursive seq as (select 1 as id union all select id + 1 from seq where seq.id < 100) select id as id_1, id as id_2 from seq;
analyze table t;
select * from t use index(idx_1) where id_1 > 60;
select * from sys.schema_unused_indexes;

It'll give:

mysql> select * from sys.schema_unused_indexes;
+---------------+-------------+------------+
| object_schema | object_name | index_name |
+---------------+-------------+------------+
| test          | t           | idx_2      |
+---------------+-------------+------------+
1 row in set (0.00 sec)

The plan is like:

+----------------------------------+----------+-----------+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                               | estRows  | task      | access object                  | operator info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+----------------------------------+----------+-----------+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection_9                     | 1846.12  | root      |                                | information_schema.cluster_tidb_index_usage.table_schema, information_schema.cluster_tidb_index_usage.table_name, information_schema.cluster_tidb_index_usage.index_name                                                                                                                                                                                                                                                                                                                                           |
| └─Selection_10                   | 1846.12  | root      |                                | isnull(Column#17)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|   └─HashAgg_11                   | 2307.64  | root      |                                | group by:Column#25, Column#26, Column#27, funcs:sum(Column#21)->Column#17, funcs:firstrow(Column#22)->information_schema.cluster_tidb_index_usage.table_schema, funcs:firstrow(Column#23)->information_schema.cluster_tidb_index_usage.table_name, funcs:firstrow(Column#24)->information_schema.cluster_tidb_index_usage.index_name                                                                                                                                                                               |
|     └─Projection_15              | 2884.56  | root      |                                | cast(information_schema.cluster_tidb_index_usage.last_access_time, double BINARY)->Column#21, information_schema.cluster_tidb_index_usage.table_schema->Column#22, information_schema.cluster_tidb_index_usage.table_name->Column#23, information_schema.cluster_tidb_index_usage.index_name->Column#24, information_schema.cluster_tidb_index_usage.table_schema->Column#25, information_schema.cluster_tidb_index_usage.table_name->Column#26, information_schema.cluster_tidb_index_usage.index_name->Column#27 |
|       └─TableReader_14           | 2884.56  | root      |                                | data:Selection_13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|         └─Selection_13           | 2884.56  | cop[tidb] |                                | ne(information_schema.cluster_tidb_index_usage.index_name, "PRIMARY"), not(in(information_schema.cluster_tidb_index_usage.table_schema, "sys", "mysql", "INFORMATION_SCHEMA", "PERFORMANCE_SCHEMA"))                                                                                                                                                                                                                                                                                                               |
|           └─TableFullScan_12     | 10000.00 | cop[tidb] | table:CLUSTER_TIDB_INDEX_USAGE | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+----------------------------------+----------+-----------+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
7 rows in set (0.00 sec)

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[codecov]

Codecov Report

Merging #51270 (587a436) into master (a84d57c) will decrease coverage by 15.9272%.
Report is 20 commits behind head on master.
The diff coverage is n/a.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #51270         +/-   ##
=================================================
- Coverage   70.6538%   54.7266%   -15.9272%     
=================================================
  Files          1457       1567        +110     
  Lines        433839     601681     +167842     
=================================================
+ Hits         306524     329280      +22756     
- Misses       108056     249386     +141330     
- Partials      19259      23015       +3756     

Flag	Coverage Δ
integration	36.7885% <ø> (?)
unit	70.3269% <ø> (-0.1065%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	53.9957% <ø> (ø)
parser	∅ <ø> (∅)
br	51.4250% <ø> (+5.5804%)	⬆️

[YangKeao]

/retest

[bb7133]

LGTM

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-02-26 10:09:35.857067208 +0000 UTC m=+870264.604690319: ☑️ agreed by qw4990.
• 2024-02-27 02:00:49.686636332 +0000 UTC m=+927338.434259438: ☑️ agreed by bb7133.

[easonn7]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bb7133, easonn7, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/session/OWNERS [easonn7]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment