pitr: add ingest recorder to repair indexes (#41670)

[ti-chi-bot]

This is an automated cherry-pick of #41670

What problem does this PR solve?

Issue Number: close #41668 ref #38045

Problem Summary:
PiTR is not compatible with accelerated indexing

What is changed and how it works?

1. record the ingest indexes from history ddl
2. drop the indexes
3. re-add the indexes

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

upstream SQL

CREATE TABLE test.pairs(x int auto_increment primary key, y int DEFAULT RAND(), z int DEFAULT RAND());
INSERT INTO test.pairs VALUES (), ... (skip 500), ();
ALTER TABLE test.pairs ADD INDEX i1(y, z) USING HASH COMMENT "edelw;fe?fewfe\nefwe" INVISIBLE;
ALTER TABLE test.pairs ADD UNIQUE KEY u1(x, y) USING RTREE VISIBLE;
ALTER TABLE test.pairs ADD INDEX i2(y, (z + 1)) USING BTREE COMMENT "123";
ALTER TABLE test.pairs ADD UNIQUE KEY u2(x, (y+1)) USING HASH COMMENT "243";

CREATE TABLE test.pairs2(x int, y int, z varchar(20));
INSERT INTO test.pairs2 VALUES (1,1,"1"), ... (skip 500), (500,500,"500");
ALTER TABLE test.pairs2 ADD INDEX i1(y, z(10));
ALTER TABLE test.pairs2 ADD UNIQUE KEY u1(y, z(10), (y * 2)) USING RTREE VISIBLE;
ALTER TABLE test.pairs2 ADD PRIMARY KEY (x) USING HASH;

CREATE TABLE test.pairs3(id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, name CHAR(10), custinfo JSON);
INSERT INTO test.pairs3 VALUES (1, 'pingcap', '{"zipcode": [1,2]}');
ALTER TABLE test.pairs3 ADD INDEX zips2((CAST(custinfo->'$.zipcode' AS UNSIGNED ARRAY)));

CREATE TABLE test.pairs4(x int auto_increment primary key, y int DEFAULT RAND(), z int DEFAULT RAND());
INSERT INTO test.pairs4 VALUES (), ... (skip 500), ();
ALTER TABLE test.pairs4 ADD INDEX i1(y, z) USING HASH COMMENT "edelw;fe?fewfe\nefwe" INVISIBLE;
ALTER TABLE test.pairs4 ADD UNIQUE KEY u1(x, y) USING RTREE VISIBLE;
ALTER TABLE test.pairs4 ADD INDEX i2(y, (z + 1)) USING BTREE COMMENT "123";
ALTER TABLE test.pairs4 ADD UNIQUE KEY u2(x, (y+1)) USING HASH COMMENT "243";
ALTER TABLE test.pairs4 DROP INDEX i1;
ALTER TABLE test.pairs4 DROP INDEX u1;
ALTER TABLE test.pairs4 DROP INDEX i2;
ALTER TABLE test.pairs4 DROP INDEX u2;

CREATE TABLE test.pairs5(x int, y int, z varchar(20));
INSERT INTO test.pairs5 VALUES (1,1,"1"), ... (skip 500), (500,500,"500");
ALTER TABLE test.pairs5 ADD INDEX i1(y, z(10));
ALTER TABLE test.pairs5 ADD UNIQUE KEY u1(y, z(10), (y * 2)) USING RTREE VISIBLE;
ALTER TABLE test.pairs5 ADD PRIMARY KEY (x) USING HASH;
ALTER TABLE test.pairs5 DROP INDEX i1;
ALTER TABLE test.pairs5 DROP INDEX u1;
ALTER TABLE test.pairs5 DROP INDEX `PRIMARY`;

CREATE TABLE test.pairs6(id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, `nam``;e` int, `cust``;info` JSON);
INSERT INTO test.pairs6 VALUES (1, 1, '{"zipcode": [1,2]}');
ALTER TABLE test.pairs6 ADD INDEX zips2((CAST(`cust``;info`->'$.zipcode' AS UNSIGNED ARRAY)));
ALTER TABLE test.pairs6 ADD INDEX i1(`nam``;e`, (`nam``;e` * 2));
RENAME TABLE test.pairs6 TO test.pairs7;
ALTER TABLE test.pairs7 RENAME INDEX i1 to i2;

restore repair ingest index SQL from BR log

ALTER TABLE `test`.`pairs` DROP INDEX `i1`
ALTER TABLE `test`.`pairs` ADD INDEX `i1`(`y`,`z`) USING HASH COMMENT 'edelw;fe?fewfe\\nefwe' INVISIBLE
ALTER TABLE `test`.`pairs` DROP INDEX `u1`
ALTER TABLE `test`.`pairs` ADD UNIQUE KEY `u1`(`x`,`y`) USING RTREE VISIBLE
ALTER TABLE `test`.`pairs` DROP INDEX `i2`
ALTER TABLE `test`.`pairs` ADD INDEX `i2`(`y`,(`z` + 1)) USING BTREE COMMENT '123' VISIBLE
ALTER TABLE `test`.`pairs` DROP INDEX `u2`
ALTER TABLE `test`.`pairs` ADD UNIQUE KEY `u2`(`x`,(`y` + 1)) USING HASH COMMENT '243' VISIBLE
ALTER TABLE `test`.`pairs2` DROP INDEX `PRIMARY`
ALTER TABLE `test`.`pairs2` ADD PRIMARY KEY (`x`) NONCLUSTERED USING HASH VISIBLE
ALTER TABLE `test`.`pairs2` DROP INDEX `i1`
ALTER TABLE `test`.`pairs2` ADD INDEX `i1`(`y`,`z`(10)) USING BTREE VISIBLE
ALTER TABLE `test`.`pairs2` DROP INDEX `u1`
ALTER TABLE `test`.`pairs2` ADD UNIQUE KEY `u1`(`y`,`z`(10),(`y` * 2)) USING RTREE VISIBLE
ALTER TABLE `test`.`pairs3` DROP INDEX `zips2`
ALTER TABLE `test`.`pairs3` ADD INDEX `zips2`((cast(json_extract(`custinfo`, _utf8'$.zipcode') as unsigned array))) USING BTREE VISIBLE
ALTER TABLE `test`.`pairs7` DROP INDEX `zips2`
ALTER TABLE `test`.`pairs7` ADD INDEX `zips2`((cast(json_extract(`cust``;info`, _utf8'$.zipcode') as unsigned array))) USING BTREE VISIBLE
ALTER TABLE `test`.`pairs7` DROP INDEX `i2`
ALTER TABLE `test`.`pairs7` ADD INDEX `i2`(`nam``;e`,(`nam``;e` * 2)) USING BTREE VISIBLE

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[ti-chi-bot]

@ti-chi-bot: This cherry pick PR is for a release branch and has not yet been approved by release team.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick, it must first be approved by the collaborators.

AFTER it has been approved by collaborators, please ping the release team in a comment to request a cherry pick review.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ti-chi-bot]

@ti-chi-bot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/unit-test	13adb2c	link	true	/test unit-test
idc-jenkins-ci-tidb/check_dev	13adb2c	link	true	/test check-dev
idc-jenkins-ci-tidb/build	13adb2c	link	true	/test build
idc-jenkins-ci-tidb/check_dev_2	13adb2c	link	true	/test check-dev2
idc-jenkins-ci-tidb/mysql-test	13adb2c	link	true	/test mysql-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.