Skip to content
This repository has been archived by the owner on Aug 21, 2023. It is now read-only.

dump: always split TiDB v3.* tables through tidb rowid to save TiDB's memory (#301) #306

Merged

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #301

What problem does this PR solve?

fix #295, an extension of #273
In TiDB v3.x:

  1. table INFORMATION_SCHEMA.TIKV_REGION_STATUS doesn't have table_name/table_schema.
  2. doesn't have SHOW TABLE REGIONS statement.
  3. table INFORMATION_SCHEMA.PARTITIONS doesn't actually have data.
  4. doesn't have tidb_decode_key function.
mysql> show create table information_schema.TIKV_REGION_STATUS;
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table              | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TIKV_REGION_STATUS | CREATE TABLE `TIKV_REGION_STATUS` (
  `REGION_ID` bigint(21) unsigned DEFAULT NULL,
  `START_KEY` text DEFAULT NULL,
  `END_KEY` text DEFAULT NULL,
  `EPOCH_CONF_VER` bigint(21) unsigned DEFAULT NULL,
  `EPOCH_VERSION` bigint(21) unsigned DEFAULT NULL,
  `WRITTEN_BYTES` bigint(21) unsigned DEFAULT NULL,
  `READ_BYTES` bigint(21) unsigned DEFAULT NULL,
  `APPROXIMATE_SIZE` bigint(21) unsigned DEFAULT NULL,
  `APPROXIMATE_KEYS` bigint(21) unsigned DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

What is changed and how it works?

Split chunks by PKIsHandle for TiDB v3.*. Make sure TiDB won't OOM through splitting tables by rowids.
We do some staff by our own to solve the problem that tidb lacks some info.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Test dump with v3.0.0. The result is looks expected.

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Always split TiDB v3.* tables through tidb rowid to save TiDB's memory.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@lichunzhu
Copy link
Contributor

Get blocked by pingcap/tidb#22756 is not cherry-picked to release-4.0.
If not merged, dumpling will have to consider using tidb 5.0 dependency or block this PR until next version.

Copy link
Contributor

@lichunzhu lichunzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member Author

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lichunzhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer approved (LGTM1) label Jul 15, 2021
@lichunzhu
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member Author

This pull request has been accepted and is ready to merge.

Commit hash: 09eafa2

@ti-chi-bot ti-chi-bot merged commit d28f469 into pingcap:release-4.0 Jul 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants