add tikv compaction and flow control setting (#21086)

[ti-chi-bot]

This is an automated cherry-pick of #21086

First-time contributors' checklist

• I've signed Contributor License Agreement that's required for repo owners to accept my contribution.

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

• master (the latest development version)
• v9.0 (TiDB 9.0 versions)
• v8.5 (TiDB 8.5 versions)
• v8.1 (TiDB 8.1 versions)
• v7.5 (TiDB 7.5 versions)
• v7.1 (TiDB 7.1 versions)
• v6.5 (TiDB 6.5 versions)
• v6.1 (TiDB 6.1 versions)
• v5.4 (TiDB 5.4 versions)

What is the related PR or file link(s)?

• This PR is translated from:
• Other reference link(s):

Do your changes match any of the following descriptions?

• Delete files
• Change aliases
• Need modification after applied to another branch
• Might cause conflicts after applied to another branch

[gemini-code-assist]

Summary of Changes

Hello @ti-chi-bot, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands and refines the TiDB performance tuning documentation, specifically focusing on TiKV's compaction and flow control mechanisms. The changes introduce new configurable parameters, update existing ones, and provide detailed recommendations to help users optimize TiKV performance and stability, particularly for write-intensive database workloads.

Highlights

• Enhanced TiKV Performance Tuning Documentation: I've added a new general recommendation to the performance tuning guide, emphasizing the importance of fine-tuning TiKV's compaction and flow control configurations for optimal and stable performance, especially under write-intensive workloads.
• New TiKV Configuration Parameters Documented: I've documented several new TiKV configuration parameters, including those for pessimistic transactions (in-memory-peer-size-limit, in-memory-instance-size-limit), RocksDB (max-manifest-file-size), and storage scheduler (scheduler-pending-write-threshold). These additions provide users with more granular control over TiKV's behavior.
• Updated TiKV Flow Control Settings: The storage.flow-control.l0-files-threshold example value has been adjusted from 60 to 50. Additionally, new flow control parameters like storage.flow-control.soft-pending-compaction-bytes-limit and specific RocksDB column family settings (level0-slowdown-writes-trigger, soft-pending-compaction-bytes-limit for writecf, defaultcf, lockcf) have been introduced and explained.
• Recommended Configurations for Write-Intensive Workloads: A new section provides specific recommendations for adjusting compaction and flow control parameters, such as rocksdb.rate-bytes-per-sec and storage.flow-control limits, tailored for TiKV deployments on instances with high CPU, memory, and disk throughput, to optimize performance and stability under heavy write loads.
• Example Configuration Update: An example SET GLOBAL statement in the TiDB performance tuning section has been updated. The tidb_enable_inl_join_inner_multi_pattern variable has been replaced with tidb_opt_limit_push_down_threshold in both the example code block and its corresponding description table.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds documentation for TiKV compaction and flow control settings. The changes are mostly in tidb-performance-tuning-config.md.

My review has identified several critical issues related to merge conflicts that must be resolved. There are merge conflict markers left in the file which will break rendering and present incorrect information.

I've also included several suggestions to improve the documentation's clarity, grammar, and formatting, in line with the repository's style guide. These include fixing an incorrect link, improving sentence structure for better readability, and ensuring consistent formatting for error messages.

[hfxsd]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hfxsd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hfxsd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment