Enhancement Placement Rule In SQL User Journey

[nolouch]

Enhancement

Placement Rule functionality, which allows users to define a set of rules to control the placement of data across the cluster. These rules can be specified using SQL statements to ensure that data is distributed in a way that meets specific requirements, such as balancing data across nodes or ensuring that replicas are located in different data centers.

While the Placement Rule functionality is powerful, users have reported several issues with its usability and some bug issues that arise due to the diverse user scenarios, which have not been addressed or recommended. we should add more test cases add do some improvements, including:

• Improved syntax and documentation
• Limited constraint
• More diagnosable

Docs

• Enhance the Document about recommended usage (On-Prem Only) *: refine placement rule in sql docs docs-cn#15190
  • Cluster Level Availability Scenario
  • Data Consolidation (Physical Isolation) Scenario
  • Limitations and Maintenance
    • limit the rules number, limit the apps in isolation

Function Enhancement

Core Function

• System level (unify the entry of rule to tidb) Add Support for "Built In Placement Policies" #29677
  • *: supports setting global/meta range placement policy #47561
• Optimize the generated rule
  • reduce rule numbers if not set the role in the rule SQL #45395
    • placement: merge the rules if the constraints are the same #46529
• Improved syntax
  • Placement rule supports set follower strict #46304
    • placement: supports follower role with evict-leader hint #47307
  • Rule in SQL supports to define 2:2:1 cluster topology #47254
    • placement: make Constraints can use dict format #47344
  • add REPLICAS, substitute the FOLLOWERS keywords in placement rule syntax #47303
• New way to allocate Table ID, Partition ID. Let the Table ID be continuous that belongs the same database. then we can reduce rule numbers. and more easily isolate database data.

Observability

• Add check range region's rule fit interface in PD
• Add a SQL interface the check the rule health of the tables
• Supports show origin placement rules #46617
  • placement_rule: supports show origin placement rules #46750
• Add resource_group information in USER_ATTRIBUTES #45743
• Improve the error report of placement rule #47253

Test Cases

• Availability Scenario
  • Cluster Level
  • DB/Table/Partition Level
  • With Tiflash
• Isolation Scenario
  • 1-10 Apps Physical Isolation
  • Hot/Cold Data Isolation (not recommended for user)
• Others
  • The Boundary of Rule numbers (performance)
  • Verify SQL generates the right rule on the PD side