Improve the Accuracy and Robustness of Index Selection

[zz-jason]

Description

Improve the optimizer accuracy: don't select a wrong index on up-to-date statistics. Query optimizer usually adopts assumptions that usually not hold in the real-world dataset. For example, independent assumption between attributes of a relation, uniformly distribution inside a histogram bucket. These assumptions usually lead to bad estimation accuracy, which further causes a sub-optimal execution plan. In this issue, let's focussed on the wrong index selection issue.

Improve robustness: prevent plan regression when the underlying data is changed.

Category

Stability, Performance

Value

• Value Score: 5

Usually, there are many applications running on a TiDB cluster. If one of the applications runs a SQL whose index selection is incorrect, for example, a full-range table scan is chosen instead of an index range scan. The whole cluster can be influenced, all the applications run on the same cluster can be slowed down. Which causes a bad production incident.

It's hard to totally prevent incorrect index selection, but we need to try our best to fix all the existing known issues and increase the robustness of the index selection.

Progress Tracking

• Weekly Report: Index Selection Work Plan & Weekly Report
• Join Us: #sig-planner in the TiDB Community slack workspace

Time

GanttStart: 2020-07-01
GanttDue: 2020-11-28
GanttProgress: 30%