statistics: fix repetitive selectivity accounting and stabilify the result #15536
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
requested review from
francis0407 and
winoros
and removed request for
a team
zz-jason
reviewed
Mar 31, 2020
| // getUsableSetsByGreedy will select the indices and pk used for calculate selectivity by greedy algorithm. | ||
| func getUsableSetsByGreedy(nodes []*StatsNode) (newBlocks []*StatsNode) { | ||
| // GetUsableSetsByGreedy will select the indices and pk used for calculate selectivity by greedy algorithm. | ||
| func GetUsableSetsByGreedy(nodes []*StatsNode) (newBlocks []*StatsNode) { |
There was a problem hiding this comment.
would it impact the performance?
There was a problem hiding this comment.
Maybe, because the input order is changed, so the result of greedy search may change.
lzmhhh123
reviewed
Apr 3, 2020
lzmhhh123
added
status/can-merge
|
/run-all-tests |
|
do we have this problem in release 3.0 and release 3.1? |
|
@zz-jason Yes |
|
cherry pick to release-2.1 in PR #16050 |
sre-bot
pushed a commit
to sre-bot/tidb
that referenced
this pull request
Apr 3, 2020
|
cherry pick to release-3.0 in PR #16051 |
sre-bot
pushed a commit
to sre-bot/tidb
that referenced
this pull request
Apr 3, 2020
|
cherry pick to release-3.1 in PR #16052 |
eurekaka
added a commit
to sre-bot/tidb
that referenced
this pull request
Apr 3, 2020
Signed-off-by: sre-bot <sre-bot@pingcap.com>
eurekaka
added a commit
to eurekaka/tidb
that referenced
this pull request
Apr 3, 2020
sre-bot
added a commit
that referenced
this pull request
Apr 3, 2020
sre-bot
pushed a commit
that referenced
this pull request
Apr 3, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Problem Summary:
Selectivity, index order incoll.Indicesis non-deterministic, so the greedy search algorithm may return different results in different runs, that would confuse users since the stats is not changed at all;t.a = 1 and t.b > 1 and t.c > 1, and there are 2 indexesidx1(a,b)andidx2(a,c), the greedy algorithm would choose both indexes and multiply their selectivity computed respectively. Obviously, this is wrong, because selectivity oft.a = 1is accounted twice.What is changed and how it works?
What's Changed:
StatsNodeslice before greedy search;How it Works:
Note that, how we sort the
StatsNodeslice impacts the greedy search result. I put the PK in the end of the slice, indexes in the middle and columns in the front, to enforce the heuristic rule that, PK is preferred over indexes in estimation, and indexes are preferred over columns.Related changes
Check List
Tests
Side effects
compareTypefunction, instead of changing the values ofIndexType/PkType/ColType, because feedback encoding uses these constants.Release note