Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: fix wrong selectivity for inner selection in index join #10633

Merged
merged 9 commits into from
Jun 12, 2019

Conversation

eurekaka
Copy link
Contributor

@eurekaka eurekaka commented May 29, 2019

What problem does this PR solve?

Fix #10630

What is changed and how it works?

  • recompute selectivity for index filters of index scan which is inner child of index join;
  • do not use ds.stats as table plan's stats when it is double read in index join's inner side, because for inner child of index join, we are building new stats using average row count from scratch, so it is different from normal DataSource::DeriveStats;

Check List

Tests

  • Unit test

Code changes

N/A

Side effects

N/A

Related changes

  • Need to cherry-pick to the release branch

@eurekaka eurekaka added type/bugfix This PR fixes a bug. sig/planner SIG: Planner status/WIP and removed status/WIP labels May 29, 2019
@eurekaka
Copy link
Contributor Author

/run-all-tests

@eurekaka
Copy link
Contributor Author

/run-integration-common-test

logutil.Logger(context.Background()).Warn("calculate selectivity faild, use selection factor", zap.Error(err))
selectivity = selectionFactor
}
path.countAfterIndex = rowCount * selectivity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about make a new method on path for line 596~603 and exhaust_physical_plans.go:506~513.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, these lines are simple enough and are unnecessary to be extracted?

@codecov
Copy link

codecov bot commented May 31, 2019

Codecov Report

Merging #10633 into master will decrease coverage by 0.0208%.
The diff coverage is 82.8571%.

@@               Coverage Diff                @@
##             master     #10633        +/-   ##
================================================
- Coverage   80.1788%   80.1579%   -0.0209%     
================================================
  Files           415        415                
  Lines         88350      88358         +8     
================================================
- Hits          70838      70826        -12     
- Misses        12273      12293        +20     
  Partials       5239       5239

├─IndexScan_15 10.00 cop table:rr, index:aid, dic, range: decided by [eq(test.rr.aid, test.dt.aid) eq(test.rr.dic, test.dt.dic)], keep order:false, stats:pseudo
└─Selection_17 3.33 cop eq(test.rr.pt, "ios"), gt(test.rr.t, 1478185592)
└─Selection_17 0.00 cop eq(test.rr.pt, "ios"), gt(test.rr.t, 1478185592)
Copy link
Contributor Author

@eurekaka eurekaka Jun 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: the real row count here is 0.003333333333333334, which is reasonable; it is displayed as 0.00 because we are using count := string(strconv.AppendFloat([]byte{}, p.statsInfo().RowCount, 'f', 2, 64)) in prepareOperatorInfo.

@eurekaka
Copy link
Contributor Author

eurekaka commented Jun 1, 2019

/run-all-tests

@eurekaka eurekaka force-pushed the selectivity_index_join_inner branch from 50fe0e5 to eacdab0 Compare June 12, 2019 03:29
@eurekaka eurekaka force-pushed the selectivity_index_join_inner branch from a920e5d to 3046a25 Compare June 12, 2019 03:30
@eurekaka eurekaka requested a review from zz-jason June 12, 2019 03:37
@eurekaka eurekaka requested review from qw4990 and alivxxx June 12, 2019 03:37
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 12, 2019
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alivxxx alivxxx added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 12, 2019
@alivxxx alivxxx merged commit fbf58fc into pingcap:master Jun 12, 2019
@eurekaka eurekaka deleted the selectivity_index_join_inner branch June 13, 2019 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/planner SIG: Planner status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wrong selectivity for selection above inner child of index join
4 participants