Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: keep unfoldable exprs when pruning columns for ORDER BY items #10064

Merged
merged 5 commits into from
Apr 15, 2019
Merged

planner: keep unfoldable exprs when pruning columns for ORDER BY items #10064

merged 5 commits into from
Apr 15, 2019

Conversation

lysu
Copy link
Contributor

@lysu lysu commented Apr 8, 2019

What problem does this PR solve?

fixes #9033

What is changed and how it works?

pruning column logic will do prune if order by expression with zero columns(it's right for sql like order by abs(1)), but rand() shouldn't be pruned, so do skip logic.

Check List

Tests

  • Unit test

Code changes

  • implement change

Side effects

  • N/A

Related changes

  • Need to cherry-pick to the release branch

This change is Reviewable

@lysu lysu added type/bugfix This PR fixes a bug. sig/planner SIG: Planner labels Apr 8, 2019
@lysu lysu assigned zz-jason and eurekaka and unassigned zz-jason and eurekaka Apr 8, 2019
@lysu lysu requested review from eurekaka and zz-jason April 8, 2019 07:08
Copy link
Contributor

@eurekaka eurekaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eurekaka eurekaka added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 9, 2019
@eurekaka eurekaka changed the title planner: scalar function rand skip pruning planner: keep RAND() when pruning columns for ORDER BY items Apr 9, 2019
@lysu
Copy link
Contributor Author

lysu commented Apr 15, 2019

/run-all-tests

@lysu
Copy link
Contributor Author

lysu commented Apr 15, 2019

@qw4990 @winoros have changed to use ConstItem to determine prune PTAL thx~

@lysu
Copy link
Contributor Author

lysu commented Apr 15, 2019

we need:
desc select * from t1 where t1.a in (select t2.a as a from t2 where t2.b > t1.b order by t1.b limit 1);
id	count	task	operator info
Apply_15	9990.00	root	semi join, inner:Selection_19, equal:[eq(test.t1.a, test.t2.a)]
├─TableReader_18	9990.00	root	data:Selection_17
│ └─Selection_17	9990.00	cop	not(isnull(test.t1.a))
│   └─TableScan_16	10000.00	cop	table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─Selection_19	0.80	root	not(isnull(test.t2.a))
  └─Limit_20	1.00	root	offset:0, count:1
    └─TableReader_26	1.00	root	data:Limit_25
      └─Limit_25	1.00	cop	offset:0, count:1
        └─Selection_24	1.00	cop	gt(test.t2.b, test.t1.b)
          └─TableScan_23	1.25	cop	table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
desc select * from t1 where t1.a i
but got:
desc select * from t1 where t1.a in (select t2.a as a from t2 where t2.b > t1.b order by t1.b limit 1);
id	count	task	operator info
Apply_14	9990.00	root	semi join, inner:Selection_18, equal:[eq(test.t1.a, test.t2.a)]
├─TableReader_17	9990.00	root	data:Selection_16
│ └─Selection_16	9990.00	cop	not(isnull(test.t1.a))
│   └─TableScan_15	10000.00	cop	table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─Selection_18	0.80	root	not(isnull(test.t2.a))
  └─TopN_19	1.00	root	test.t1.b:asc, offset:0, count:1
    └─TableReader_25	1.00	root	data:TopN_24
      └─TopN_24	1.00	cop	test.t1.b:asc, offset:0, count:1
        └─Selection_23	8000.00	cop	gt(test.t2.b, test.t1.b)
          └─TableScan_22	10000.00	cop	table:t2, range:[-inf,+inf], keep order:false, stats:pseudo

it seems there are another order by pruning bug when subquery order by correlate column - - but solved by this PR

@lysu
Copy link
Contributor Author

lysu commented Apr 15, 2019

/run-all-tests

qw4990
qw4990 previously approved these changes Apr 15, 2019
Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lysu
Copy link
Contributor Author

lysu commented Apr 15, 2019

/run-integration-ddl-test
/run-sqllogic-test-2

@zz-jason
Copy link
Member

we need:
desc select * from t1 where t1.a in (select t2.a as a from t2 where t2.b > t1.b order by t1.b limit 1);
id	count	task	operator info
Apply_15	9990.00	root	semi join, inner:Selection_19, equal:[eq(test.t1.a, test.t2.a)]
├─TableReader_18	9990.00	root	data:Selection_17
│ └─Selection_17	9990.00	cop	not(isnull(test.t1.a))
│   └─TableScan_16	10000.00	cop	table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─Selection_19	0.80	root	not(isnull(test.t2.a))
  └─Limit_20	1.00	root	offset:0, count:1
    └─TableReader_26	1.00	root	data:Limit_25
      └─Limit_25	1.00	cop	offset:0, count:1
        └─Selection_24	1.00	cop	gt(test.t2.b, test.t1.b)
          └─TableScan_23	1.25	cop	table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
desc select * from t1 where t1.a i
but got:
desc select * from t1 where t1.a in (select t2.a as a from t2 where t2.b > t1.b order by t1.b limit 1);
id	count	task	operator info
Apply_14	9990.00	root	semi join, inner:Selection_18, equal:[eq(test.t1.a, test.t2.a)]
├─TableReader_17	9990.00	root	data:Selection_16
│ └─Selection_16	9990.00	cop	not(isnull(test.t1.a))
│   └─TableScan_15	10000.00	cop	table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─Selection_18	0.80	root	not(isnull(test.t2.a))
  └─TopN_19	1.00	root	test.t1.b:asc, offset:0, count:1
    └─TableReader_25	1.00	root	data:TopN_24
      └─TopN_24	1.00	cop	test.t1.b:asc, offset:0, count:1
        └─Selection_23	8000.00	cop	gt(test.t2.b, test.t1.b)
          └─TableScan_22	10000.00	cop	table:t2, range:[-inf,+inf], keep order:false, stats:pseudo

it seems there are another order by pruning bug when subquery order by correlate column - - but solved by this PR

This change is because CorrelatedColumn.ConstItems() returns false, which is incorrect, it's should be true.

zz-jason
zz-jason previously approved these changes Apr 15, 2019
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/planner SIG: Planner status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

order by rand() don't return random value
5 participants