Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner/core: make join reorder by dp work #8816

Merged
merged 24 commits into from
Apr 28, 2019
Merged

planner/core: make join reorder by dp work #8816

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0fc7469
planner/core: make join reorder by dp work
winoros Dec 25, 2018
373d455
Merge branch 'master' into join-reorder-dp
winoros Dec 25, 2018
d38fc88
fix explain test
winoros Dec 25, 2018
0c1cb35
fix lint
winoros Dec 25, 2018
868a44d
fix unit-test and integration test
winoros Dec 26, 2018
70099c9
Merge branch 'master' into join-reorder-dp
winoros Dec 26, 2018
dd7cd1a
address comments
winoros Jan 2, 2019
db4039f
address commnets
winoros Jan 7, 2019
52cd6ed
Merge branch 'master' into join-reorder-dp
winoros Jan 7, 2019
fe980e5
Merge branch 'master' into join-reorder-dp
winoros Feb 25, 2019
cd6b696
Merge branch 'master' into join-reorder-dp
winoros Mar 1, 2019
9babf7d
Merge branch 'master' into join-reorder-dp
winoros Mar 4, 2019
4fef276
refactor code and disable dp by default
winoros Mar 4, 2019
b28a496
fix unit-test
winoros Mar 4, 2019
1eb891f
address comments
winoros Mar 6, 2019
a0b3661
address comments
winoros Mar 11, 2019
43b75c0
Merge branch 'master' into join-reorder-dp
winoros Mar 11, 2019
3f116ee
address comments
winoros Mar 11, 2019
4eced58
Merge branch 'master' into join-reorder-dp
winoros Mar 15, 2019
99c45c2
fix small bug
winoros Mar 15, 2019
3d8a025
Merge branch 'master' into join-reorder-dp
winoros Apr 24, 2019
30fb2af
Merge branch 'master' into join-reorder-dp
winoros Apr 24, 2019
3235989
Merge branch 'master' into join-reorder-dp
zz-jason Apr 28, 2019
eadc50d
Merge branch 'master' into join-reorder-dp
zz-jason Apr 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions cmd/explaintest/r/explain_complex.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,15 +118,15 @@ id count task operator info
Projection_13 1.00 root gad.id, test.dd.id, gad.aid, gad.cm, test.dd.dic, test.dd.ip, test.dd.t, gad.p1, gad.p2, gad.p3, gad.p4, gad.p5, gad.p6_md5, gad.p7_md5, gad.ext, gad.t
└─Limit_16 1.00 root offset:0, count:2500
└─HashAgg_19 1.00 root group by:gad.aid, test.dd.dic, funcs:firstrow(gad.id), firstrow(gad.aid), firstrow(gad.cm), firstrow(gad.p1), firstrow(gad.p2), firstrow(gad.p3), firstrow(gad.p4), firstrow(gad.p5), firstrow(gad.p6_md5), firstrow(gad.p7_md5), firstrow(gad.ext), firstrow(gad.t), firstrow(test.dd.id), firstrow(test.dd.dic), firstrow(test.dd.ip), firstrow(test.dd.t)
└─IndexJoin_24 0.00 root inner join, inner:IndexLookUp_23, outer key:gad.aid, inner key:test.dd.aid, other cond:eq(test.dd.ip, gad.ip), gt(test.dd.t, gad.t)
├─IndexLookUp_23 0.00 root
│ ├─IndexScan_20 10.00 cop table:dd, index:aid, dic, range: decided by [gad.aid gad.ip], keep order:false, stats:pseudo
│ └─Selection_22 0.00 cop eq(test.dd.bm, 0), eq(test.dd.pt, "android"), gt(test.dd.t, 1478143908)
│ └─TableScan_21 10.00 cop table:dd, keep order:false, stats:pseudo
└─IndexLookUp_33 3.33 root
├─IndexScan_30 3333.33 cop table:gad, index:t, range:(1478143908,+inf], keep order:false, stats:pseudo
└─Selection_32 3.33 cop eq(gad.pt, "android")
└─TableScan_31 3333.33 cop table:st, keep order:false, stats:pseudo
└─IndexJoin_24 0.00 root inner join, inner:IndexLookUp_23, outer key:gad.aid, inner key:test.dd.aid, other cond:eq(gad.ip, test.dd.ip), gt(test.dd.t, gad.t)
├─IndexLookUp_33 3.33 root
│ ├─IndexScan_30 3333.33 cop table:gad, index:t, range:(1478143908,+inf], keep order:false, stats:pseudo
│ └─Selection_32 3.33 cop eq(gad.pt, "android")
│ └─TableScan_31 3333.33 cop table:st, keep order:false, stats:pseudo
└─IndexLookUp_23 0.00 root
├─IndexScan_20 10.00 cop table:dd, index:aid, dic, range: decided by [gad.aid gad.ip], keep order:false, stats:pseudo
└─Selection_22 0.00 cop eq(test.dd.bm, 0), eq(test.dd.pt, "android"), gt(test.dd.t, 1478143908)
└─TableScan_21 10.00 cop table:dd, keep order:false, stats:pseudo
explain select gad.id as gid,sdk.id as sid,gad.aid as aid,gad.cm as cm,sdk.dic as dic,sdk.ip as ip, sdk.t as t, gad.p1 as p1, gad.p2 as p2, gad.p3 as p3, gad.p4 as p4, gad.p5 as p5, gad.p6_md5 as p6, gad.p7_md5 as p7, gad.ext as ext from st gad join dd sdk on gad.aid = sdk.aid and gad.dic = sdk.mac and gad.t < sdk.t where gad.t > 1477971479 and gad.bm = 0 and gad.pt = 'ios' and gad.dit = 'mac' and sdk.t > 1477971479 and sdk.bm = 0 and sdk.pt = 'ios' limit 3000;
id count task operator info
Projection_10 0.00 root gad.id, sdk.id, gad.aid, gad.cm, sdk.dic, sdk.ip, sdk.t, gad.p1, gad.p2, gad.p3, gad.p4, gad.p5, gad.p6_md5, gad.p7_md5, gad.ext
Expand Down
73 changes: 36 additions & 37 deletions cmd/explaintest/r/explain_easy.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,12 @@ explain select count(b.c2) from t1 a, t2 b where a.c1 = b.c2 group by a.c1;
id count task operator info
Projection_11 10000.00 root cast(join_agg_0)
└─IndexJoin_14 10000.00 root inner join, inner:TableReader_13, outer key:b.c2, inner key:a.c1
├─HashAgg_21 8000.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
│ └─TableReader_22 8000.00 root data:HashAgg_17
│ └─HashAgg_17 8000.00 cop group by:b.c2, funcs:count(b.c2), firstrow(b.c2)
│ └─TableScan_20 10000.00 cop table:b, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_13 10.00 root data:TableScan_12
└─TableScan_12 10.00 cop table:a, range: decided by [b.c2], keep order:false, stats:pseudo
├─TableReader_13 10.00 root data:TableScan_12
│ └─TableScan_12 10.00 cop table:a, range: decided by [b.c2], keep order:false, stats:pseudo
└─HashAgg_21 8000.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
└─TableReader_22 8000.00 root data:HashAgg_17
└─HashAgg_17 8000.00 cop group by:b.c2, funcs:count(b.c2), firstrow(b.c2)
└─TableScan_20 10000.00 cop table:b, range:[-inf,+inf], keep order:false, stats:pseudo
explain select * from t2 order by t2.c2 limit 0, 1;
id count task operator info
TopN_7 1.00 root test.t2.c2:asc, offset:0, count:1
Expand Down Expand Up @@ -96,12 +96,12 @@ explain select c1 from t1 where c1 in (select c2 from t2);
id count task operator info
Projection_9 10000.00 root test.t1.c1
└─IndexJoin_12 10000.00 root inner join, inner:TableReader_11, outer key:test.t2.c2, inner key:test.t1.c1
├─HashAgg_19 8000.00 root group by:col_1, funcs:firstrow(col_0)
│ └─TableReader_20 8000.00 root data:HashAgg_15
│ └─HashAgg_15 8000.00 cop group by:test.t2.c2, funcs:firstrow(test.t2.c2)
│ └─TableScan_18 10000.00 cop table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_11 10.00 root data:TableScan_10
└─TableScan_10 10.00 cop table:t1, range: decided by [test.t2.c2], keep order:false, stats:pseudo
├─TableReader_11 10.00 root data:TableScan_10
│ └─TableScan_10 10.00 cop table:t1, range: decided by [test.t2.c2], keep order:false, stats:pseudo
└─HashAgg_19 8000.00 root group by:col_1, funcs:firstrow(col_0)
└─TableReader_20 8000.00 root data:HashAgg_15
└─HashAgg_15 8000.00 cop group by:test.t2.c2, funcs:firstrow(test.t2.c2)
└─TableScan_18 10000.00 cop table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
explain select (select count(1) k from t1 s where s.c1 = t1.c1 having k != 0) from t1;
id count task operator info
Projection_12 10000.00 root k
Expand Down Expand Up @@ -356,13 +356,12 @@ id count task operator info
TableDual_5 0.00 root rows:0
explain select * from t t1 join t t2 where t1.b = t2.b and t2.b is null;
id count task operator info
Projection_7 12.50 root t1.a, t1.b, t2.a, t2.b
└─HashRightJoin_9 12.50 root inner join, inner:TableReader_12, equal:[eq(t2.b, t1.b)]
├─TableReader_12 10.00 root data:Selection_11
│ └─Selection_11 10.00 cop isnull(t2.b)
│ └─TableScan_10 10000.00 cop table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_14 10000.00 root data:TableScan_13
└─TableScan_13 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
HashLeftJoin_8 12.50 root inner join, inner:TableReader_14, equal:[eq(t1.b, t2.b)]
├─TableReader_11 10000.00 root data:TableScan_10
│ └─TableScan_10 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_14 10.00 root data:Selection_13
└─Selection_13 10.00 cop isnull(t2.b)
└─TableScan_12 10000.00 cop table:t2, range:[-inf,+inf], keep order:false, stats:pseudo
explain select * from t t1 where not exists (select * from t t2 where t1.b = t2.b);
id count task operator info
HashLeftJoin_9 8000.00 root anti semi join, inner:TableReader_13, equal:[eq(t1.b, t2.b)]
Expand Down Expand Up @@ -480,15 +479,15 @@ Projection_12 10000.00 root 9_aux_0
├─TableReader_16 10000.00 root data:TableScan_15
│ └─TableScan_15 10000.00 cop table:t, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashAgg_19 1.00 root funcs:count(join_agg_0)
└─HashRightJoin_21 10.00 root inner join, inner:HashAgg_27, equal:[eq(t1.a, s.a)]
├─HashAgg_27 8.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
│ └─TableReader_28 8.00 root data:HashAgg_22
│ └─HashAgg_22 8.00 cop group by:t1.a, funcs:count(1), firstrow(t1.a)
│ └─Selection_26 10.00 cop eq(t1.a, test.t.a)
│ └─TableScan_25 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_34 10.00 root data:Selection_33
└─Selection_33 10.00 cop eq(s.a, test.t.a)
└─TableScan_32 10000.00 cop table:s, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashLeftJoin_20 10.00 root inner join, inner:HashAgg_30, equal:[eq(s.a, t1.a)]
├─TableReader_24 10.00 root data:Selection_23
│ └─Selection_23 10.00 cop eq(s.a, test.t.a)
│ └─TableScan_22 10000.00 cop table:s, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashAgg_30 8.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
└─TableReader_31 8.00 root data:HashAgg_25
└─HashAgg_25 8.00 cop group by:t1.a, funcs:count(1), firstrow(t1.a)
└─Selection_29 10.00 cop eq(t1.a, test.t.a)
└─TableScan_28 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
explain select * from t ta left outer join t tb on ta.nb = tb.nb and ta.a > 1 where ifnull(tb.a, 1) or tb.a is null;
id count task operator info
Selection_7 10000.00 root or(ifnull(tb.a, 1), isnull(tb.a))
Expand All @@ -514,15 +513,15 @@ Projection_14 10000.00 root 9_aux_0
│ └─TableReader_19 10000.00 root data:TableScan_18
│ └─TableScan_18 10000.00 cop table:t, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashAgg_23 1.00 root funcs:count(join_agg_0)
└─HashRightJoin_25 10.00 root inner join, inner:HashAgg_31, equal:[eq(t1.a, s.a)]
├─HashAgg_31 8.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
│ └─TableReader_32 8.00 root data:HashAgg_26
│ └─HashAgg_26 8.00 cop group by:t1.a, funcs:count(1), firstrow(t1.a)
│ └─Selection_30 10.00 cop eq(t1.a, test.t.a)
│ └─TableScan_29 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─TableReader_38 10.00 root data:Selection_37
└─Selection_37 10.00 cop eq(s.a, test.t.a)
└─TableScan_36 10000.00 cop table:s, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashLeftJoin_24 10.00 root inner join, inner:HashAgg_34, equal:[eq(s.a, t1.a)]
├─TableReader_28 10.00 root data:Selection_27
│ └─Selection_27 10.00 cop eq(s.a, test.t.a)
│ └─TableScan_26 10000.00 cop table:s, range:[-inf,+inf], keep order:false, stats:pseudo
└─HashAgg_34 8.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
└─TableReader_35 8.00 root data:HashAgg_29
└─HashAgg_29 8.00 cop group by:t1.a, funcs:count(1), firstrow(t1.a)
└─Selection_33 10.00 cop eq(t1.a, test.t.a)
└─TableScan_32 10000.00 cop table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
drop table if exists t;
create table t(a int);
explain select * from t where _tidb_rowid = 0;
Expand Down
24 changes: 12 additions & 12 deletions cmd/explaintest/r/explain_easy_stats.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@ explain select count(b.c2) from t1 a, t2 b where a.c1 = b.c2 group by a.c1;
id count task operator info
Projection_11 1985.00 root cast(join_agg_0)
└─IndexJoin_14 1985.00 root inner join, inner:TableReader_13, outer key:b.c2, inner key:a.c1
├─HashAgg_21 1985.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
│ └─TableReader_22 1985.00 root data:HashAgg_17
│ └─HashAgg_17 1985.00 cop group by:b.c2, funcs:count(b.c2), firstrow(b.c2)
│ └─TableScan_20 1985.00 cop table:b, range:[-inf,+inf], keep order:false
└─TableReader_13 1.00 root data:TableScan_12
└─TableScan_12 1.00 cop table:a, range: decided by [b.c2], keep order:false
├─TableReader_13 1.00 root data:TableScan_12
│ └─TableScan_12 1.00 cop table:a, range: decided by [b.c2], keep order:false
└─HashAgg_21 1985.00 root group by:col_2, funcs:count(col_0), firstrow(col_1)
└─TableReader_22 1985.00 root data:HashAgg_17
└─HashAgg_17 1985.00 cop group by:b.c2, funcs:count(b.c2), firstrow(b.c2)
└─TableScan_20 1985.00 cop table:b, range:[-inf,+inf], keep order:false
explain select * from t2 order by t2.c2 limit 0, 1;
id count task operator info
TopN_7 1.00 root test.t2.c2:asc, offset:0, count:1
Expand All @@ -94,12 +94,12 @@ explain select c1 from t1 where c1 in (select c2 from t2);
id count task operator info
Projection_9 1985.00 root test.t1.c1
└─IndexJoin_12 1985.00 root inner join, inner:TableReader_11, outer key:test.t2.c2, inner key:test.t1.c1
├─HashAgg_19 1985.00 root group by:col_1, funcs:firstrow(col_0)
│ └─TableReader_20 1985.00 root data:HashAgg_15
│ └─HashAgg_15 1985.00 cop group by:test.t2.c2, funcs:firstrow(test.t2.c2)
│ └─TableScan_18 1985.00 cop table:t2, range:[-inf,+inf], keep order:false
└─TableReader_11 1.00 root data:TableScan_10
└─TableScan_10 1.00 cop table:t1, range: decided by [test.t2.c2], keep order:false
├─TableReader_11 1.00 root data:TableScan_10
│ └─TableScan_10 1.00 cop table:t1, range: decided by [test.t2.c2], keep order:false
└─HashAgg_19 1985.00 root group by:col_1, funcs:firstrow(col_0)
└─TableReader_20 1985.00 root data:HashAgg_15
└─HashAgg_15 1985.00 cop group by:test.t2.c2, funcs:firstrow(test.t2.c2)
└─TableScan_18 1985.00 cop table:t2, range:[-inf,+inf], keep order:false
explain select * from information_schema.columns;
id count task operator info
MemTableScan_4 10000.00 root
Expand Down
Loading