Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[regression](nereids) add test case for partition prune #26849

Merged
merged 3 commits into from
Nov 14, 2023
Merged

[regression](nereids) add test case for partition prune #26849

merged 3 commits into from
Nov 14, 2023

Conversation

englefly
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit c36b94b5714bdbcc9374199529a46ef520523b84, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5224	5094	5245	5094
q2	375	181	201	181
q3	2078	2036	2015	2015
q4	1475	1434	1434	1434
q5	4148	4186	4095	4095
q6	256	138	130	130
q7	2069	1604	1621	1604
q8	2757	2756	2765	2756
q9	10467	10280	10286	10280
q10	3487	3565	3562	3562
q11	377	261	249	249
q12	453	299	294	294
q13	4508	4085	4087	4085
q14	332	289	300	289
q15	652	582	581	581
q16	707	634	595	595
q17	1155	1098	1097	1097
q18	7801	7427	7421	7421
q19	1697	1706	1717	1706
q20	601	376	369	369
q21	4960	4576	4563	4563
q22	547	410	441	410
Total cold run time: 56126 ms
Total hot run time: 52810 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5032	5428	5112	5112
q2	329	236	228	228
q3	3983	3994	3946	3946
q4	2789	2766	2724	2724
q5	6503	6527	6508	6508
q6	243	126	129	126
q7	3173	2693	2709	2693
q8	4766	4769	4731	4731
q9	17769	17598	17745	17598
q10	4087	4175	4144	4144
q11	732	651	659	651
q12	1005	845	827	827
q13	4311	3961	3923	3923
q14	400	360	347	347
q15	649	549	569	549
q16	774	677	703	677
q17	3900	3994	3916	3916
q18	9396	9335	9201	9201
q19	1860	1790	1780	1780
q20	2368	2068	2073	2068
q21	8737	8657	8899	8657
q22	915	849	860	849
Total cold run time: 83721 ms
Total hot run time: 81255 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.73 seconds
stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17162436359 Bytes

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.5 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162270323 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit a2d609fa1fb1d659eaf46a76df3404e1678c41e4, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5353	5085	5169	5085
q2	370	158	139	139
q3	2021	2023	2017	2017
q4	1397	1378	1371	1371
q5	3965	3995	4008	3995
q6	258	134	134	134
q7	1440	876	897	876
q8	2793	2785	2768	2768
q9	9669	9707	9412	9412
q10	3440	3531	3512	3512
q11	388	267	261	261
q12	455	283	292	283
q13	4607	4194	4146	4146
q14	329	279	303	279
q15	629	554	580	554
q16	677	597	589	589
q17	1131	1092	1099	1092
q18	8023	7697	7626	7626
q19	1671	1696	1692	1692
q20	544	315	314	314
q21	4711	4357	4367	4357
q22	510	420	402	402
Total cold run time: 54381 ms
Total hot run time: 50904 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5230	5179	5039	5039
q2	354	222	229	222
q3	4120	4023	4020	4020
q4	2803	2764	2777	2764
q5	9588	9600	9487	9487
q6	254	129	125	125
q7	2736	2371	2343	2343
q8	4905	4868	4870	4868
q9	13097	13028	12891	12891
q10	4066	4155	4138	4138
q11	762	659	647	647
q12	1035	845	842	842
q13	4344	3905	3873	3873
q14	397	347	361	347
q15	602	557	565	557
q16	752	699	678	678
q17	3930	3926	3904	3904
q18	9745	9371	9591	9371
q19	1922	1793	1794	1793
q20	2378	2084	2068	2068
q21	8921	8943	8997	8943
q22	958	882	861	861
Total cold run time: 82899 ms
Total hot run time: 79781 ms

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@englefly englefly merged commit f1d90ff into apache:master Nov 14, 2023
26 of 28 checks passed
@englefly englefly deleted the prune-fix branch November 14, 2023 03:51
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 14, 2023
@englefly @seawinde
[regression](nereids) add test case for partition prune (apache#26849)
75a21b2 
* list selected partition name in explain
* add prune partition test case （multi-range key）
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 15, 2023
@englefly @seawinde
[regression](nereids) add test case for partition prune (apache#26849)
19447f3 
* list selected partition name in explain
* add prune partition test case （multi-range key）
englefly added a commit to englefly/incubator-doris that referenced this pull request Nov 24, 2023
@englefly
[regression](nereids) add test case for partition prune (apache#26849)
40a36c4 
* list selected partition name in explain
* add prune partition test case （multi-range key）
@xiaokang xiaokang mentioned this pull request Nov 24, 2023
Merged
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@englefly
[regression](nereids) add test case for partition prune (apache#26849)
f7d2626 
* list selected partition name in explain
* add prune partition test case （multi-range key）
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@starocean999 starocean999 starocean999 approved these changes

@Kikyou1997 Kikyou1997 Kikyou1997 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@englefly @doris-robot @Kikyou1997 @starocean999 @xiaokang