Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](Export) add feut for Cancel Export #27178

Merged
merged 2 commits into from
Nov 23, 2023

Conversation

BePPPower
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.22 seconds
stream load tsv: 582 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17099658220 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit ba368dc19bc220059e34d4c3624c4573dac84915, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4978	4686	4661	4661
q2	358	166	159	159
q3	2036	1932	1859	1859
q4	1393	1288	1269	1269
q5	4018	3985	4059	3985
q6	250	133	128	128
q7	1446	877	887	877
q8	2790	2819	2799	2799
q9	9843	9706	9592	9592
q10	3479	3538	3572	3538
q11	373	236	249	236
q12	436	294	297	294
q13	4566	3786	3785	3785
q14	318	282	286	282
q15	590	535	532	532
q16	665	579	583	579
q17	1145	954	928	928
q18	7882	7408	7547	7408
q19	1696	1692	1698	1692
q20	531	313	306	306
q21	4411	4024	4017	4017
q22	475	379	376	376
Total cold run time: 53679 ms
Total hot run time: 49302 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4585	4572	4583	4572
q2	340	265	251	251
q3	4035	4008	3992	3992
q4	2722	2691	2725	2691
q5	9608	9689	9653	9653
q6	246	123	125	123
q7	2627	2317	2264	2264
q8	4462	4518	4482	4482
q9	13245	13178	13165	13165
q10	4098	4207	4203	4203
q11	788	647	700	647
q12	975	807	808	807
q13	4310	3580	3576	3576
q14	382	340	355	340
q15	575	519	541	519
q16	728	682	683	682
q17	3852	3886	3883	3883
q18	9674	9039	9081	9039
q19	1822	1780	1802	1780
q20	2429	2046	2039	2039
q21	8869	8649	8676	8649
q22	877	809	822	809
Total cold run time: 81249 ms
Total hot run time: 78166 ms

@BePPPower
Copy link
Contributor Author

run feut

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.41 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17103518189 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit e7cfdb29ec7a6fea8e5b46d790bd1c0dda8813f0, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4976	4699	4659	4659
q2	361	153	158	153
q3	2049	1917	1908	1908
q4	1400	1273	1251	1251
q5	4036	3971	4025	3971
q6	247	128	130	128
q7	1382	861	896	861
q8	2759	2793	2787	2787
q9	9738	9721	9580	9580
q10	3462	3552	3504	3504
q11	392	238	246	238
q12	437	294	298	294
q13	4579	3815	3790	3790
q14	329	294	288	288
q15	579	538	526	526
q16	672	582	581	581
q17	1134	955	902	902
q18	7917	7324	7486	7324
q19	1698	1692	1695	1692
q20	564	318	327	318
q21	4389	3952	3979	3952
q22	472	381	373	373
Total cold run time: 53572 ms
Total hot run time: 49080 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4588	4585	4597	4585
q2	357	223	235	223
q3	4036	3994	3999	3994
q4	2708	2687	2704	2687
q5	9658	9672	9702	9672
q6	243	120	123	120
q7	2576	2266	2268	2266
q8	4421	4432	4437	4432
q9	13291	13182	13148	13148
q10	4122	4204	4149	4149
q11	799	731	660	660
q12	962	810	823	810
q13	4274	3596	3536	3536
q14	385	345	353	345
q15	578	519	518	518
q16	739	667	660	660
q17	3878	3991	3888	3888
q18	9652	9083	8975	8975
q19	1858	1788	1785	1785
q20	2370	2066	2047	2047
q21	8876	8628	8695	8628
q22	906	789	798	789
Total cold run time: 81277 ms
Total hot run time: 77917 ms

@BePPPower
Copy link
Contributor Author

run p0

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 20, 2023
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit eb878ad into apache:master Nov 23, 2023
28 of 30 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.4 reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants