Skip to content

Conversation

@Jibing-Li
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

* fix visible column (apache#33023)
* Collect high priority columns. (apache#31235)
* High priority queue and map. (apache#31509)
* Support column level health value. (apache#31794)
* Support follower sync query columns to master. (apache#31859)
* Support show auto analyze pending jobs. (apache#31926)
* Check column health value earlier, show job priority. (apache#32064)
* support window (apache#32094)
* Refactor. (apache#32273)
* refactor2 (apache#32278)
* Unit test (apache#32398)
* Support auto analyze mv (apache#32433)
* Fix bug (apache#32454)
* Support identical column name in different index. (apache#32957)
* Fix visible column
* Use future to block auto analyze before job finish. (apache#33083)
* Fix ut. (apache#33147)
* Fix ut (apache#33161)
* fix p0 (apache#33210)
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li
Copy link
Contributor Author

run feut

@github-actions
Copy link
Contributor

github-actions bot commented Apr 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@Jibing-Li
Copy link
Contributor Author

run p0

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40266 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6038033cbe6909bb8655da91a0d5fdd6620a55a4, data reload: false

------ Round 1 ----------------------------------
q1	17604	4176	4077	4077
q2	2023	191	184	184
q3	10543	1344	1504	1344
q4	10166	962	1006	962
q5	7508	3107	3049	3049
q6	221	137	141	137
q7	1100	565	575	565
q8	9514	2140	2147	2140
q9	6875	6406	6357	6357
q10	8601	3634	3730	3634
q11	422	249	239	239
q12	398	220	233	220
q13	17869	3118	3074	3074
q14	289	246	256	246
q15	526	493	478	478
q16	555	411	415	411
q17	999	950	929	929
q18	7659	6922	6766	6766
q19	1666	1565	1564	1564
q20	630	323	318	318
q21	3777	3315	3250	3250
q22	389	322	323	322
Total cold run time: 109334 ms
Total hot run time: 40266 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4288	4229	4237	4229
q2	342	226	225	225
q3	3133	3150	3157	3150
q4	2038	2039	2001	2001
q5	5538	5520	5334	5334
q6	207	122	125	122
q7	1785	1397	1409	1397
q8	3337	3387	3383	3383
q9	8815	8777	8760	8760
q10	3958	4025	4022	4022
q11	542	461	455	455
q12	766	609	574	574
q13	16155	3115	2962	2962
q14	295	255	255	255
q15	516	466	473	466
q16	456	421	434	421
q17	1735	1734	1739	1734
q18	8081	7835	7403	7403
q19	1696	1643	1649	1643
q20	1948	1723	1711	1711
q21	5034	4817	4835	4817
q22	509	436	406	406
Total cold run time: 71174 ms
Total hot run time: 55470 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 181309 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6038033cbe6909bb8655da91a0d5fdd6620a55a4, data reload: false

query1	1250	1129	1104	1104
query2	6448	1728	1769	1728
query3	6580	206	224	206
query4	23526	21547	21622	21547
query5	4183	425	412	412
query6	272	197	203	197
query7	4540	305	297	297
query8	221	178	174	174
query9	8309	2228	2204	2204
query10	428	260	256	256
query11	14988	14558	14475	14475
query12	152	102	104	102
query13	1598	397	387	387
query14	8521	6933	6793	6793
query15	213	177	177	177
query16	7153	275	280	275
query17	988	594	589	589
query18	1923	302	282	282
query19	214	167	172	167
query20	101	96	99	96
query21	198	129	129	129
query22	4881	4778	4750	4750
query23	33612	32892	33089	32892
query24	11939	3169	3217	3169
query25	674	404	404	404
query26	1776	172	168	168
query27	2943	326	337	326
query28	6675	1851	1828	1828
query29	1282	618	599	599
query30	254	168	176	168
query31	964	748	740	740
query32	101	65	62	62
query33	714	260	272	260
query34	965	496	513	496
query35	825	708	704	704
query36	1000	874	846	846
query37	286	81	84	81
query38	3549	3432	3383	3383
query39	1589	1548	1539	1539
query40	259	135	142	135
query41	51	47	47	47
query42	118	116	110	110
query43	441	397	397	397
query44	1085	714	714	714
query45	286	256	231	231
query46	1084	804	794	794
query47	1891	1767	1795	1767
query48	395	318	316	316
query49	957	380	381	380
query50	801	406	418	406
query51	6916	6860	6752	6752
query52	116	94	101	94
query53	367	300	292	292
query54	310	245	237	237
query55	88	80	80	80
query56	246	224	226	224
query57	1220	1140	1113	1113
query58	253	225	233	225
query59	2577	2391	2308	2308
query60	260	241	257	241
query61	91	88	92	88
query62	678	454	449	449
query63	317	283	287	283
query64	6128	2945	3397	2945
query65	3056	3033	3031	3031
query66	1369	316	327	316
query67	15397	14919	14782	14782
query68	9367	567	589	567
query69	565	368	354	354
query70	1363	1116	1077	1077
query71	502	276	281	276
query72	6384	2594	2434	2434
query73	1486	325	330	325
query74	6853	6248	6276	6248
query75	3624	2341	2331	2331
query76	5339	1171	1213	1171
query77	656	257	251	251
query78	10858	10251	10102	10102
query79	9512	542	535	535
query80	1241	457	428	428
query81	500	249	237	237
query82	280	106	106	106
query83	195	171	163	163
query84	274	88	90	88
query85	989	289	280	280
query86	385	279	271	271
query87	3744	3507	3489	3489
query88	3801	2281	2275	2275
query89	542	373	372	372
query90	2080	183	182	182
query91	135	107	103	103
query92	66	55	54	54
query93	6626	547	529	529
query94	1341	202	195	195
query95	447	347	334	334
query96	595	273	267	267
query97	2638	2493	2448	2448
query98	231	222	215	215
query99	1208	842	833	833
Total cold run time: 296419 ms
Total hot run time: 181309 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.03 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6038033cbe6909bb8655da91a0d5fdd6620a55a4, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.06	0.06
query5	0.49	0.50	0.48
query6	1.13	0.64	0.66
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.57	0.52	0.50
query10	0.56	0.59	0.56
query11	0.14	0.11	0.12
query12	0.13	0.11	0.12
query13	0.63	0.60	0.59
query14	0.76	0.78	0.80
query15	0.85	0.84	0.82
query16	0.35	0.35	0.36
query17	0.97	0.98	0.98
query18	0.26	0.24	0.26
query19	1.81	1.72	1.73
query20	0.01	0.01	0.01
query21	16.99	0.84	0.70
query22	3.53	4.98	2.63
query23	17.52	1.33	1.13
query24	1.49	0.30	0.22
query25	0.14	0.09	0.08
query26	0.28	0.17	0.18
query27	0.08	0.09	0.09
query28	13.61	0.98	0.95
query29	12.55	3.35	3.42
query30	0.23	0.06	0.05
query31	2.85	0.38	0.38
query32	3.28	0.46	0.46
query33	2.83	2.89	2.89
query34	15.53	4.40	4.32
query35	4.39	4.37	4.39
query36	0.68	0.47	0.48
query37	0.20	0.17	0.17
query38	0.17	0.16	0.15
query39	0.05	0.03	0.04
query40	0.21	0.17	0.16
query41	0.10	0.06	0.05
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.56 s
Total hot run time: 31.03 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6038033cbe6909bb8655da91a0d5fdd6620a55a4 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.8 seconds inserted 10000000 Rows, about 675K ops/s

@Jibing-Li Jibing-Li closed this Apr 8, 2024
@Jibing-Li Jibing-Li deleted the merge branch April 8, 2024 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants