[enhance](function) refactor from_format_str and support more format #30452

zclllyybb · 2024-01-26T15:15:33Z

Proposed changes

Issue Number: close #xxx

now support incomplete date parsing with any legal format, like:

mysql> select STR_TO_DATE('2020%2', '%Y%%%m');
+---------------------------------+
| str_to_date('2020%2', '%Y%%%m') |
+---------------------------------+
| 2020-02-01                      |
+---------------------------------+

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

zclllyybb · 2024-01-26T15:15:39Z

run buildall

github-actions

clang-tidy made some suggestions

be/src/vec/runtime/vdatetime_value.cpp

doris-robot · 2024-01-26T16:26:56Z

TPC-H: Total hot run time: 39451 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34dd8654599456cbf726c7802773576283699b17, data reload: false

------ Round 1 ----------------------------------
q1	17647	5568	5292	5292
q2	2076	150	132	132
q3	10696	950	944	944
q4	4729	795	854	795
q5	7679	2986	3032	2986
q6	185	122	119	119
q7	1106	753	736	736
q8	9322	1988	2041	1988
q9	7334	6466	6431	6431
q10	8104	2457	2476	2457
q11	408	204	199	199
q12	784	295	290	290
q13	18008	3363	3369	3363
q14	298	259	261	259
q15	548	534	505	505
q16	471	411	426	411
q17	946	569	538	538
q18	7608	6846	6763	6763
q19	1557	1434	1366	1366
q20	590	341	340	340
q21	6927	3249	3247	3247
q22	822	297	290	290
Total cold run time: 107845 ms
Total hot run time: 39451 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5372	5388	5266	5266
q2	329	216	210	210
q3	3359	3162	3224	3162
q4	2075	2083	2012	2012
q5	6061	5900	5923	5900
q6	196	120	116	116
q7	2367	1909	1884	1884
q8	3232	3372	3397	3372
q9	9032	8762	8820	8762
q10	5432	3799	3868	3799
q11	549	465	449	449
q12	807	629	652	629
q13	16906	3147	3164	3147
q14	294	265	266	265
q15	547	506	513	506
q16	514	476	475	475
q17	1886	1856	1863	1856
q18	9510	19724	9587	9587
q19	24505	1567	1504	1504
q20	4579	1938	1914	1914
q21	14842	5647	5687	5647
q22	988	565	541	541
Total cold run time: 113382 ms
Total hot run time: 61003 ms

doris-robot · 2024-01-26T16:38:19Z

TPC-DS: Total hot run time: 176495 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34dd8654599456cbf726c7802773576283699b17, data reload: false

query1	946	332	336	332
query2	6560	2207	1855	1855
query3	6701	205	198	198
query4	31135	22307	22168	22168
query5	4447	373	432	373
query6	242	154	150	150
query7	4611	274	263	263
query8	228	178	180	178
query9	8365	2546	2544	2544
query10	414	209	221	209
query11	17030	15615	15494	15494
query12	124	77	65	65
query13	1684	376	373	373
query14	9714	6755	6778	6755
query15	207	178	181	178
query16	5725	255	253	253
query17	950	471	469	469
query18	1787	252	255	252
query19	181	132	125	125
query20	68	68	67	67
query21	184	138	115	115
query22	4880	4810	4777	4777
query23	31614	30876	30856	30856
query24	11718	2838	2819	2819
query25	585	305	317	305
query26	1744	140	137	137
query27	3189	282	284	282
query28	7568	1831	1822	1822
query29	1677	614	611	611
query30	274	135	137	135
query31	936	739	753	739
query32	76	51	47	47
query33	695	211	203	203
query34	1149	466	474	466
query35	898	766	791	766
query36	1359	1221	1248	1221
query37	96	59	58	58
query38	3328	3216	3200	3200
query39	1318	1255	1255	1255
query40	200	89	82	82
query41	37	38	35	35
query42	89	84	83	83
query43	475	469	505	469
query44	1141	681	696	681
query45	191	178	180	178
query46	1047	653	660	653
query47	1646	1549	1516	1516
query48	396	312	308	308
query49	1125	299	292	292
query50	683	318	318	318
query51	5284	5200	5188	5188
query52	88	83	73	73
query53	317	266	257	257
query54	237	186	186	186
query55	78	76	67	67
query56	185	168	158	158
query57	1004	944	944	944
query58	187	153	152	152
query59	2923	2688	2645	2645
query60	207	181	185	181
query61	90	82	82	82
query62	613	376	371	371
query63	283	255	263	255
query64	4962	1817	1761	1761
query65	3315	3271	3238	3238
query66	1323	321	315	315
query67	15503	15538	15438	15438
query68	10203	501	509	501
query69	606	298	293	293
query70	1703	1570	1425	1425
query71	540	216	212	212
query72	4571	2849	2869	2849
query73	1989	314	302	302
query74	7122	6431	6339	6339
query75	4492	2384	2260	2260
query76	6474	1016	1039	1016
query77	636	236	232	232
query78	9013	8822	8526	8526
query79	1018	507	503	503
query80	731	319	314	314
query81	483	199	198	198
query82	259	79	83	79
query83	169	115	115	115
query84	277	76	67	67
query85	1083	340	320	320
query86	385	377	367	367
query87	3509	3306	3398	3306
query88	3084	2198	2173	2173
query89	446	342	338	338
query90	1975	188	178	178
query91	149	116	117	116
query92	50	45	43	43
query93	1351	445	429	429
query94	1216	162	159	159
query95	495	458	458	458
query96	607	329	306	306
query97	4292	4114	4146	4114
query98	214	189	180	180
query99	1018	739	736	736
Total cold run time: 288196 ms
Total hot run time: 176495 ms

doris-robot · 2024-01-26T16:43:45Z

ClickBench: Total hot run time: 30.63 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34dd8654599456cbf726c7802773576283699b17, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.68	0.10	0.10
query5	0.54	0.52	0.52
query6	1.28	0.63	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.49
query10	0.55	0.56	0.57
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.60	0.62	0.60
query14	0.79	0.82	0.80
query15	0.80	0.77	0.77
query16	0.38	0.39	0.38
query17	0.97	1.02	1.00
query18	0.24	0.27	0.25
query19	1.90	1.78	1.73
query20	0.02	0.01	0.01
query21	15.43	0.58	0.55
query22	2.92	2.63	1.84
query23	17.19	0.78	0.76
query24	2.62	1.30	0.58
query25	0.47	0.23	0.25
query26	0.39	0.15	0.14
query27	0.06	0.06	0.05
query28	11.59	0.80	0.77
query29	12.56	3.22	3.19
query30	0.53	0.49	0.46
query31	2.78	0.35	0.36
query32	3.37	0.49	0.48
query33	3.20	3.26	3.24
query34	15.71	4.24	4.31
query35	4.34	4.23	4.27
query36	1.11	1.08	1.07
query37	0.06	0.05	0.05
query38	0.03	0.02	0.03
query39	0.02	0.02	0.02
query40	0.15	0.14	0.13
query41	0.08	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.56 s
Total hot run time: 30.63 s

doris-robot · 2024-01-26T16:46:51Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 34dd8654599456cbf726c7802773576283699b17 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.3 seconds inserted 10000000 Rows, about 699K ops/s

zclllyybb · 2024-01-29T07:06:53Z

run buildall

github-actions

clang-tidy made some suggestions

be/src/vec/runtime/vdatetime_value.cpp

doris-robot · 2024-01-29T08:11:17Z

TPC-H: Total hot run time: 37654 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c800387b4cd93a4729de3a76e75d01e305170daa, data reload: false

------ Round 1 ----------------------------------
q1	17658	5706	4898	4898
q2	2050	144	138	138
q3	10680	944	962	944
q4	4664	764	745	745
q5	7682	2836	2765	2765
q6	189	121	121	121
q7	1166	733	720	720
q8	9326	2023	2015	2015
q9	7195	6379	6341	6341
q10	8096	2461	2440	2440
q11	419	211	206	206
q12	773	298	294	294
q13	18017	3314	3293	3293
q14	272	247	257	247
q15	541	493	477	477
q16	480	406	405	405
q17	937	556	511	511
q18	7007	6260	6075	6075
q19	1565	1363	1296	1296
q20	637	349	321	321
q21	7076	3210	3100	3100
q22	797	302	308	302
Total cold run time: 107227 ms
Total hot run time: 37654 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5131	5061	4906	4906
q2	337	231	239	231
q3	3039	2695	2690	2690
q4	1834	1570	1480	1480
q5	5386	5312	5258	5258
q6	200	116	116	116
q7	2152	1862	1764	1764
q8	3127	3267	3180	3180
q9	8697	8536	11095	8536
q10	5397	3452	3506	3452
q11	549	473	471	471
q12	754	599	580	580
q13	16936	3113	3094	3094
q14	290	255	264	255
q15	528	485	482	482
q16	520	466	463	463
q17	2041	1625	1738	1625
q18	9030	10433	9068	9068
q19	24427	1567	1502	1502
q20	4633	1930	1902	1902
q21	15866	4775	4812	4775
q22	1053	452	457	452
Total cold run time: 111927 ms
Total hot run time: 56282 ms

doris-robot · 2024-01-29T08:22:23Z

TPC-DS: Total hot run time: 173056 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c800387b4cd93a4729de3a76e75d01e305170daa, data reload: false

query1	941	338	326	326
query2	6586	1995	1884	1884
query3	6699	206	195	195
query4	33012	22160	21964	21964
query5	4455	364	360	360
query6	240	158	164	158
query7	4605	261	255	255
query8	256	170	176	170
query9	8492	2263	2250	2250
query10	408	203	195	195
query11	17893	15324	15263	15263
query12	121	70	66	66
query13	1682	375	363	363
query14	9274	6496	6424	6424
query15	208	185	181	181
query16	5696	258	244	244
query17	960	469	467	467
query18	1803	255	247	247
query19	173	128	125	125
query20	73	71	62	62
query21	194	147	125	125
query22	4881	4720	4742	4720
query23	31225	30628	30473	30473
query24	11696	2746	2744	2744
query25	599	300	302	300
query26	1764	141	137	137
query27	3185	280	275	275
query28	7705	1826	1801	1801
query29	1841	628	623	623
query30	286	136	133	133
query31	902	711	717	711
query32	78	52	47	47
query33	687	216	204	204
query34	1160	449	461	449
query35	860	758	774	758
query36	1349	1202	1104	1104
query37	91	57	63	57
query38	3323	3146	3159	3146
query39	1321	1256	1246	1246
query40	201	91	83	83
query41	37	35	34	34
query42	89	80	84	80
query43	506	465	503	465
query44	1083	690	692	690
query45	192	176	176	176
query46	1039	659	647	647
query47	1543	1461	1544	1461
query48	400	296	296	296
query49	1131	283	266	266
query50	678	299	310	299
query51	5271	5281	5108	5108
query52	96	85	77	77
query53	328	250	262	250
query54	249	184	189	184
query55	79	82	72	72
query56	183	157	152	152
query57	987	917	909	909
query58	180	160	156	156
query59	2660	2578	2553	2553
query60	205	181	178	178
query61	81	88	84	84
query62	614	369	344	344
query63	281	271	261	261
query64	4843	1779	1729	1729
query65	3308	3233	3233	3233
query66	1264	319	308	308
query67	14379	14197	14343	14197
query68	3875	493	493	493
query69	445	290	288	288
query70	1549	1506	1513	1506
query71	268	207	224	207
query72	3749	2837	2820	2820
query73	680	314	310	310
query74	6729	6302	6282	6282
query75	2821	2291	2261	2261
query76	2639	1054	1019	1019
query77	367	227	231	227
query78	9042	8785	8646	8646
query79	2040	507	477	477
query80	1599	316	322	316
query81	526	196	202	196
query82	678	90	79	79
query83	221	111	116	111
query84	272	79	75	75
query85	2136	342	336	336
query86	526	392	360	360
query87	3457	3260	3302	3260
query88	3837	2182	2179	2179
query89	430	361	345	345
query90	1867	189	182	182
query91	146	115	115	115
query92	50	44	44	44
query93	2137	429	429	429
query94	1391	160	158	158
query95	505	447	451	447
query96	632	317	317	317
query97	4292	4173	4155	4155
query98	203	186	186	186
query99	1013	686	701	686
Total cold run time: 278753 ms
Total hot run time: 173056 ms

doris-robot · 2024-01-29T08:27:48Z

ClickBench: Total hot run time: 30.82 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c800387b4cd93a4729de3a76e75d01e305170daa, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.69	0.10	0.09
query5	0.53	0.50	0.52
query6	1.18	0.63	0.64
query7	0.02	0.02	0.01
query8	0.03	0.03	0.03
query9	0.56	0.47	0.47
query10	0.54	0.53	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.60	0.60	0.61
query14	0.79	0.80	0.78
query15	0.81	0.77	0.78
query16	0.39	0.38	0.38
query17	1.04	1.03	0.99
query18	0.21	0.26	0.21
query19	1.88	1.77	1.77
query20	0.02	0.01	0.01
query21	15.42	0.58	0.58
query22	2.38	2.46	1.60
query23	17.04	1.05	0.76
query24	2.32	1.09	1.38
query25	0.30	0.25	0.18
query26	0.54	0.14	0.15
query27	0.05	0.04	0.04
query28	11.38	0.86	0.84
query29	12.52	3.12	3.25
query30	0.65	0.59	0.51
query31	2.79	0.36	0.36
query32	3.34	0.48	0.49
query33	3.23	3.22	3.22
query34	15.85	4.31	4.27
query35	4.28	4.35	4.26
query36	1.11	1.05	1.06
query37	0.07	0.06	0.05
query38	0.04	0.03	0.03
query39	0.03	0.02	0.01
query40	0.17	0.13	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.48 s
Total hot run time: 30.82 s

doris-robot · 2024-01-29T08:30:52Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c800387b4cd93a4729de3a76e75d01e305170daa with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       15.1 seconds inserted 10000000 Rows, about 662K ops/s

zclllyybb · 2024-01-29T14:32:35Z

run buildall

github-actions

clang-tidy made some suggestions

be/src/vec/runtime/vdatetime_value.cpp

doris-robot · 2024-01-29T14:53:09Z

TeamCity be ut coverage result:
Function Coverage: 36.19% (8621/23823)
Line Coverage: 28.25% (70526/249613)
Region Coverage: 27.23% (36361/133516)
Branch Coverage: 24.03% (18635/77536)
Coverage Report: http://coverage.selectdb-in.cc/coverage/903f069511052fe70a014cd5473fb087214110fd_903f069511052fe70a014cd5473fb087214110fd/report/index.html

doris-robot · 2024-01-29T15:03:23Z

TPC-H: Total hot run time: 37351 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 903f069511052fe70a014cd5473fb087214110fd, data reload: false

------ Round 1 ----------------------------------
q1	17634	5732	4862	4862
q2	2049	137	133	133
q3	10698	915	918	915
q4	4688	654	749	654
q5	7689	2797	2840	2797
q6	185	118	118	118
q7	1108	719	717	717
q8	9328	1956	1991	1956
q9	7204	6283	6300	6283
q10	8089	2454	2403	2403
q11	416	211	211	211
q12	781	273	289	273
q13	17987	3320	3303	3303
q14	273	245	242	242
q15	516	492	511	492
q16	459	403	407	403
q17	919	511	563	511
q18	7192	6246	6007	6007
q19	1535	1380	1439	1380
q20	570	339	321	321
q21	6929	3133	3082	3082
q22	799	297	288	288
Total cold run time: 107048 ms
Total hot run time: 37351 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5170	4979	4804	4804
q2	351	232	232	232
q3	3008	2692	2660	2660
q4	1828	1599	1538	1538
q5	5383	5113	5370	5113
q6	198	115	115	115
q7	2127	1830	1783	1783
q8	3122	3182	3261	3182
q9	8785	8698	19180	8698
q10	6021	3522	3521	3521
q11	557	476	480	476
q12	766	634	625	625
q13	16961	3135	3080	3080
q14	293	247	284	247
q15	546	506	490	490
q16	521	499	477	477
q17	2071	1661	1643	1643
q18	8881	8536	11520	8536
q19	24540	1588	1523	1523
q20	5156	1989	1956	1956
q21	14955	5129	5208	5129
q22	2376	501	564	501
Total cold run time: 113616 ms
Total hot run time: 56329 ms

doris-robot · 2024-01-29T15:14:32Z

TPC-DS: Total hot run time: 172764 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 903f069511052fe70a014cd5473fb087214110fd, data reload: false

query1	940	331	328	328
query2	6559	1955	1917	1917
query3	6697	202	199	199
query4	32991	22174	22172	22172
query5	4457	426	342	342
query6	251	157	156	156
query7	4605	256	256	256
query8	250	192	172	172
query9	8588	2264	2255	2255
query10	415	213	212	212
query11	17974	15505	15359	15359
query12	115	68	65	65
query13	1669	365	376	365
query14	9185	6318	6446	6318
query15	220	183	183	183
query16	5704	252	257	252
query17	942	466	464	464
query18	1805	256	254	254
query19	168	133	128	128
query20	73	74	69	69
query21	202	137	133	133
query22	4998	4819	4712	4712
query23	31320	30400	30374	30374
query24	12440	2731	2761	2731
query25	591	321	300	300
query26	2000	142	133	133
query27	3114	278	286	278
query28	7609	1813	1812	1812
query29	2055	600	616	600
query30	287	138	142	138
query31	904	694	715	694
query32	73	55	49	49
query33	692	206	215	206
query34	1129	440	451	440
query35	855	778	763	763
query36	1286	1219	1149	1149
query37	100	60	61	60
query38	3260	3159	3140	3140
query39	1288	1251	1239	1239
query40	336	85	85	85
query41	48	35	34	34
query42	89	80	79	79
query43	501	484	488	484
query44	1056	685	686	685
query45	195	174	174	174
query46	1048	650	640	640
query47	1569	1510	1495	1495
query48	387	305	303	303
query49	1205	285	272	272
query50	667	300	307	300
query51	5230	5120	5110	5110
query52	86	90	78	78
query53	314	250	257	250
query54	247	183	182	182
query55	79	76	74	74
query56	182	167	158	158
query57	1001	858	902	858
query58	182	149	156	149
query59	2737	2486	2604	2486
query60	199	173	175	173
query61	83	82	83	82
query62	657	368	344	344
query63	286	262	270	262
query64	5988	1750	1713	1713
query65	3313	3234	3221	3221
query66	1332	317	312	312
query67	14688	13912	14300	13912
query68	4089	489	493	489
query69	419	294	291	291
query70	1555	1519	1542	1519
query71	285	209	201	201
query72	3768	2833	2847	2833
query73	681	312	306	306
query74	6720	6268	6255	6255
query75	2819	2384	2310	2310
query76	2622	1047	999	999
query77	454	224	223	223
query78	9137	8703	8645	8645
query79	2367	479	490	479
query80	1422	318	315	315
query81	505	193	198	193
query82	757	79	79	79
query83	237	115	115	115
query84	274	75	77	75
query85	1550	331	333	331
query86	501	394	398	394
query87	3432	3249	3301	3249
query88	3692	2143	2147	2143
query89	427	335	339	335
query90	2161	185	185	185
query91	149	141	116	116
query92	57	45	41	41
query93	1905	418	435	418
query94	1505	161	162	161
query95	514	456	458	456
query96	600	314	309	309
query97	4227	4119	4142	4119
query98	215	201	186	186
query99	1065	690	662	662
Total cold run time: 281627 ms
Total hot run time: 172764 ms

doris-robot · 2024-01-29T15:19:57Z

ClickBench: Total hot run time: 31.34 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 903f069511052fe70a014cd5473fb087214110fd, data reload: false

query1	0.04	0.02	0.03
query2	0.06	0.02	0.03
query3	0.22	0.06	0.06
query4	1.68	0.09	0.10
query5	0.53	0.52	0.51
query6	1.19	0.62	0.60
query7	0.01	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.50
query10	0.56	0.55	0.55
query11	0.11	0.08	0.08
query12	0.10	0.08	0.09
query13	0.60	0.61	0.62
query14	0.77	0.80	0.79
query15	0.80	0.78	0.76
query16	0.38	0.37	0.40
query17	0.99	0.99	0.96
query18	0.20	0.28	0.22
query19	1.85	1.76	1.76
query20	0.01	0.00	0.01
query21	15.40	0.59	0.56
query22	2.49	2.13	2.04
query23	17.39	0.76	0.90
query24	2.24	0.92	1.38
query25	0.38	0.17	0.12
query26	0.60	0.15	0.15
query27	0.05	0.07	0.04
query28	11.30	0.85	0.84
query29	12.71	3.38	3.39
query30	0.62	0.55	0.52
query31	2.80	0.34	0.36
query32	3.36	0.48	0.48
query33	3.20	3.24	3.19
query34	15.94	4.36	4.37
query35	4.39	4.32	4.32
query36	1.10	1.05	1.08
query37	0.07	0.05	0.05
query38	0.04	0.03	0.02
query39	0.02	0.01	0.02
query40	0.18	0.13	0.13
query41	0.08	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 105.1 s
Total hot run time: 31.34 s

doris-robot · 2024-01-29T15:23:01Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 903f069511052fe70a014cd5473fb087214110fd with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.7 seconds inserted 10000000 Rows, about 729K ops/s

github-actions · 2024-01-30T13:29:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-30T13:29:39Z

PR approved by anyone and no changes requested.

be/src/vec/runtime/vdatetime_value.cpp

zclllyybb · 2024-01-31T03:41:11Z

run buildall

github-actions

clang-tidy made some suggestions

be/src/vec/runtime/vdatetime_value.cpp

doris-robot · 2024-01-31T04:58:29Z

TeamCity be ut coverage result:
Function Coverage: 36.19% (8626/23834)
Line Coverage: 28.25% (70555/249732)
Region Coverage: 27.24% (36388/133582)
Branch Coverage: 24.02% (18633/77560)
Coverage Report: http://coverage.selectdb-in.cc/coverage/230233901fe07731409b433e63a68990c0aff245_230233901fe07731409b433e63a68990c0aff245/report/index.html

zclllyybb · 2024-01-31T06:54:14Z

run buildall

doris-robot · 2024-01-31T07:50:15Z

TPC-H: Total hot run time: 36996 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5daf0a6f2a42369160d290d51ecbfb495279facc, data reload: false

------ Round 1 ----------------------------------
q1	17658	5091	4465	4465
q2	2037	140	131	131
q3	10597	915	922	915
q4	4655	786	723	723
q5	7660	2789	2786	2786
q6	183	118	117	117
q7	1135	727	718	718
q8	9260	1995	2035	1995
q9	7235	6351	6317	6317
q10	8104	2443	2446	2443
q11	426	197	205	197
q12	772	271	267	267
q13	18015	3298	3259	3259
q14	282	257	251	251
q15	517	490	486	486
q16	466	439	411	411
q17	931	591	508	508
q18	6850	6013	5984	5984
q19	1557	1382	1372	1372
q20	608	321	313	313
q21	6561	3044	3181	3044
q22	804	317	294	294
Total cold run time: 106313 ms
Total hot run time: 36996 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4464	4299	4347	4299
q2	328	224	230	224
q3	2991	2890	2880	2880
q4	1842	1639	1648	1639
q5	5168	5247	5217	5217
q6	192	112	113	112
q7	2154	1781	1743	1743
q8	3087	3303	3293	3293
q9	8302	8378	8333	8333
q10	5926	3556	3569	3556
q11	529	472	462	462
q12	740	573	581	573
q13	12999	3091	3043	3043
q14	276	250	268	250
q15	529	479	492	479
q16	518	476	461	461
q17	1849	1678	1665	1665
q18	8107	7596	7571	7571
q19	8883	1520	1550	1520
q20	2101	1901	1897	1897
q21	4937	4564	4465	4465
q22	560	474	446	446
Total cold run time: 76482 ms
Total hot run time: 54128 ms

doris-robot · 2024-01-31T07:58:06Z

TeamCity be ut coverage result:
Function Coverage: 36.17% (8623/23838)
Line Coverage: 28.24% (70528/249767)
Region Coverage: 27.24% (36385/133591)
Branch Coverage: 24.02% (18628/77566)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5daf0a6f2a42369160d290d51ecbfb495279facc_5daf0a6f2a42369160d290d51ecbfb495279facc/report/index.html

doris-robot · 2024-01-31T08:01:22Z

TPC-DS: Total hot run time: 175469 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5daf0a6f2a42369160d290d51ecbfb495279facc, data reload: false

query1	926	332	327	327
query2	6575	1975	2032	1975
query3	6698	214	206	206
query4	32060	22144	22175	22144
query5	4456	408	372	372
query6	245	164	160	160
query7	4606	269	255	255
query8	248	171	172	171
query9	9001	2257	2255	2255
query10	410	231	199	199
query11	18402	15387	15314	15314
query12	127	70	64	64
query13	1617	365	367	365
query14	9982	7206	7497	7206
query15	235	173	188	173
query16	7346	267	253	253
query17	1817	491	467	467
query18	1932	261	250	250
query19	316	136	130	130
query20	71	71	79	71
query21	200	130	125	125
query22	5017	4856	4707	4707
query23	31707	30412	30274	30274
query24	8952	2756	2721	2721
query25	517	316	317	316
query26	728	139	136	136
query27	2210	280	277	277
query28	5581	1847	1830	1830
query29	920	630	632	630
query30	277	131	136	131
query31	901	720	712	712
query32	93	60	55	55
query33	520	217	208	208
query34	816	449	458	449
query35	835	753	786	753
query36	1230	1173	1061	1061
query37	93	58	60	58
query38	3293	3183	3149	3149
query39	1311	1252	1244	1244
query40	196	87	94	87
query41	39	34	34	34
query42	88	86	85	85
query43	504	482	488	482
query44	1007	680	693	680
query45	191	183	173	173
query46	1033	639	622	622
query47	1615	1492	1500	1492
query48	390	319	306	306
query49	1126	285	275	275
query50	676	316	318	316
query51	5321	5126	5159	5126
query52	85	86	80	80
query53	324	258	257	257
query54	215	182	181	181
query55	81	76	83	76
query56	177	162	161	161
query57	983	915	933	915
query58	187	157	148	148
query59	2436	2338	2469	2338
query60	197	169	176	169
query61	87	87	86	86
query62	606	342	370	342
query63	285	254	261	254
query64	4720	3678	3170	3170
query65	3250	3212	3216	3212
query66	943	319	309	309
query67	14357	14175	14389	14175
query68	4422	482	497	482
query69	442	326	309	309
query70	1467	1557	1528	1528
query71	299	216	214	214
query72	5806	3144	2814	2814
query73	688	326	318	318
query74	6595	6359	6264	6264
query75	3000	2270	2268	2268
query76	2540	928	910	910
query77	367	234	231	231
query78	9151	8872	8762	8762
query79	2144	510	500	500
query80	1428	329	308	308
query81	535	203	200	200
query82	606	82	86	82
query83	244	121	124	121
query84	230	75	76	75
query85	1609	341	342	341
query86	499	420	407	407
query87	3440	3254	3257	3254
query88	3846	2284	2286	2284
query89	416	336	360	336
query90	1910	190	194	190
query91	154	118	123	118
query92	57	46	42	42
query93	2401	431	432	431
query94	1315	165	158	158
query95	498	456	447	447
query96	655	345	326	326
query97	4217	4128	4114	4114
query98	207	197	181	181
query99	1049	710	735	710
Total cold run time: 275627 ms
Total hot run time: 175469 ms

doris-robot · 2024-01-31T08:06:48Z

ClickBench: Total hot run time: 30.39 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5daf0a6f2a42369160d290d51ecbfb495279facc, data reload: false

query1	0.04	0.04	0.02
query2	0.06	0.03	0.02
query3	0.22	0.06	0.07
query4	1.67	0.10	0.10
query5	0.53	0.51	0.52
query6	1.22	0.63	0.66
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.54	0.50	0.50
query10	0.56	0.56	0.54
query11	0.11	0.09	0.09
query12	0.12	0.08	0.10
query13	0.60	0.60	0.62
query14	0.78	0.80	0.80
query15	0.79	0.81	0.77
query16	0.38	0.40	0.39
query17	1.01	1.04	1.00
query18	0.22	0.29	0.21
query19	1.86	1.79	1.79
query20	0.02	0.01	0.01
query21	15.40	0.58	0.56
query22	2.42	2.28	1.97
query23	17.39	0.78	0.78
query24	2.42	1.64	0.24
query25	0.36	0.20	0.12
query26	0.50	0.14	0.14
query27	0.05	0.05	0.06
query28	11.79	0.88	0.83
query29	12.55	3.09	3.16
query30	0.65	0.49	0.55
query31	2.78	0.35	0.36
query32	3.37	0.48	0.49
query33	3.23	3.23	3.25
query34	15.70	4.40	4.33
query35	4.32	4.34	4.29
query36	1.10	1.07	1.05
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.12	0.13
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.21 s
Total hot run time: 30.39 s

doris-robot · 2024-01-31T08:09:51Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5daf0a6f2a42369160d290d51ecbfb495279facc with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

HappenLee

LGTM

github-actions · 2024-02-01T07:18:08Z

PR approved by at least one committer and no changes requested.

…30452)

…pache#30452)

github-actions bot reviewed Jan 26, 2024

View reviewed changes

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Outdated Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Outdated Show resolved Hide resolved

zclllyybb force-pushed the from_date_str branch from 34dd865 to c800387 Compare January 29, 2024 07:06

github-actions bot reviewed Jan 29, 2024

View reviewed changes

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

zclllyybb added 3 commits January 29, 2024 20:11

base ok

64d19e7

fix cases

92b0ccb

fix cases

903f069

zclllyybb force-pushed the from_date_str branch from c800387 to 903f069 Compare January 29, 2024 14:29

github-actions bot reviewed Jan 29, 2024

View reviewed changes

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

Gabriel39 previously approved these changes Jan 30, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 30, 2024

github-actions bot added the reviewed label Jan 30, 2024

HappenLee reviewed Jan 31, 2024

View reviewed changes

be/src/vec/runtime/vdatetime_value.cpp Outdated Show resolved Hide resolved

zclllyybb dismissed Gabriel39’s stale review via 2302339 January 31, 2024 03:39

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 31, 2024

github-actions bot reviewed Jan 31, 2024

View reviewed changes

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

be/src/vec/runtime/vdatetime_value.cpp Show resolved Hide resolved

format

5daf0a6

zclllyybb force-pushed the from_date_str branch from 2302339 to 5daf0a6 Compare January 31, 2024 06:53

HappenLee approved these changes Feb 1, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 1, 2024

yiguolei approved these changes Feb 1, 2024

View reviewed changes

yiguolei merged commit f1961d5 into apache:master Feb 1, 2024

yiguolei pushed a commit that referenced this pull request Feb 1, 2024

[enhance](function) refactor from_format_str and support more format (#…

3315c16

…30452)

zclllyybb deleted the from_date_str branch February 1, 2024 12:15

zclllyybb added a commit to zclllyybb/doris that referenced this pull request Feb 1, 2024

[enhance](function) refactor from_format_str and support more format (a…

fe6f9a2

…pache#30452)

zclllyybb mentioned this pull request Feb 1, 2024

[branch-2.0](datetime) refactor and support more format for str_to_date #30719

Merged

yiguolei added the dev/2.0.5-merged label Feb 7, 2024

[enhance](function) refactor from_format_str and support more format #30452

[enhance](function) refactor from_format_str and support more format #30452

Uh oh!

Conversation

zclllyybb commented Jan 26, 2024

Proposed changes

Further comments

Uh oh!

zclllyybb commented Jan 26, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Jan 26, 2024

Uh oh!

doris-robot commented Jan 26, 2024

Uh oh!

doris-robot commented Jan 26, 2024

Uh oh!

doris-robot commented Jan 26, 2024

Uh oh!

zclllyybb commented Jan 29, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

zclllyybb commented Jan 29, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

doris-robot commented Jan 29, 2024

Uh oh!

github-actions bot commented Jan 30, 2024

Uh oh!

github-actions bot commented Jan 30, 2024

Uh oh!

Uh oh!

zclllyybb commented Jan 31, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Jan 31, 2024

Uh oh!

zclllyybb commented Jan 31, 2024

Uh oh!

doris-robot commented Jan 31, 2024

Uh oh!

doris-robot commented Jan 31, 2024

Uh oh!

doris-robot commented Jan 31, 2024

Uh oh!

doris-robot commented Jan 31, 2024