Skip to content

[Enhancement] (nereids)implement CreateResourceCommand in nereids #48424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
starocean999 merged 1 commit into from
Apr 22, 2025
Merged

[Enhancement] (nereids)implement CreateResourceCommand in nereids #48424

starocean999 merged 1 commit into from
Apr 22, 2025

Conversation

@codeDing18
Copy link
Contributor

@codeDing18 codeDing18 commented Feb 27, 2025

What problem does this PR solve?

Issue Number: close #42587

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@codeDing18 codeDing18 force-pushed the create-resource-nereids branch from 2096288 to e05d4ff Compare February 27, 2025 08:09
@codeDing18
Copy link
Contributor Author

codeDing18 commented Feb 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 27, 2025

TPC-H: Total hot run time: 31426 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e05d4ffab378c629d3878165669f74b2b63fc989, data reload: false

------ Round 1 ----------------------------------
q1	17579	5291	5084	5084
q2	2052	319	178	178
q3	10381	1237	738	738
q4	10214	1030	534	534
q5	7510	2371	2349	2349
q6	190	166	133	133
q7	1009	749	614	614
q8	9323	1246	1094	1094
q9	4874	4500	4725	4500
q10	6824	2316	1906	1906
q11	470	281	266	266
q12	350	350	223	223
q13	17749	3735	3083	3083
q14	224	237	207	207
q15	511	460	443	443
q16	620	628	577	577
q17	587	851	347	347
q18	6501	6118	6109	6109
q19	1222	950	553	553
q20	315	334	203	203
q21	2879	2208	1979	1979
q22	363	341	306	306
Total cold run time: 101747 ms
Total hot run time: 31426 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5459	5172	5150	5150
q2	238	328	240	240
q3	2145	2702	2304	2304
q4	1452	1833	1356	1356
q5	4248	4150	4146	4146
q6	210	162	129	129
q7	1911	1840	1652	1652
q8	2586	2651	2553	2553
q9	7197	7197	7122	7122
q10	3126	3202	2760	2760
q11	575	525	482	482
q12	668	782	633	633
q13	3528	3809	3370	3370
q14	284	309	275	275
q15	497	472	457	457
q16	646	695	662	662
q17	1135	1588	1363	1363
q18	7663	7309	7197	7197
q19	855	845	854	845
q20	1951	2003	1908	1908
q21	5445	5015	4911	4911
q22	613	607	567	567
Total cold run time: 52432 ms
Total hot run time: 50082 ms

@doris-robot
Copy link

doris-robot commented Feb 27, 2025

TPC-DS: Total hot run time: 191414 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e05d4ffab378c629d3878165669f74b2b63fc989, data reload: false

query1	1306	938	971	938
query2	6222	1893	1901	1893
query3	10973	4362	4574	4362
query4	57380	25820	23768	23768
query5	5161	522	498	498
query6	383	194	195	194
query7	5086	492	303	303
query8	317	248	228	228
query9	6866	2593	2594	2593
query10	409	304	254	254
query11	15304	15022	15095	15022
query12	158	117	107	107
query13	1213	556	421	421
query14	10191	6972	6764	6764
query15	213	192	204	192
query16	7222	648	467	467
query17	1090	718	563	563
query18	1666	402	307	307
query19	198	201	165	165
query20	137	128	122	122
query21	226	121	105	105
query22	4309	4565	4433	4433
query23	33813	33288	33249	33249
query24	5728	2465	2461	2461
query25	469	460	396	396
query26	706	282	159	159
query27	1720	495	337	337
query28	2834	2483	2455	2455
query29	578	548	461	461
query30	214	206	176	176
query31	881	871	826	826
query32	76	64	62	62
query33	460	369	325	325
query34	795	870	509	509
query35	824	863	749	749
query36	955	1015	933	933
query37	129	105	80	80
query38	4180	4289	4184	4184
query39	1482	1467	1455	1455
query40	214	122	109	109
query41	58	56	59	56
query42	131	123	108	108
query43	513	529	481	481
query44	1390	825	822	822
query45	179	187	175	175
query46	941	1074	678	678
query47	1840	1869	1772	1772
query48	384	422	315	315
query49	760	513	416	416
query50	746	782	434	434
query51	4234	4352	4222	4222
query52	106	105	102	102
query53	233	283	195	195
query54	497	512	430	430
query55	86	79	79	79
query56	284	282	260	260
query57	1143	1179	1161	1161
query58	247	246	254	246
query59	2770	3032	2752	2752
query60	308	264	275	264
query61	119	119	117	117
query62	760	757	672	672
query63	230	205	198	198
query64	1797	1052	684	684
query65	3316	3283	3215	3215
query66	779	396	302	302
query67	15724	15438	15059	15059
query68	7695	886	490	490
query69	558	305	263	263
query70	1176	1140	1101	1101
query71	513	298	277	277
query72	5864	3611	3728	3611
query73	1288	715	355	355
query74	9006	9195	8857	8857
query75	3681	3165	2723	2723
query76	4251	1194	786	786
query77	610	358	291	291
query78	9953	10117	9180	9180
query79	2780	817	600	600
query80	688	616	448	448
query81	516	285	243	243
query82	611	127	96	96
query83	308	174	153	153
query84	289	94	73	73
query85	802	356	303	303
query86	423	328	303	303
query87	4428	4513	4444	4444
query88	3634	2225	2207	2207
query89	451	320	287	287
query90	1815	198	193	193
query91	141	140	111	111
query92	71	60	63	60
query93	2043	1023	569	569
query94	667	410	292	292
query95	343	269	255	255
query96	505	560	267	267
query97	3355	3345	3276	3276
query98	229	204	198	198
query99	1474	1440	1262	1262
Total cold run time: 301831 ms
Total hot run time: 191414 ms

@doris-robot
Copy link

doris-robot commented Feb 27, 2025

ClickBench: Total hot run time: 31.01 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e05d4ffab378c629d3878165669f74b2b63fc989, data reload: false

query1	0.03	0.05	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.10	0.10
query5	0.55	0.56	0.55
query6	1.22	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.53	0.50
query10	0.56	0.58	0.56
query11	0.16	0.11	0.10
query12	0.15	0.12	0.11
query13	0.63	0.60	0.59
query14	2.68	2.81	2.73
query15	0.90	0.85	0.85
query16	0.38	0.39	0.38
query17	1.02	1.05	1.00
query18	0.21	0.20	0.20
query19	1.90	1.85	1.99
query20	0.01	0.02	0.01
query21	15.36	0.91	0.54
query22	0.75	1.24	0.75
query23	14.79	1.41	0.63
query24	7.31	1.18	0.91
query25	0.55	0.21	0.12
query26	0.58	0.17	0.14
query27	0.05	0.06	0.05
query28	9.44	0.86	0.43
query29	12.54	3.96	3.30
query30	0.25	0.09	0.06
query31	2.81	0.58	0.38
query32	3.24	0.55	0.46
query33	3.00	2.99	3.03
query34	15.83	5.10	4.52
query35	4.54	4.53	4.51
query36	0.66	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.18	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 105.19 s
Total hot run time: 31.01 s

@starocean999
Copy link
Contributor

starocean999 commented Mar 3, 2025

please rebase code the resolve conflicts

@codeDing18 codeDing18 force-pushed the create-resource-nereids branch from e05d4ff to faa4e32 Compare March 3, 2025 08:31
@codeDing18
Copy link
Contributor Author

codeDing18 commented Mar 3, 2025

run buildall

@codeDing18
Copy link
Contributor Author

codeDing18 commented Mar 3, 2025

@starocean999 I have rebased.Please help review

@doris-robot
Copy link

doris-robot commented Mar 3, 2025

TPC-H: Total hot run time: 31760 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit faa4e32b1f90c63fa085a7c228dae0c314ec2423, data reload: false

------ Round 1 ----------------------------------
q1	17590	5149	5093	5093
q2	2059	301	175	175
q3	10403	1284	717	717
q4	10223	1028	540	540
q5	7517	2344	2392	2344
q6	191	165	134	134
q7	896	765	608	608
q8	9313	1378	1134	1134
q9	5124	4625	4660	4625
q10	6814	2295	1899	1899
q11	472	284	269	269
q12	347	361	235	235
q13	17774	3787	3089	3089
q14	228	226	200	200
q15	503	479	460	460
q16	630	609	570	570
q17	591	858	336	336
q18	6583	6428	6261	6261
q19	1223	971	586	586
q20	318	322	185	185
q21	2737	2162	1994	1994
q22	365	341	306	306
Total cold run time: 101901 ms
Total hot run time: 31760 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5126	5102	5135	5102
q2	227	333	238	238
q3	2185	2706	2298	2298
q4	1447	1832	1377	1377
q5	4255	4150	4143	4143
q6	205	163	123	123
q7	1874	1837	1673	1673
q8	2587	2504	2608	2504
q9	7295	7299	7285	7285
q10	2925	3162	2764	2764
q11	583	526	505	505
q12	688	774	616	616
q13	3560	3890	3269	3269
q14	290	281	284	281
q15	544	484	455	455
q16	670	679	646	646
q17	1164	1586	1363	1363
q18	7730	7499	7279	7279
q19	806	843	964	843
q20	1998	2082	1796	1796
q21	5250	5014	4785	4785
q22	618	584	538	538
Total cold run time: 52027 ms
Total hot run time: 49883 ms

@doris-robot
Copy link

doris-robot commented Mar 3, 2025

TPC-DS: Total hot run time: 191174 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit faa4e32b1f90c63fa085a7c228dae0c314ec2423, data reload: false

query1	1314	950	931	931
query2	6279	1947	1955	1947
query3	10948	4647	4533	4533
query4	53203	26184	23540	23540
query5	5124	575	511	511
query6	348	199	193	193
query7	4946	530	296	296
query8	324	251	244	244
query9	6153	2569	2573	2569
query10	437	331	263	263
query11	15194	15254	14920	14920
query12	162	114	117	114
query13	1105	557	377	377
query14	11261	6776	6873	6776
query15	211	220	188	188
query16	7160	671	496	496
query17	1087	779	592	592
query18	1535	431	314	314
query19	204	214	183	183
query20	132	129	127	127
query21	214	129	106	106
query22	4365	4432	4314	4314
query23	33939	33434	33318	33318
query24	5698	2470	2454	2454
query25	468	482	418	418
query26	726	281	162	162
query27	1799	527	343	343
query28	2872	2432	2449	2432
query29	578	596	441	441
query30	212	192	164	164
query31	882	878	835	835
query32	83	66	64	64
query33	460	389	320	320
query34	787	871	516	516
query35	828	842	776	776
query36	973	1009	902	902
query37	121	102	77	77
query38	4305	4229	4306	4229
query39	1483	1434	1435	1434
query40	211	117	106	106
query41	80	54	56	54
query42	139	111	116	111
query43	537	534	504	504
query44	1369	817	811	811
query45	189	175	166	166
query46	915	1074	673	673
query47	1832	1838	1759	1759
query48	393	424	306	306
query49	727	523	429	429
query50	745	800	449	449
query51	4275	4304	4251	4251
query52	111	114	105	105
query53	245	271	194	194
query54	502	507	452	452
query55	96	83	83	83
query56	279	286	282	282
query57	1171	1199	1101	1101
query58	244	266	279	266
query59	3013	3092	2693	2693
query60	280	282	264	264
query61	127	117	122	117
query62	712	698	693	693
query63	238	198	192	192
query64	1964	1030	741	741
query65	3217	3147	3142	3142
query66	742	397	299	299
query67	15640	15458	15205	15205
query68	6529	902	503	503
query69	539	303	272	272
query70	1197	1145	1112	1112
query71	481	295	271	271
query72	5970	3386	3798	3386
query73	1466	769	354	354
query74	8976	8986	8732	8732
query75	3728	3160	2712	2712
query76	4182	1171	763	763
query77	630	395	299	299
query78	10022	10171	9283	9283
query79	2294	854	605	605
query80	634	531	455	455
query81	495	278	244	244
query82	678	129	96	96
query83	171	173	162	162
query84	289	95	73	73
query85	784	382	317	317
query86	369	311	283	283
query87	4402	4399	4469	4399
query88	3510	2239	2223	2223
query89	410	328	288	288
query90	1880	195	199	195
query91	145	145	111	111
query92	72	70	61	61
query93	1242	1072	566	566
query94	680	422	294	294
query95	353	273	261	261
query96	494	569	267	267
query97	3350	3407	3245	3245
query98	233	213	206	206
query99	1388	1417	1269	1269
Total cold run time: 295506 ms
Total hot run time: 191174 ms

@doris-robot
Copy link

doris-robot commented Mar 3, 2025

ClickBench: Total hot run time: 31.07 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit faa4e32b1f90c63fa085a7c228dae0c314ec2423, data reload: false

query1	0.04	0.06	0.03
query2	0.08	0.03	0.03
query3	0.24	0.07	0.06
query4	1.61	0.10	0.11
query5	0.57	0.54	0.55
query6	1.20	0.72	0.72
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.53	0.51
query10	0.56	0.57	0.58
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.61	0.60	0.59
query14	2.78	2.69	2.70
query15	0.92	0.85	0.84
query16	0.38	0.38	0.38
query17	1.01	1.04	1.08
query18	0.20	0.19	0.19
query19	1.83	1.83	1.95
query20	0.01	0.01	0.01
query21	15.36	0.89	0.54
query22	0.76	1.31	0.70
query23	14.75	1.39	0.58
query24	6.83	2.01	1.16
query25	0.50	0.11	0.15
query26	0.59	0.17	0.13
query27	0.05	0.06	0.05
query28	9.43	0.80	0.42
query29	12.55	3.95	3.33
query30	0.25	0.09	0.06
query31	2.81	0.59	0.37
query32	3.22	0.56	0.47
query33	2.97	3.00	3.00
query34	15.63	5.09	4.49
query35	4.53	4.59	4.50
query36	0.66	0.51	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.14	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 104.36 s
Total hot run time: 31.07 s

@codeDing18 codeDing18 force-pushed the create-resource-nereids branch from faa4e32 to 0458b50 Compare April 16, 2025 04:22
@codeDing18
Copy link
Contributor Author

codeDing18 commented Apr 16, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 16, 2025

TPC-H: Total hot run time: 34039 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0458b501b1c7ef8895df54134b0c00b8ab3b4855, data reload: false

------ Round 1 ----------------------------------
q1	25966	5109	5004	5004
q2	2057	287	196	196
q3	10402	1283	666	666
q4	10227	1005	518	518
q5	7550	2290	2382	2290
q6	186	165	136	136
q7	901	766	606	606
q8	9304	1277	1060	1060
q9	6785	5115	5119	5115
q10	6803	2312	1880	1880
q11	474	279	284	279
q12	348	355	219	219
q13	17790	3763	3190	3190
q14	236	235	220	220
q15	549	502	496	496
q16	455	458	414	414
q17	617	861	391	391
q18	7819	7312	7228	7228
q19	1304	972	571	571
q20	336	345	228	228
q21	4065	3386	2389	2389
q22	1033	1011	943	943
Total cold run time: 115207 ms
Total hot run time: 34039 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5115	5077	5103	5077
q2	237	325	237	237
q3	2145	2666	2326	2326
q4	1402	1819	1411	1411
q5	4462	4380	4396	4380
q6	224	171	128	128
q7	2002	1935	1721	1721
q8	2587	2587	2560	2560
q9	7262	7187	7124	7124
q10	2989	3176	2779	2779
q11	643	502	492	492
q12	680	766	626	626
q13	3576	3964	3358	3358
q14	292	296	264	264
q15	520	490	471	471
q16	477	501	466	466
q17	1166	1541	1412	1412
q18	7839	7592	7502	7502
q19	830	818	814	814
q20	1926	1966	1836	1836
q21	5358	4872	4883	4872
q22	1102	1041	1032	1032
Total cold run time: 52834 ms
Total hot run time: 50888 ms

@doris-robot
Copy link

doris-robot commented Apr 16, 2025

TPC-DS: Total hot run time: 192374 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0458b501b1c7ef8895df54134b0c00b8ab3b4855, data reload: false

query1	1409	1067	1034	1034
query2	6069	1784	1803	1784
query3	11099	4658	4458	4458
query4	54514	24877	23088	23088
query5	5111	475	436	436
query6	370	213	196	196
query7	4930	497	278	278
query8	330	239	237	237
query9	6082	2567	2583	2567
query10	441	335	255	255
query11	14957	15051	14870	14870
query12	163	112	105	105
query13	1068	516	401	401
query14	10187	6412	6406	6406
query15	191	212	175	175
query16	7130	711	480	480
query17	1063	775	584	584
query18	1533	404	308	308
query19	197	197	152	152
query20	127	125	115	115
query21	215	122	107	107
query22	4466	4661	4597	4597
query23	34391	33634	33386	33386
query24	6671	2445	2433	2433
query25	464	471	387	387
query26	661	274	147	147
query27	2184	508	327	327
query28	3009	2171	2136	2136
query29	576	563	430	430
query30	275	217	196	196
query31	890	845	769	769
query32	76	61	64	61
query33	444	355	304	304
query34	761	880	514	514
query35	827	847	749	749
query36	957	1011	905	905
query37	118	103	75	75
query38	4157	4194	4217	4194
query39	1544	1425	1476	1425
query40	203	120	115	115
query41	52	55	50	50
query42	132	111	110	110
query43	507	508	490	490
query44	1329	831	796	796
query45	184	182	174	174
query46	839	1074	658	658
query47	1873	1877	1779	1779
query48	377	417	314	314
query49	711	506	434	434
query50	678	714	412	412
query51	4222	4232	4221	4221
query52	111	120	104	104
query53	225	261	183	183
query54	575	579	503	503
query55	89	79	78	78
query56	305	288	291	288
query57	1165	1217	1157	1157
query58	270	248	253	248
query59	2670	2847	2745	2745
query60	345	323	298	298
query61	134	125	132	125
query62	760	766	678	678
query63	228	190	190	190
query64	1580	1135	782	782
query65	4320	4276	4244	4244
query66	738	396	304	304
query67	15744	15750	15394	15394
query68	7858	878	510	510
query69	541	298	260	260
query70	1209	1053	1107	1053
query71	486	322	273	273
query72	5825	4703	4764	4703
query73	1328	615	338	338
query74	8911	9135	8678	8678
query75	3733	3227	2679	2679
query76	4241	1200	768	768
query77	641	369	281	281
query78	10029	10137	9431	9431
query79	2697	815	560	560
query80	720	504	441	441
query81	483	249	213	213
query82	489	127	95	95
query83	385	249	227	227
query84	303	108	80	80
query85	875	343	306	306
query86	414	306	268	268
query87	4487	4488	4302	4302
query88	3570	2194	2201	2194
query89	413	314	280	280
query90	1789	204	209	204
query91	142	141	110	110
query92	74	56	52	52
query93	2240	930	581	581
query94	667	428	371	371
query95	362	294	286	286
query96	481	570	281	281
query97	3171	3245	3077	3077
query98	229	201	200	200
query99	1437	1428	1327	1327
Total cold run time: 300253 ms
Total hot run time: 192374 ms

@doris-robot
Copy link

doris-robot commented Apr 16, 2025

ClickBench: Total hot run time: 29.58 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0458b501b1c7ef8895df54134b0c00b8ab3b4855, data reload: false

query1	0.03	0.03	0.03
query2	0.12	0.12	0.11
query3	0.24	0.19	0.21
query4	1.59	0.19	0.19
query5	0.61	0.61	0.61
query6	1.19	0.70	0.72
query7	0.03	0.02	0.02
query8	0.04	0.04	0.03
query9	0.60	0.53	0.51
query10	0.57	0.57	0.55
query11	0.16	0.10	0.11
query12	0.16	0.11	0.11
query13	0.62	0.59	0.60
query14	1.18	1.19	1.21
query15	0.89	0.86	0.86
query16	0.40	0.37	0.38
query17	1.03	1.00	1.03
query18	0.21	0.20	0.19
query19	1.90	1.80	1.77
query20	0.01	0.00	0.01
query21	15.41	0.92	0.55
query22	0.75	1.16	0.62
query23	15.04	1.40	0.62
query24	7.29	0.79	0.88
query25	0.50	0.12	0.07
query26	0.64	0.18	0.13
query27	0.06	0.05	0.05
query28	9.08	0.95	0.45
query29	12.53	4.03	3.31
query30	0.25	0.10	0.07
query31	2.82	0.61	0.39
query32	3.24	0.54	0.48
query33	3.01	3.09	3.08
query34	15.80	5.14	4.54
query35	4.51	4.55	4.54
query36	0.68	0.51	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.02
Total cold run time: 103.68 s
Total hot run time: 29.58 s

@codeDing18
Copy link
Contributor Author

codeDing18 commented Apr 16, 2025

@starocean999 please help review,thanks!

@codeDing18 codeDing18 force-pushed the create-resource-nereids branch from 0458b50 to 80b6e0d Compare April 18, 2025 01:23
@codeDing18
Copy link
Contributor Author

codeDing18 commented Apr 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-H: Total hot run time: 34017 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80b6e0d26533e090ed9a56af145896839ff1d13f, data reload: false

------ Round 1 ----------------------------------
q1	26422	5030	4962	4962
q2	2070	274	188	188
q3	10525	1252	679	679
q4	10243	1025	526	526
q5	7713	2352	2367	2352
q6	182	158	131	131
q7	925	727	602	602
q8	9313	1282	1078	1078
q9	6830	5145	5147	5145
q10	6821	2270	1880	1880
q11	483	282	282	282
q12	350	353	210	210
q13	17760	3648	3084	3084
q14	228	231	205	205
q15	530	470	499	470
q16	456	451	397	397
q17	602	857	366	366
q18	7629	7253	7211	7211
q19	1922	953	581	581
q20	322	318	216	216
q21	3954	2741	2478	2478
q22	1036	1003	974	974
Total cold run time: 116316 ms
Total hot run time: 34017 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5504	5048	5002	5002
q2	248	328	230	230
q3	2170	2655	2225	2225
q4	1440	1799	1464	1464
q5	4443	4395	4458	4395
q6	222	169	124	124
q7	1988	1924	1717	1717
q8	2579	2554	2537	2537
q9	7387	7179	7177	7177
q10	2985	3180	2732	2732
q11	583	516	490	490
q12	692	738	614	614
q13	3513	3900	3287	3287
q14	294	315	261	261
q15	523	478	474	474
q16	467	495	463	463
q17	1152	1626	1376	1376
q18	7791	7610	7518	7518
q19	809	871	997	871
q20	2003	1967	1830	1830
q21	5330	4915	4768	4768
q22	1064	1052	1032	1032
Total cold run time: 53187 ms
Total hot run time: 50587 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-DS: Total hot run time: 192088 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80b6e0d26533e090ed9a56af145896839ff1d13f, data reload: false

query1	1424	1052	1033	1033
query2	6061	1792	1777	1777
query3	11024	4477	4564	4477
query4	56979	25535	23361	23361
query5	4878	454	447	447
query6	362	186	193	186
query7	4959	485	272	272
query8	304	258	227	227
query9	6220	2553	2540	2540
query10	433	316	275	275
query11	15080	14981	14771	14771
query12	157	109	100	100
query13	1083	508	378	378
query14	10105	6102	6171	6102
query15	199	201	183	183
query16	7223	666	485	485
query17	1049	722	576	576
query18	1614	397	319	319
query19	193	189	159	159
query20	122	146	123	123
query21	209	129	105	105
query22	4545	4518	4606	4518
query23	34168	33312	33502	33312
query24	6767	2398	2420	2398
query25	483	526	427	427
query26	686	281	153	153
query27	2281	507	331	331
query28	3268	2143	2151	2143
query29	584	567	469	469
query30	276	228	193	193
query31	884	885	789	789
query32	78	66	61	61
query33	444	390	305	305
query34	787	865	545	545
query35	803	837	744	744
query36	963	979	897	897
query37	115	103	79	79
query38	4194	4310	4146	4146
query39	1512	1456	1455	1455
query40	222	127	111	111
query41	58	55	55	55
query42	144	110	116	110
query43	503	506	481	481
query44	1306	810	802	802
query45	183	176	169	169
query46	842	1031	634	634
query47	1856	1885	1838	1838
query48	380	404	299	299
query49	715	541	449	449
query50	664	716	401	401
query51	4231	4224	4272	4224
query52	113	107	102	102
query53	257	257	187	187
query54	578	562	516	516
query55	89	83	82	82
query56	346	280	292	280
query57	1158	1209	1143	1143
query58	268	263	249	249
query59	2726	2790	2754	2754
query60	327	325	322	322
query61	127	126	129	126
query62	749	760	708	708
query63	231	191	190	190
query64	1897	1037	682	682
query65	4297	4217	4177	4177
query66	689	437	296	296
query67	15826	15513	15355	15355
query68	7747	872	501	501
query69	543	309	257	257
query70	1143	1087	1116	1087
query71	499	312	279	279
query72	5737	4698	4791	4698
query73	1280	607	342	342
query74	9199	9142	8999	8999
query75	3766	3272	2693	2693
query76	4212	1186	730	730
query77	592	383	280	280
query78	10068	10155	9258	9258
query79	2539	815	557	557
query80	656	497	452	452
query81	490	248	221	221
query82	456	126	96	96
query83	377	240	224	224
query84	296	97	91	91
query85	809	363	314	314
query86	417	307	261	261
query87	4404	4506	4274	4274
query88	3618	2183	2166	2166
query89	397	306	272	272
query90	1749	209	215	209
query91	152	152	155	152
query92	78	57	60	57
query93	2149	939	561	561
query94	657	383	306	306
query95	366	290	276	276
query96	478	560	267	267
query97	3164	3187	3123	3123
query98	227	198	202	198
query99	1500	1405	1284	1284
Total cold run time: 303063 ms
Total hot run time: 192088 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

ClickBench: Total hot run time: 29.04 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 80b6e0d26533e090ed9a56af145896839ff1d13f, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.26	0.19	0.20
query4	1.59	0.20	0.20
query5	0.58	0.59	0.58
query6	1.18	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.58	0.54	0.51
query10	0.57	0.57	0.57
query11	0.15	0.11	0.10
query12	0.15	0.12	0.11
query13	0.61	0.59	0.60
query14	1.16	1.20	1.17
query15	0.87	0.85	0.85
query16	0.38	0.38	0.38
query17	1.00	1.04	1.00
query18	0.21	0.20	0.20
query19	1.90	1.75	1.81
query20	0.01	0.02	0.01
query21	15.40	0.90	0.54
query22	0.75	1.27	0.77
query23	14.75	1.39	0.64
query24	7.86	0.74	0.33
query25	0.33	0.10	0.15
query26	0.71	0.17	0.14
query27	0.05	0.05	0.04
query28	8.57	0.87	0.43
query29	12.53	4.03	3.32
query30	0.25	0.10	0.06
query31	2.83	0.58	0.38
query32	3.23	0.55	0.48
query33	2.98	3.07	3.01
query34	15.63	5.12	4.45
query35	4.52	4.52	4.45
query36	0.67	0.50	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.14	0.13
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 103.01 s
Total hot run time: 29.04 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 22, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 22, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 22, 2025

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit dbc66e4 into apache:master Apr 22, 2025
25 of 26 checks passed
@morningman morningman mentioned this pull request Apr 23, 2025
Merged
16 tasks
morningman added a commit that referenced this pull request Apr 26, 2025
@morningman
[fix](create-resource) fix potential concurrent modification exceptio…
8d1ddea 
…n when creating resource (#50356)

### What problem does this PR solve?

Related PR: #48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
morningman added a commit to morningman/doris that referenced this pull request Apr 29, 2025
@morningman
[fix](create-resource) fix potential concurrent modification exceptio…
38b58b6 
…n when creating resource (apache#50356)

Related PR: apache#48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
morningman added a commit to morningman/doris that referenced this pull request May 7, 2025
@morningman
[fix](create-resource) fix potential concurrent modification exceptio…
5a0167d 
…n when creating resource (apache#50356)

Related PR: apache#48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
morningman added a commit to morningman/doris that referenced this pull request May 7, 2025
@morningman
[fix](create-resource) fix potential concurrent modification exceptio…
76fd857 
…n when creating resource (apache#50356)

Related PR: apache#48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
morningman added a commit to morningman/doris that referenced this pull request May 14, 2025
@morningman
[fix](create-resource) fix potential concurrent modification exceptio…
1c5ff56 
…n when creating resource (apache#50356)

Related PR: apache#48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
@codeDing18 codeDing18 deleted the create-resource-nereids branch May 14, 2025 09:15
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@codeDing18 @koarz
[Enhancement] (nereids)implement CreateResourceCommand in nereids (ap…
7b0956c 
…ache#48424)

Issue Number: close
[apache#42587](apache#42587)
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@morningman @koarz
[fix](create-resource) fix potential concurrent modification exceptio…
39dfdef 
…n when creating resource (apache#50356)

### What problem does this PR solve?

Related PR: apache#48424

Problem Summary:

The `CreateResourceCommandTest` may throw concurrent modification
exception.
This is because in `JdbcResource`, the origin property map may be
modified.

This PR changes；
1. Change the origin property map to ImmutableMap, to avoid potential
exception.
2. Copy the map to the jdbc resource to avoid modifying the origin map
@morningman morningman mentioned this pull request Jul 14, 2025
Merged
morningman added a commit that referenced this pull request Jul 15, 2025
@morningman
[fix](test) fix CreateResourceCommandTest (#53191)
7394120 
followup #48424
The spark resource has been removed by #48424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Enhancement] (nereids)implement CreateResourceCommand in nereids

5 participants

@codeDing18 @Thearas @doris-robot @starocean999 @englefly