Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](binlog) support rename binlog #37031

Merged
merged 4 commits into from
Jul 3, 2024
Merged

[opt](binlog) support rename binlog #37031

merged 4 commits into from
Jul 3, 2024

Conversation

lsy3993
Copy link
Contributor

@lsy3993 lsy3993 commented Jun 29, 2024 •
edited
Loading

Proposed changes

Rename operator doesn't have binlog now. This PR will create binlog when execute rename. The rename operator means :

  1. rename table : ALTER TABLE table1 RENAME table2;
  2. rename rollup : ALTER TABLE example_table RENAME ROLLUP rollup1 rollup2;
  3. rename partition : ALTER TABLE example_table RENAME PARTITION p1 p2;

After SQL analyzing, we can get the old and new table name (rollup name or partition name) , then record the info to binlog, so we can use the info from binlog.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Contributor

run buildall

@lsy3993
Copy link
Contributor Author

lsy3993 commented Jun 30, 2024

run buildall

@lsy3993
Copy link
Contributor Author

lsy3993 commented Jun 30, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40067 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

------ Round 1 ----------------------------------
q1	18291	4907	4326	4326
q2	2043	190	190	190
q3	10550	1222	1136	1136
q4	10230	789	829	789
q5	7482	2668	2630	2630
q6	217	134	134	134
q7	953	599	608	599
q8	9232	2090	2097	2090
q9	9140	6540	6495	6495
q10	8931	3736	3735	3735
q11	450	247	244	244
q12	401	246	228	228
q13	17765	3003	3006	3003
q14	273	225	218	218
q15	536	487	493	487
q16	534	386	375	375
q17	985	709	730	709
q18	8148	7496	7390	7390
q19	5942	1437	1548	1437
q20	673	324	330	324
q21	4933	3180	3989	3180
q22	419	361	348	348
Total cold run time: 118128 ms
Total hot run time: 40067 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4478	4266	4243	4243
q2	386	270	262	262
q3	3046	2929	2946	2929
q4	1976	1798	1732	1732
q5	5533	5530	5468	5468
q6	241	132	133	132
q7	2248	1867	1857	1857
q8	3284	3478	3458	3458
q9	8726	8799	8649	8649
q10	4139	3731	3769	3731
q11	617	515	509	509
q12	808	671	654	654
q13	16492	3201	3174	3174
q14	297	279	268	268
q15	527	499	476	476
q16	522	428	427	427
q17	1831	1531	1484	1484
q18	8272	7977	7835	7835
q19	1852	1601	1562	1562
q20	3087	1911	1854	1854
q21	5276	4834	4883	4834
q22	679	558	589	558
Total cold run time: 74317 ms
Total hot run time: 56096 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174928 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	932	379	378	378
query2	6568	2366	2474	2366
query3	6630	206	212	206
query4	19172	17761	17344	17344
query5	3702	495	475	475
query6	259	163	162	162
query7	4590	296	285	285
query8	312	297	294	294
query9	8475	2382	2367	2367
query10	566	297	279	279
query11	10857	10340	10103	10103
query12	113	84	92	84
query13	1647	381	370	370
query14	9743	7661	7840	7661
query15	263	192	185	185
query16	7813	275	278	275
query17	1883	556	536	536
query18	1553	289	277	277
query19	205	155	155	155
query20	91	85	85	85
query21	212	128	127	127
query22	4358	4451	4068	4068
query23	33925	33615	33520	33520
query24	10740	2853	2902	2853
query25	593	404	380	380
query26	706	160	159	159
query27	2242	332	323	323
query28	5923	2090	2095	2090
query29	900	646	646	646
query30	271	159	156	156
query31	968	751	776	751
query32	103	57	55	55
query33	737	303	294	294
query34	944	471	484	471
query35	766	645	625	625
query36	1120	983	986	983
query37	155	75	78	75
query38	2992	2824	2877	2824
query39	913	844	821	821
query40	224	132	124	124
query41	60	55	54	54
query42	110	103	106	103
query43	629	585	565	565
query44	1134	736	717	717
query45	196	168	164	164
query46	1076	707	752	707
query47	1911	1823	1802	1802
query48	367	308	307	307
query49	854	504	405	405
query50	752	382	391	382
query51	6909	6874	6695	6695
query52	103	92	91	91
query53	356	282	283	282
query54	931	442	438	438
query55	73	71	70	70
query56	284	252	265	252
query57	1110	1099	1098	1098
query58	241	237	237	237
query59	3719	3313	3098	3098
query60	295	278	286	278
query61	89	88	89	88
query62	601	476	438	438
query63	325	278	279	278
query64	8645	2258	1733	1733
query65	3165	3096	3094	3094
query66	743	327	320	320
query67	15617	15364	15109	15109
query68	8595	551	549	549
query69	741	485	421	421
query70	1411	1094	1139	1094
query71	539	280	278	278
query72	8902	5408	5651	5408
query73	2278	325	319	319
query74	5992	5612	5598	5598
query75	4869	2665	2652	2652
query76	5126	977	907	907
query77	815	302	297	297
query78	10469	9886	9766	9766
query79	9675	524	517	517
query80	1180	506	473	473
query81	553	224	216	216
query82	486	109	104	104
query83	344	169	166	166
query84	274	91	84	84
query85	988	282	267	267
query86	367	308	282	282
query87	3325	3087	3052	3052
query88	4592	2388	2337	2337
query89	523	389	379	379
query90	2014	189	188	188
query91	164	103	98	98
query92	62	49	51	49
query93	6799	534	520	520
query94	1287	187	189	187
query95	413	317	321	317
query96	607	275	267	267
query97	3236	3057	3082	3057
query98	215	211	199	199
query99	1217	843	849	843
Total cold run time: 292519 ms
Total hot run time: 174928 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.68 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	0.05	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.06	0.05
query4	1.67	0.09	0.09
query5	0.52	0.49	0.48
query6	1.13	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.54	0.50	0.50
query10	0.55	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.13	0.13
query13	0.59	0.59	0.58
query14	0.77	0.78	0.78
query15	0.86	0.82	0.82
query16	0.37	0.35	0.36
query17	1.03	0.94	1.00
query18	0.20	0.24	0.23
query19	1.76	1.83	1.73
query20	0.02	0.01	0.01
query21	15.44	0.75	0.65
query22	3.76	7.70	2.06
query23	18.28	1.48	1.30
query24	2.14	0.23	0.21
query25	0.16	0.09	0.08
query26	0.27	0.18	0.17
query27	0.08	0.07	0.08
query28	13.28	1.03	1.00
query29	12.59	3.27	3.27
query30	0.26	0.06	0.05
query31	2.91	0.39	0.40
query32	3.24	0.47	0.48
query33	2.83	2.89	2.84
query34	17.15	4.42	4.41
query35	4.49	4.48	4.50
query36	0.65	0.46	0.47
query37	0.18	0.17	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.04
query40	0.17	0.16	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 108.99 s
Total hot run time: 30.68 s

@lsy3993
Copy link
Contributor Author

lsy3993 commented Jun 30, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40014 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

------ Round 1 ----------------------------------
q1	18317	4711	4408	4408
q2	2550	194	198	194
q3	10991	1270	1220	1220
q4	10850	846	757	757
q5	7991	2743	2688	2688
q6	227	142	140	140
q7	982	603	616	603
q8	9602	2161	2081	2081
q9	8889	6501	6511	6501
q10	9037	3693	3717	3693
q11	454	240	238	238
q12	407	233	236	233
q13	17789	2945	2987	2945
q14	255	240	220	220
q15	534	482	476	476
q16	527	376	367	367
q17	982	677	672	672
q18	8102	7379	7340	7340
q19	5538	1379	1571	1379
q20	666	315	331	315
q21	4939	3205	3851	3205
q22	398	339	347	339
Total cold run time: 120027 ms
Total hot run time: 40014 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4213	4267	4213
q2	367	272	266	266
q3	2995	2684	2714	2684
q4	1846	1637	1585	1585
q5	5262	5322	5278	5278
q6	221	127	129	127
q7	2117	1748	1732	1732
q8	3230	3340	3329	3329
q9	8363	8358	8393	8358
q10	3866	3675	3632	3632
q11	575	488	493	488
q12	771	601	608	601
q13	16413	3001	2995	2995
q14	290	252	254	252
q15	530	488	481	481
q16	457	405	422	405
q17	1775	1486	1467	1467
q18	7711	7411	7478	7411
q19	1717	1581	1530	1530
q20	1992	1751	1776	1751
q21	4975	4620	4655	4620
q22	619	535	574	535
Total cold run time: 70454 ms
Total hot run time: 53740 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172690 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	920	380	374	374
query2	6454	2464	2477	2464
query3	6654	204	212	204
query4	19832	17426	17330	17330
query5	4208	483	505	483
query6	281	188	159	159
query7	4598	307	309	307
query8	316	289	285	285
query9	8479	2393	2369	2369
query10	611	299	295	295
query11	10594	10087	10022	10022
query12	129	88	86	86
query13	1645	397	371	371
query14	10394	7208	7793	7208
query15	242	189	192	189
query16	7937	280	279	279
query17	1881	581	544	544
query18	1846	291	283	283
query19	256	143	150	143
query20	99	82	79	79
query21	217	127	124	124
query22	4292	4253	4034	4034
query23	33617	32924	33073	32924
query24	11937	2805	2785	2785
query25	649	368	358	358
query26	1671	153	156	153
query27	2863	317	314	314
query28	7475	2075	2050	2050
query29	1002	616	603	603
query30	288	149	148	148
query31	944	722	729	722
query32	93	50	54	50
query33	767	292	293	292
query34	920	470	477	470
query35	774	631	610	610
query36	1063	950	882	882
query37	153	77	76	76
query38	2878	2742	2761	2742
query39	865	811	795	795
query40	276	128	126	126
query41	55	53	51	51
query42	123	103	99	99
query43	582	558	549	549
query44	1172	728	736	728
query45	191	158	159	158
query46	1081	745	714	714
query47	1859	1796	1784	1784
query48	376	305	290	290
query49	1152	431	428	428
query50	762	380	391	380
query51	6872	6805	6822	6805
query52	105	89	90	89
query53	361	299	282	282
query54	880	443	441	441
query55	74	72	73	72
query56	281	256	277	256
query57	1163	1059	1022	1022
query58	260	241	243	241
query59	3552	3093	3124	3093
query60	309	272	301	272
query61	92	89	92	89
query62	678	436	450	436
query63	321	287	289	287
query64	9858	2259	1769	1769
query65	3184	3102	3119	3102
query66	1319	341	331	331
query67	15384	15084	14771	14771
query68	4838	541	536	536
query69	582	436	332	332
query70	1173	1127	1083	1083
query71	432	289	280	280
query72	7352	5107	5443	5107
query73	758	328	324	324
query74	5978	5554	5438	5438
query75	3571	2675	2647	2647
query76	3415	1010	944	944
query77	659	304	308	304
query78	10368	9927	9820	9820
query79	2192	569	511	511
query80	1137	468	462	462
query81	563	228	218	218
query82	773	106	99	99
query83	193	165	165	165
query84	271	82	89	82
query85	1403	274	265	265
query86	465	325	305	305
query87	3271	3109	3061	3061
query88	3982	2382	2380	2380
query89	479	376	379	376
query90	1838	190	187	187
query91	134	100	101	100
query92	63	48	48	48
query93	2101	517	502	502
query94	1174	190	185	185
query95	402	317	367	317
query96	598	280	274	274
query97	3264	3063	3026	3026
query98	229	205	195	195
query99	1360	850	846	846
Total cold run time: 277184 ms
Total hot run time: 172690 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.05	0.04
query3	0.23	0.05	0.06
query4	1.66	0.10	0.10
query5	0.50	0.48	0.49
query6	1.14	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.54	0.50	0.48
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.16	0.12	0.12
query13	0.60	0.58	0.58
query14	0.76	0.80	0.79
query15	0.82	0.81	0.81
query16	0.37	0.36	0.36
query17	0.95	0.94	1.00
query18	0.21	0.26	0.25
query19	1.80	1.67	1.75
query20	0.02	0.01	0.01
query21	15.46	0.74	0.64
query22	4.02	7.17	1.77
query23	18.23	1.38	1.16
query24	2.10	0.23	0.22
query25	0.17	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.21	1.02	0.98
query29	12.61	3.32	3.30
query30	0.26	0.07	0.06
query31	2.84	0.39	0.38
query32	3.29	0.47	0.47
query33	2.86	2.87	2.95
query34	17.16	4.37	4.49
query35	4.49	4.48	4.44
query36	0.65	0.48	0.47
query37	0.18	0.15	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.17	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.1 s
Total hot run time: 30.2 s

@lsy3993
Copy link
Contributor Author

lsy3993 commented Jun 30, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40005 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

------ Round 1 ----------------------------------
q1	18651	7777	4350	4350
q2	2034	189	184	184
q3	10616	1192	1121	1121
q4	10360	805	901	805
q5	7466	2716	2651	2651
q6	226	141	136	136
q7	948	605	618	605
q8	9249	2087	2073	2073
q9	9025	6524	6479	6479
q10	8993	3677	3699	3677
q11	460	238	241	238
q12	416	226	230	226
q13	18729	2961	2951	2951
q14	283	227	226	226
q15	536	467	483	467
q16	524	391	378	378
q17	986	679	705	679
q18	8183	7833	7493	7493
q19	7285	1433	1418	1418
q20	683	330	320	320
q21	4881	3190	3294	3190
q22	402	338	346	338
Total cold run time: 120936 ms
Total hot run time: 40005 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4393	4259	4409	4259
q2	395	281	268	268
q3	3181	2975	2967	2967
q4	1985	1685	1630	1630
q5	5522	5531	5567	5531
q6	235	131	132	131
q7	2238	1838	1858	1838
q8	3282	3436	3458	3436
q9	8743	8765	8759	8759
q10	3963	3793	3881	3793
q11	603	492	497	492
q12	824	669	634	634
q13	15986	3160	3172	3160
q14	302	269	283	269
q15	534	510	478	478
q16	492	417	422	417
q17	1834	1540	1519	1519
q18	8236	7921	7852	7852
q19	1774	1420	1557	1420
q20	3024	1903	1888	1888
q21	5003	4768	4918	4768
q22	602	551	569	551
Total cold run time: 73151 ms
Total hot run time: 56060 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171631 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	926	380	376	376
query2	6369	2372	2449	2372
query3	6636	208	214	208
query4	20798	17527	17518	17518
query5	3759	466	488	466
query6	262	164	164	164
query7	4592	297	291	291
query8	328	308	303	303
query9	8478	2372	2356	2356
query10	583	309	284	284
query11	10574	10018	10196	10018
query12	118	88	85	85
query13	1647	374	360	360
query14	9975	7089	7803	7089
query15	260	195	201	195
query16	8055	277	279	277
query17	1894	563	537	537
query18	2092	284	273	273
query19	216	163	155	155
query20	95	83	84	83
query21	219	132	130	130
query22	4355	4041	3912	3912
query23	33884	33818	33559	33559
query24	10828	2927	2934	2927
query25	604	411	400	400
query26	711	165	161	161
query27	2316	334	327	327
query28	6229	2135	2137	2135
query29	922	678	664	664
query30	270	155	159	155
query31	958	778	758	758
query32	96	59	57	57
query33	676	315	305	305
query34	892	479	489	479
query35	824	678	662	662
query36	1134	972	966	966
query37	147	81	79	79
query38	2999	2876	2805	2805
query39	853	826	828	826
query40	204	125	120	120
query41	52	55	54	54
query42	127	100	102	100
query43	596	564	543	543
query44	1050	728	719	719
query45	190	169	162	162
query46	1077	695	723	695
query47	1837	1795	1770	1770
query48	365	290	290	290
query49	880	393	411	393
query50	766	388	388	388
query51	6971	6903	6735	6735
query52	97	104	89	89
query53	360	300	283	283
query54	854	449	438	438
query55	73	74	73	73
query56	281	266	269	266
query57	1144	1046	1057	1046
query58	259	243	248	243
query59	3218	3176	3232	3176
query60	300	324	280	280
query61	90	93	90	90
query62	577	465	478	465
query63	325	291	286	286
query64	8563	2272	1763	1763
query65	3150	3135	3135	3135
query66	737	320	322	320
query67	15803	14997	14880	14880
query68	8541	548	545	545
query69	771	469	342	342
query70	1247	1181	1099	1099
query71	489	275	275	275
query72	9028	2741	2619	2619
query73	1905	320	318	318
query74	5887	5532	5564	5532
query75	5113	2682	2680	2680
query76	4625	965	900	900
query77	793	306	300	300
query78	10461	9862	9830	9830
query79	3312	522	518	518
query80	2081	468	466	466
query81	547	221	216	216
query82	742	110	104	104
query83	291	178	169	169
query84	282	94	94	94
query85	1077	293	270	270
query86	400	321	321	321
query87	3268	3082	3085	3082
query88	4083	2419	2350	2350
query89	491	390	384	384
query90	1920	193	187	187
query91	129	98	103	98
query92	65	47	50	47
query93	4766	508	514	508
query94	1246	193	186	186
query95	410	306	323	306
query96	615	272	263	263
query97	3227	3056	3052	3052
query98	216	200	200	200
query99	1157	851	855	851
Total cold run time: 285705 ms
Total hot run time: 171631 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 177485f494d4bc1779b5049a997665adbe28b4d0, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.04
query4	1.69	0.08	0.07
query5	0.49	0.47	0.48
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.49	0.48
query10	0.56	0.53	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.60	0.59
query14	0.75	0.81	0.79
query15	0.85	0.85	0.81
query16	0.37	0.37	0.36
query17	0.96	0.96	1.02
query18	0.21	0.26	0.20
query19	1.77	1.73	1.73
query20	0.01	0.01	0.02
query21	15.42	0.75	0.65
query22	4.82	6.91	2.03
query23	18.22	1.46	1.33
query24	2.10	0.23	0.22
query25	0.16	0.09	0.09
query26	0.26	0.18	0.17
query27	0.09	0.08	0.08
query28	13.28	1.04	1.01
query29	12.65	3.32	3.26
query30	0.26	0.06	0.06
query31	2.85	0.40	0.40
query32	3.26	0.48	0.47
query33	2.87	3.00	2.92
query34	17.19	4.43	4.40
query35	4.50	4.53	4.51
query36	0.65	0.46	0.46
query37	0.19	0.15	0.14
query38	0.15	0.15	0.15
query39	0.05	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.02 s
Total hot run time: 30.77 s

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 1, 2024

PR approved by anyone and no changes requested.

CalvinKirs
CalvinKirs approved these changes Jul 2, 2024
View reviewed changes
Copy link
Member

@CalvinKirs CalvinKirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no problems with the changes made to the ENV file.

@CalvinKirs
Copy link
Member

trigger ci

@CalvinKirs
Copy link
Member

run buildall

1 similar comment
@lsy3993
Copy link
Contributor Author

lsy3993 commented Jul 2, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39873 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14a57ae5c1da8252ac347a76468f1502535e01ff, data reload: false

------ Round 1 ----------------------------------
q1	17606	4642	4289	4289
q2	2018	197	189	189
q3	10470	1148	1057	1057
q4	10195	754	791	754
q5	7499	2628	2652	2628
q6	223	140	137	137
q7	963	602	610	602
q8	9228	2092	2097	2092
q9	9035	6512	6479	6479
q10	9013	3749	3702	3702
q11	453	241	244	241
q12	493	239	238	238
q13	17778	2989	3033	2989
q14	268	222	211	211
q15	536	500	475	475
q16	516	383	380	380
q17	966	621	641	621
q18	8118	7518	7456	7456
q19	4704	1507	1485	1485
q20	658	317	332	317
q21	4949	3184	4086	3184
q22	414	351	347	347
Total cold run time: 116103 ms
Total hot run time: 39873 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4439	4316	4229	4229
q2	365	277	272	272
q3	3002	2800	2875	2800
q4	2026	1742	1691	1691
q5	5688	5532	5468	5468
q6	240	136	132	132
q7	2211	1761	1907	1761
q8	3263	3428	3423	3423
q9	8740	8664	8834	8664
q10	4089	3955	3721	3721
q11	586	485	496	485
q12	813	658	650	650
q13	17260	3145	3203	3145
q14	315	286	275	275
q15	532	495	500	495
q16	473	427	450	427
q17	1828	1563	1512	1512
q18	8208	8013	7943	7943
q19	1799	1596	1618	1596
q20	2150	1892	1887	1887
q21	10176	4724	4738	4724
q22	658	573	565	565
Total cold run time: 78861 ms
Total hot run time: 55865 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173824 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14a57ae5c1da8252ac347a76468f1502535e01ff, data reload: false

query1	922	386	386	386
query2	6457	2459	2419	2419
query3	6637	202	213	202
query4	19809	17372	17341	17341
query5	3894	482	483	482
query6	249	189	161	161
query7	4579	320	291	291
query8	315	298	301	298
query9	8574	2367	2361	2361
query10	583	343	289	289
query11	10605	9818	9781	9781
query12	117	88	85	85
query13	1639	388	377	377
query14	10345	7875	7111	7111
query15	225	198	191	191
query16	7731	276	274	274
query17	1870	569	555	555
query18	1850	278	283	278
query19	198	149	162	149
query20	93	84	84	84
query21	210	135	129	129
query22	4413	3972	4006	3972
query23	33895	33578	33833	33578
query24	11857	2913	2844	2844
query25	694	397	404	397
query26	1668	166	161	161
query27	2997	341	338	338
query28	7387	2156	2157	2156
query29	1054	656	664	656
query30	209	170	160	160
query31	968	784	760	760
query32	99	55	58	55
query33	801	327	317	317
query34	932	490	488	488
query35	777	668	755	668
query36	1135	970	1000	970
query37	276	78	82	78
query38	2926	2830	2792	2792
query39	886	849	877	849
query40	259	128	125	125
query41	55	50	52	50
query42	123	105	101	101
query43	613	565	540	540
query44	1322	767	768	767
query45	184	177	173	173
query46	1120	755	736	736
query47	1932	1781	1767	1767
query48	380	308	310	308
query49	918	433	407	407
query50	781	400	395	395
query51	6939	7199	6785	6785
query52	104	94	93	93
query53	365	302	300	300
query54	899	455	444	444
query55	74	71	71	71
query56	283	260	269	260
query57	1152	1038	1032	1032
query58	257	256	243	243
query59	3386	3250	3136	3136
query60	307	270	279	270
query61	97	112	92	92
query62	617	453	438	438
query63	322	293	292	292
query64	9683	2249	1775	1775
query65	3171	3109	3089	3089
query66	1218	327	328	327
query67	15508	14863	14889	14863
query68	9008	561	562	561
query69	739	472	329	329
query70	1220	1150	1140	1140
query71	546	282	281	281
query72	9277	5171	5084	5084
query73	794	335	326	326
query74	5895	5473	5391	5391
query75	5304	2674	2656	2656
query76	5605	932	986	932
query77	776	306	307	306
query78	10385	9884	9808	9808
query79	6032	507	523	507
query80	923	472	469	469
query81	573	223	223	223
query82	830	108	106	106
query83	375	173	173	173
query84	267	90	88	88
query85	1366	278	277	277
query86	410	381	325	325
query87	3250	3062	3050	3050
query88	4157	2425	2425	2425
query89	510	383	393	383
query90	2041	188	183	183
query91	130	101	97	97
query92	58	47	47	47
query93	4865	521	516	516
query94	1205	188	183	183
query95	398	315	308	308
query96	611	267	267	267
query97	3176	3048	2994	2994
query98	227	200	191	191
query99	1156	858	836	836
Total cold run time: 294054 ms
Total hot run time: 173824 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.18 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 14a57ae5c1da8252ac347a76468f1502535e01ff, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.06
query4	1.66	0.08	0.08
query5	0.49	0.49	0.49
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.56	0.51	0.49
query10	0.54	0.55	0.51
query11	0.16	0.11	0.12
query12	0.15	0.12	0.11
query13	0.61	0.59	0.58
query14	0.76	0.77	0.77
query15	0.86	0.81	0.81
query16	0.37	0.36	0.36
query17	1.02	1.00	1.04
query18	0.20	0.25	0.24
query19	1.92	1.69	1.71
query20	0.01	0.01	0.01
query21	15.44	0.78	0.66
query22	4.48	6.51	2.65
query23	18.34	1.47	1.22
query24	2.04	0.25	0.24
query25	0.17	0.09	0.09
query26	0.26	0.16	0.18
query27	0.09	0.08	0.08
query28	13.17	1.02	1.01
query29	12.63	3.25	3.21
query30	0.25	0.06	0.05
query31	2.86	0.39	0.40
query32	3.30	0.47	0.46
query33	2.89	2.90	2.89
query34	17.23	4.40	4.38
query35	4.47	4.45	4.48
query36	0.65	0.50	0.49
query37	0.19	0.15	0.16
query38	0.14	0.14	0.15
query39	0.04	0.04	0.04
query40	0.18	0.15	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.87 s
Total hot run time: 31.18 s

@w41ter w41ter merged commit ad48d00 into apache:master Jul 3, 2024
27 of 32 checks passed
dataroaring pushed a commit to dataroaring/incubator-doris that referenced this pull request Jul 5, 2024
@lsy3993 @dataroaring
[opt](binlog) support rename binlog (apache#37031)
749e1df 
Rename operator doesn't have binlog now. This PR will create binlog when
execute rename. The rename operator means :
1. rename table : ALTER TABLE table1 RENAME table2;
2. rename rollup : ALTER TABLE example_table RENAME ROLLUP rollup1
rollup2;
3. rename partition : ALTER TABLE example_table RENAME PARTITION p1 p2;

After SQL analyzing, we can get the old and new table name (rollup name
or partition name), then record the info to binlog, so we can use the
info from binlog.
@lsy3993 lsy3993 mentioned this pull request Jul 17, 2024
Merged
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@lsy3993
[opt](binlog) Log table rename when reply (#38034)
d189b8a 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Related PR is : #37031 , when replay rename table, addTableRename is
needed
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@lsy3993 @dataroaring
[opt](binlog) Log table rename when reply (#38034)
1ac05ad 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Related PR is : #37031 , when replay rename table, addTableRename is
needed
This was referenced Aug 19, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@w41ter w41ter w41ter approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

@morningman morningman Awaiting requested review from morningman morningman is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.x dev/3.0.0-merged meta-change reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lsy3993 @doris-robot @dataroaring @CalvinKirs @w41ter