Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](memory) Process profile add metadata profile #44647

Merged
merged 1 commit into from
Nov 27, 2024
Merged

[opt](memory) Process profile add metadata profile #44647

merged 1 commit into from
Nov 27, 2024

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Nov 26, 2024 •
edited
Loading

What problem does this PR solve?

Problem Summary:

http://127.0.0.1:8040/profile

ObjectHeapDump:
   - AllRowsetsMemBytes: 16.33 MB
   - AllRowsetsNum: 76.45K
   - AllSegmentsMemBytes: 4.16 KB
   - AllSegmentsNum: 1
   - BitmapIndexReaderMemBytes: 0
   - BitmapIndexReaderNum: 0
   - BloomFilterIndexReaderMemBytes: 0
   - BloomFilterIndexReaderNum: 0
   - ColumnReaderMemBytes: 0
   - ColumnReaderNum: 0
   - IndexPageReaderMemBytes: 0
   - IndexPageReaderNum: 0
   - IndexedColumnReaderMemBytes: 0
   - IndexedColumnReaderNum: 0
   - InvertedIndexReaderMemBytes: 0
   - InvertedIndexReaderNum: 0
   - OrdinalIndexReaderMemBytes: 0
   - OrdinalIndexReaderNum: 0
   - RowsetMetaMemBytes: 45.47 MB
   - RowsetMetaNum: 76.45K
   - TabletColumnMemBytes: 253.28 MB
   - TabletColumnNum: 922.18K
   - TabletIndexMemBytes: 16.78 MB
   - TabletIndexNum: 104.74K
   - TabletMetaMemBytes: 17.50 MB
   - TabletMetaNum: 45.89K
   - TabletSchemaMemBytes: 60.15 MB
   - TabletSchemaNum: 49.19K
   - ZoneMapIndexReaderMemBytes: 0
   - ZoneMapIndexReaderNum: 0

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xinyiZzz
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40273 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55c8f8fcc1c7adb0bd75575fdc1580ae05a5c72f, data reload: false

------ Round 1 ----------------------------------
q1	17588	7592	7387	7387
q2	2052	176	187	176
q3	10549	1094	1162	1094
q4	10585	786	694	694
q5	7624	2767	2810	2767
q6	238	150	147	147
q7	983	636	607	607
q8	9251	1859	1950	1859
q9	6623	6456	6351	6351
q10	6989	2335	2358	2335
q11	458	263	258	258
q12	402	213	227	213
q13	17778	3037	3013	3013
q14	257	220	227	220
q15	578	547	519	519
q16	655	576	609	576
q17	997	633	636	633
q18	7432	6702	6687	6687
q19	1338	1022	1014	1014
q20	494	191	183	183
q21	4097	3318	3227	3227
q22	390	323	313	313
Total cold run time: 107358 ms
Total hot run time: 40273 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7357	7510	7382	7382
q2	337	227	239	227
q3	2935	3006	2923	2923
q4	2099	1872	1940	1872
q5	5629	5671	5727	5671
q6	237	141	143	141
q7	2281	1837	1878	1837
q8	3422	3583	3534	3534
q9	8929	8939	8970	8939
q10	3659	3586	3605	3586
q11	607	517	501	501
q12	846	672	653	653
q13	12947	3226	3246	3226
q14	318	289	277	277
q15	588	534	530	530
q16	687	657	649	649
q17	1832	1610	1657	1610
q18	8555	7759	7790	7759
q19	1792	1630	1526	1526
q20	2136	1887	1895	1887
q21	5625	5410	5425	5410
q22	648	576	569	569
Total cold run time: 73466 ms
Total hot run time: 60709 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197603 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55c8f8fcc1c7adb0bd75575fdc1580ae05a5c72f, data reload: false

query1	1219	963	951	951
query2	6244	2134	2077	2077
query3	10847	4114	4174	4114
query4	68520	29217	23502	23502
query5	4933	473	466	466
query6	417	180	203	180
query7	5676	294	300	294
query8	311	236	228	228
query9	9355	2726	2722	2722
query10	479	252	255	252
query11	17588	15431	15861	15431
query12	159	109	103	103
query13	1557	425	419	419
query14	10629	7066	7691	7066
query15	224	197	184	184
query16	7155	481	491	481
query17	1253	581	584	581
query18	1778	319	313	313
query19	255	163	161	161
query20	121	117	114	114
query21	210	110	109	109
query22	4685	4518	4487	4487
query23	35393	34493	34678	34493
query24	5479	2514	2504	2504
query25	509	391	400	391
query26	655	160	181	160
query27	1843	280	291	280
query28	4283	2483	2483	2483
query29	688	428	424	424
query30	222	151	150	150
query31	1034	854	857	854
query32	64	62	58	58
query33	413	279	289	279
query34	931	529	534	529
query35	873	762	734	734
query36	1104	950	984	950
query37	124	76	73	73
query38	4586	4500	4371	4371
query39	1582	1504	1493	1493
query40	197	100	106	100
query41	46	42	42	42
query42	112	98	101	98
query43	600	521	506	506
query44	1197	847	843	843
query45	191	172	169	169
query46	1165	739	719	719
query47	2030	1913	1934	1913
query48	434	324	326	324
query49	740	376	389	376
query50	853	408	404	404
query51	7515	7332	7152	7152
query52	98	84	89	84
query53	263	181	175	175
query54	520	407	408	407
query55	80	77	74	74
query56	269	248	239	239
query57	1282	1174	1158	1158
query58	220	213	215	213
query59	3426	3141	3099	3099
query60	270	254	253	253
query61	105	109	108	108
query62	797	711	680	680
query63	225	192	191	191
query64	1398	654	650	650
query65	3313	3199	3220	3199
query66	639	300	307	300
query67	16034	15833	15476	15476
query68	4384	550	549	549
query69	406	257	269	257
query70	1195	1163	1138	1138
query71	325	242	251	242
query72	6512	3998	3927	3927
query73	858	359	366	359
query74	10144	9160	9036	9036
query75	3416	2696	2687	2687
query76	1896	1128	1157	1128
query77	457	279	285	279
query78	10457	9455	9544	9455
query79	1442	596	592	592
query80	874	433	450	433
query81	542	239	230	230
query82	239	122	118	118
query83	198	158	150	150
query84	284	75	70	70
query85	942	305	295	295
query86	367	307	314	307
query87	4817	4635	4553	4553
query88	3636	2255	2222	2222
query89	420	288	294	288
query90	2046	194	187	187
query91	150	103	104	103
query92	61	48	48	48
query93	2322	536	538	536
query94	771	315	292	292
query95	364	254	251	251
query96	639	293	289	289
query97	2867	2677	2684	2677
query98	228	194	199	194
query99	1742	1310	1298	1298
Total cold run time: 322418 ms
Total hot run time: 197603 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55c8f8fcc1c7adb0bd75575fdc1580ae05a5c72f, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.08	0.07
query4	1.60	0.10	0.11
query5	0.44	0.42	0.42
query6	1.14	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.51
query10	0.54	0.55	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.12
query13	0.63	0.60	0.60
query14	2.84	2.72	2.79
query15	0.91	0.84	0.83
query16	0.38	0.38	0.38
query17	1.01	1.07	1.02
query18	0.23	0.21	0.22
query19	1.84	1.76	1.94
query20	0.02	0.00	0.02
query21	15.38	0.58	0.59
query22	2.39	2.24	1.79
query23	17.04	0.93	0.84
query24	3.45	0.55	0.27
query25	0.24	0.20	0.08
query26	0.25	0.14	0.14
query27	0.05	0.06	0.04
query28	11.76	1.11	1.08
query29	12.60	3.31	3.28
query30	0.26	0.06	0.06
query31	2.85	0.39	0.38
query32	3.26	0.46	0.48
query33	3.00	3.10	3.01
query34	16.95	4.61	4.54
query35	4.56	4.58	4.63
query36	0.68	0.50	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 108.02 s
Total hot run time: 32.03 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.35% (9976/26016)
Line Coverage: 29.44% (83531/283704)
Region Coverage: 28.58% (42990/150402)
Branch Coverage: 25.20% (21836/86660)
Coverage Report: http://coverage.selectdb-in.cc/coverage/55c8f8fcc1c7adb0bd75575fdc1580ae05a5c72f_55c8f8fcc1c7adb0bd75575fdc1580ae05a5c72f/report/index.html

@xinyiZzz
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39957 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34d00c699f815869a67a651e598f6d8645624ce2, data reload: false

------ Round 1 ----------------------------------
q1	17603	7505	7235	7235
q2	2044	178	172	172
q3	10545	1125	1138	1125
q4	10224	764	724	724
q5	7611	2728	2749	2728
q6	240	147	149	147
q7	975	637	613	613
q8	9249	1884	1922	1884
q9	6496	6370	6372	6370
q10	6954	2352	2304	2304
q11	475	260	261	260
q12	430	217	224	217
q13	17780	3031	3015	3015
q14	248	217	211	211
q15	571	524	521	521
q16	680	593	576	576
q17	993	519	513	513
q18	7530	6748	6686	6686
q19	1330	1048	1046	1046
q20	472	184	181	181
q21	3962	3198	3122	3122
q22	383	307	324	307
Total cold run time: 106795 ms
Total hot run time: 39957 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7242	7250	7249	7249
q2	325	231	230	230
q3	2891	2808	3031	2808
q4	2129	1860	1802	1802
q5	5563	5683	5644	5644
q6	221	139	140	139
q7	2205	1796	1766	1766
q8	3381	3562	3484	3484
q9	8903	8876	8968	8876
q10	3617	3528	3534	3528
q11	597	504	501	501
q12	831	617	618	617
q13	12518	3211	3165	3165
q14	296	268	261	261
q15	568	524	536	524
q16	682	653	645	645
q17	1791	1579	1567	1567
q18	7922	7469	7458	7458
q19	1646	1510	1435	1435
q20	2043	1818	1785	1785
q21	5477	5239	5160	5160
q22	613	524	560	524
Total cold run time: 71461 ms
Total hot run time: 59168 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191084 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34d00c699f815869a67a651e598f6d8645624ce2, data reload: false

query1	993	386	386	386
query2	6530	2237	2119	2119
query3	6702	215	214	214
query4	34213	23516	23531	23516
query5	4331	471	429	429
query6	276	191	164	164
query7	4612	294	288	288
query8	283	243	253	243
query9	9391	2710	2706	2706
query10	476	270	248	248
query11	18175	15199	15173	15173
query12	152	104	102	102
query13	1676	419	422	419
query14	10099	7203	6798	6798
query15	255	172	187	172
query16	8141	458	449	449
query17	1635	573	559	559
query18	2139	297	321	297
query19	390	154	157	154
query20	123	124	121	121
query21	209	104	104	104
query22	4624	4289	4462	4289
query23	35006	34328	34302	34302
query24	11468	2465	2492	2465
query25	668	388	397	388
query26	1806	145	152	145
query27	2884	281	277	277
query28	8298	2451	2423	2423
query29	1061	405	400	400
query30	310	146	147	146
query31	1032	806	824	806
query32	90	79	56	56
query33	791	284	288	284
query34	995	512	503	503
query35	912	762	727	727
query36	1124	959	922	922
query37	277	75	78	75
query38	4561	4219	4291	4219
query39	1492	1440	1406	1406
query40	289	101	102	101
query41	48	44	45	44
query42	108	101	98	98
query43	555	509	491	491
query44	1302	807	801	801
query45	186	169	163	163
query46	1129	686	674	674
query47	1955	1874	1837	1837
query48	408	311	337	311
query49	1283	398	391	391
query50	803	387	398	387
query51	7256	7090	7093	7090
query52	99	93	87	87
query53	256	184	178	178
query54	1260	407	411	407
query55	81	80	81	80
query56	269	240	254	240
query57	1321	1197	1134	1134
query58	235	212	215	212
query59	3171	3038	2916	2916
query60	294	275	244	244
query61	108	107	110	107
query62	858	670	662	662
query63	222	189	188	188
query64	5167	663	629	629
query65	3296	3197	3253	3197
query66	1447	317	302	302
query67	16326	15731	15650	15650
query68	5031	552	540	540
query69	429	257	255	255
query70	1225	1046	1180	1046
query71	342	246	252	246
query72	6356	4207	4095	4095
query73	761	363	366	363
query74	10639	9093	9004	9004
query75	3443	2674	2693	2674
query76	2877	1047	1002	1002
query77	508	270	292	270
query78	10474	9363	9433	9363
query79	1325	592	597	592
query80	1051	446	435	435
query81	544	243	224	224
query82	1147	123	117	117
query83	255	163	158	158
query84	240	71	69	69
query85	1299	346	296	296
query86	357	303	301	301
query87	4636	4602	4727	4602
query88	3337	2255	2193	2193
query89	415	293	292	292
query90	1951	195	189	189
query91	135	104	106	104
query92	62	49	52	49
query93	1068	532	548	532
query94	905	307	287	287
query95	362	247	249	247
query96	613	273	282	273
query97	2818	2658	2657	2657
query98	214	202	194	194
query99	1687	1319	1300	1300
Total cold run time: 304438 ms
Total hot run time: 191084 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34d00c699f815869a67a651e598f6d8645624ce2, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.03
query3	0.23	0.08	0.07
query4	1.62	0.10	0.10
query5	0.43	0.40	0.41
query6	1.18	0.68	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.53	0.51
query10	0.55	0.57	0.55
query11	0.14	0.11	0.11
query12	0.13	0.12	0.12
query13	0.61	0.61	0.59
query14	2.70	2.72	2.78
query15	0.91	0.83	0.84
query16	0.39	0.39	0.40
query17	0.98	1.02	1.01
query18	0.24	0.22	0.20
query19	1.95	1.74	1.98
query20	0.01	0.02	0.01
query21	15.36	0.60	0.59
query22	2.32	2.86	1.92
query23	17.00	1.18	0.90
query24	2.93	2.07	1.11
query25	0.20	0.24	0.24
query26	0.42	0.13	0.12
query27	0.05	0.04	0.04
query28	9.83	1.11	1.09
query29	12.56	3.21	3.25
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.28	0.46	0.47
query33	3.00	3.09	3.06
query34	17.01	4.46	4.45
query35	4.49	4.49	4.52
query36	0.66	0.48	0.48
query37	0.09	0.06	0.07
query38	0.04	0.03	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.48 s
Total hot run time: 33 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.35% (9976/26016)
Line Coverage: 29.44% (83539/283720)
Region Coverage: 28.59% (42998/150414)
Branch Coverage: 25.20% (21838/86670)
Coverage Report: http://coverage.selectdb-in.cc/coverage/34d00c699f815869a67a651e598f6d8645624ce2_34d00c699f815869a67a651e598f6d8645624ce2/report/index.html

@xinyiZzz xinyiZzz force-pushed the 20241126_fix_memory_bvar branch from 34d00c6 to 14add1e Compare November 27, 2024 10:01
@xinyiZzz
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.35% (9976/26016)
Line Coverage: 29.43% (83503/283732)
Region Coverage: 28.58% (42998/150422)
Branch Coverage: 25.19% (21835/86674)
Coverage Report: http://coverage.selectdb-in.cc/coverage/14add1e6a950f28991014acb70233f225dd86b00_14add1e6a950f28991014acb70233f225dd86b00/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40028 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14add1e6a950f28991014acb70233f225dd86b00, data reload: false

------ Round 1 ----------------------------------
q1	17591	7424	7598	7424
q2	2051	170	163	163
q3	10644	1107	1154	1107
q4	10556	805	768	768
q5	7594	2752	2739	2739
q6	241	145	146	145
q7	981	631	594	594
q8	9249	1819	1899	1819
q9	6563	6348	6326	6326
q10	7043	2280	2326	2280
q11	458	261	252	252
q12	430	217	214	214
q13	17781	3023	3058	3023
q14	236	211	213	211
q15	583	523	515	515
q16	665	571	580	571
q17	989	625	535	535
q18	7213	6667	6752	6667
q19	1341	963	1015	963
q20	458	179	180	179
q21	3958	3297	3222	3222
q22	377	317	311	311
Total cold run time: 107002 ms
Total hot run time: 40028 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7236	7204	7238	7204
q2	324	233	230	230
q3	2911	2779	2945	2779
q4	2064	1804	1789	1789
q5	5703	5660	5629	5629
q6	226	140	139	139
q7	2281	1826	1849	1826
q8	3396	3551	3471	3471
q9	8826	8962	8943	8943
q10	3601	3550	3554	3550
q11	609	507	521	507
q12	824	610	596	596
q13	12492	3233	3221	3221
q14	307	295	272	272
q15	573	524	537	524
q16	695	651	655	651
q17	1882	1628	1637	1628
q18	8283	7784	7397	7397
q19	3924	1562	1607	1562
q20	2102	1882	1872	1872
q21	5472	5409	5549	5409
q22	658	554	571	554
Total cold run time: 74389 ms
Total hot run time: 59753 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196517 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14add1e6a950f28991014acb70233f225dd86b00, data reload: false

query1	1237	980	953	953
query2	6240	2169	2086	2086
query3	10945	4028	4109	4028
query4	67763	27209	23778	23778
query5	4975	489	435	435
query6	399	180	180	180
query7	5538	306	287	287
query8	321	264	229	229
query9	8500	2637	2615	2615
query10	436	255	247	247
query11	17337	15253	16024	15253
query12	159	108	105	105
query13	1505	442	445	442
query14	10830	7648	6830	6830
query15	218	194	199	194
query16	7090	461	486	461
query17	1273	592	584	584
query18	1819	318	320	318
query19	247	164	150	150
query20	121	116	110	110
query21	210	118	101	101
query22	4750	4456	4451	4451
query23	34850	34561	34291	34291
query24	5571	2558	2475	2475
query25	485	373	397	373
query26	640	153	149	149
query27	1875	299	287	287
query28	4551	2480	2446	2446
query29	679	407	416	407
query30	225	147	153	147
query31	1038	852	828	828
query32	65	57	61	57
query33	390	284	309	284
query34	938	508	514	508
query35	862	759	769	759
query36	1098	998	958	958
query37	118	71	77	71
query38	4580	4388	4491	4388
query39	1525	1475	1458	1458
query40	207	98	94	94
query41	49	44	45	44
query42	105	98	95	95
query43	563	493	504	493
query44	1185	848	828	828
query45	185	168	177	168
query46	1145	694	693	693
query47	2057	1899	1916	1899
query48	425	321	334	321
query49	740	405	414	405
query50	845	392	387	387
query51	7427	7190	6956	6956
query52	97	87	88	87
query53	255	176	182	176
query54	511	397	386	386
query55	74	76	83	76
query56	244	239	236	236
query57	1295	1162	1167	1162
query58	226	229	217	217
query59	3268	3273	3040	3040
query60	285	264	249	249
query61	111	108	102	102
query62	799	663	664	663
query63	207	180	185	180
query64	1365	660	632	632
query65	3277	3224	3225	3224
query66	706	302	304	302
query67	15960	15740	15675	15675
query68	3921	562	573	562
query69	427	252	265	252
query70	1218	1150	1131	1131
query71	362	251	256	251
query72	6371	4159	3980	3980
query73	768	349	356	349
query74	10454	9058	9082	9058
query75	3422	2658	2665	2658
query76	1897	1063	1181	1063
query77	470	284	284	284
query78	10558	9418	9376	9376
query79	1574	603	600	600
query80	911	437	446	437
query81	510	331	232	232
query82	1311	122	120	120
query83	257	155	149	149
query84	280	81	69	69
query85	913	304	290	290
query86	341	297	305	297
query87	4842	4875	4529	4529
query88	3468	2169	2145	2145
query89	419	299	337	299
query90	2020	187	187	187
query91	134	104	104	104
query92	62	50	49	49
query93	1944	547	550	547
query94	783	296	290	290
query95	350	246	238	238
query96	623	274	272	272
query97	2879	2686	2688	2686
query98	232	201	195	195
query99	1787	1305	1327	1305
Total cold run time: 320524 ms
Total hot run time: 196517 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.84 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 14add1e6a950f28991014acb70233f225dd86b00, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.08
query4	1.62	0.10	0.11
query5	0.44	0.41	0.42
query6	1.15	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.57	0.56	0.56
query11	0.15	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.59	0.61
query14	2.72	2.84	2.76
query15	0.90	0.84	0.82
query16	0.39	0.39	0.39
query17	1.05	1.03	1.00
query18	0.23	0.21	0.21
query19	2.01	1.89	2.01
query20	0.02	0.01	0.02
query21	15.36	0.60	0.57
query22	2.27	2.26	1.98
query23	17.02	0.99	0.78
query24	3.29	1.34	1.00
query25	0.24	0.08	0.10
query26	0.58	0.15	0.14
query27	0.04	0.03	0.05
query28	10.29	1.12	1.07
query29	12.56	3.25	3.22
query30	0.24	0.06	0.05
query31	2.88	0.38	0.39
query32	3.29	0.46	0.47
query33	3.05	3.07	3.07
query34	17.27	4.44	4.46
query35	4.56	4.50	4.50
query36	0.68	0.49	0.48
query37	0.09	0.05	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.06 s
Total hot run time: 32.84 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Nov 27, 2024
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xinyiZzz xinyiZzz merged commit de8cfd1 into apache:master Nov 27, 2024
22 of 25 checks passed
@xinyiZzz xinyiZzz removed the dev/2.1.x label Dec 3, 2024
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 24, 2024
@xinyiZzz
[opt](memory) Process profile add metadata profile (apache#44647)
4b620d4 
### What problem does this PR solve?

Problem Summary:

http://127.0.0.1:8040/profile
```
ObjectHeapDump:
   - AllRowsetsMemBytes: 16.33 MB
   - AllRowsetsNum: 76.45K
   - AllSegmentsMemBytes: 4.16 KB
   - AllSegmentsNum: 1
   - BitmapIndexReaderMemBytes: 0
   - BitmapIndexReaderNum: 0
   - BloomFilterIndexReaderMemBytes: 0
   - BloomFilterIndexReaderNum: 0
   - ColumnReaderMemBytes: 0
   - ColumnReaderNum: 0
   - IndexPageReaderMemBytes: 0
   - IndexPageReaderNum: 0
   - IndexedColumnReaderMemBytes: 0
   - IndexedColumnReaderNum: 0
   - InvertedIndexReaderMemBytes: 0
   - InvertedIndexReaderNum: 0
   - OrdinalIndexReaderMemBytes: 0
   - OrdinalIndexReaderNum: 0
   - RowsetMetaMemBytes: 45.47 MB
   - RowsetMetaNum: 76.45K
   - TabletColumnMemBytes: 253.28 MB
   - TabletColumnNum: 922.18K
   - TabletIndexMemBytes: 16.78 MB
   - TabletIndexNum: 104.74K
   - TabletMetaMemBytes: 17.50 MB
   - TabletMetaNum: 45.89K
   - TabletSchemaMemBytes: 60.15 MB
   - TabletSchemaNum: 49.19K
   - ZoneMapIndexReaderMemBytes: 0
   - ZoneMapIndexReaderNum: 0
 ```
@xinyiZzz xinyiZzz mentioned this pull request Dec 24, 2024
Merged
16 tasks
yiguolei pushed a commit that referenced this pull request Dec 25, 2024
@xinyiZzz
[pick](branch-3.0) pick #44647 (#45839)
207327f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangbo wangbo wangbo approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.4-merged not-merge/2.1 reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@xinyiZzz @doris-robot @yiguolei @wangbo