Skip to content

[fix](tde) fix issues related to TDE #55692

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 4 commits into from
Sep 5, 2025
Merged

[fix](tde) fix issues related to TDE #55692

morrySnow merged 4 commits into from
Sep 5, 2025

Conversation

@luwei16
Copy link
Contributor

@luwei16 luwei16 commented Sep 4, 2025

No description provided.

@luwei16 luwei16 requested a review from morrySnow as a code owner September 4, 2025 16:47
@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 4, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@luwei16
Copy link
Contributor Author

luwei16 commented Sep 4, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 4, 2025

FE UT Coverage Report

Increment line coverage 1.54% (1/65) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 4, 2025

BE UT Coverage Report

Increment line coverage 20.62% (33/160) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.55% (12758/28008)
Line Coverage 36.37% (113716/312651)
Region Coverage 34.01% (65048/191284)
Branch Coverage 31.04% (34144/109994)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 4, 2025

BE Regression && UT Coverage Report

Increment line coverage 26.42% (42/159) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.45% (21060/27547)
Line Coverage 69.74% (217348/311677)
Region Coverage 67.66% (129941/192061)
Branch Coverage 61.18% (67613/110508)

dataroaring
dataroaring previously approved these changes Sep 5, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 5, 2025

BE Regression && UT Coverage Report

Increment line coverage 26.42% (42/159) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.45% (21060/27547)
Line Coverage 69.74% (217348/311677)
Region Coverage 67.66% (129941/192061)
Branch Coverage 61.18% (67613/110508)

@luwei16
Copy link
Contributor Author

luwei16 commented Sep 5, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 5, 2025

FE UT Coverage Report

Increment line coverage 1.54% (1/65) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Sep 5, 2025

TPC-H: Total hot run time: 32853 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 946243b44ad1a02cbd7ea9df2692d13dbc389967, data reload: false

------ Round 1 ----------------------------------
q1	17623	5543	5356	5356
q2	2027	415	281	281
q3	12269	1250	743	743
q4	10216	888	453	453
q5	8059	2413	2205	2205
q6	180	163	134	134
q7	924	762	631	631
q8	9346	1447	1215	1215
q9	5354	5027	4973	4973
q10	6770	2275	1798	1798
q11	465	278	260	260
q12	338	354	213	213
q13	17793	3586	2999	2999
q14	234	230	210	210
q15	534	470	459	459
q16	438	433	373	373
q17	599	861	356	356
q18	7202	6466	6465	6465
q19	1285	945	550	550
q20	336	340	203	203
q21	2819	2225	1992	1992
q22	1191	1063	984	984
Total cold run time: 106002 ms
Total hot run time: 32853 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5514	5453	5448	5448
q2	234	326	236	236
q3	2262	2668	2368	2368
q4	1344	1905	1404	1404
q5	4413	4966	5015	4966
q6	173	163	131	131
q7	2076	1982	1867	1867
q8	2644	2844	2734	2734
q9	7291	7285	7190	7190
q10	3059	3292	2689	2689
q11	574	510	504	504
q12	691	771	633	633
q13	3412	3815	3168	3168
q14	274	304	260	260
q15	524	472	466	466
q16	440	495	445	445
q17	1261	1730	1266	1266
q18	7704	7516	7311	7311
q19	793	990	1150	990
q20	2043	2064	1905	1905
q21	5323	4987	4473	4473
q22	1107	1070	1027	1027
Total cold run time: 53156 ms
Total hot run time: 51481 ms

@doris-robot
Copy link

doris-robot commented Sep 5, 2025

TPC-DS: Total hot run time: 193166 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 946243b44ad1a02cbd7ea9df2692d13dbc389967, data reload: false

query1	958	393	400	393
query2	6223	1955	1896	1896
query3	8681	207	197	197
query4	33393	23873	23320	23320
query5	3702	586	448	448
query6	284	195	190	190
query7	4203	489	325	325
query8	301	236	227	227
query9	9336	2604	2592	2592
query10	478	320	272	272
query11	18240	15464	15237	15237
query12	179	111	109	109
query13	1558	549	424	424
query14	10643	7097	7532	7097
query15	246	190	192	190
query16	8147	659	525	525
query17	1569	771	597	597
query18	2182	426	327	327
query19	236	197	168	168
query20	124	120	115	115
query21	214	137	110	110
query22	4847	4715	4605	4605
query23	34746	34461	34732	34461
query24	7298	2764	2717	2717
query25	530	498	440	440
query26	790	297	174	174
query27	2180	483	373	373
query28	5230	2232	2212	2212
query29	714	595	468	468
query30	248	201	171	171
query31	1032	916	833	833
query32	94	57	57	57
query33	480	365	315	315
query34	739	850	525	525
query35	793	826	750	750
query36	1000	1052	952	952
query37	104	96	66	66
query38	4030	3988	3950	3950
query39	1524	1478	1478	1478
query40	211	119	107	107
query41	48	49	48	48
query42	125	106	106	106
query43	525	520	487	487
query44	1380	836	837	836
query45	178	178	173	173
query46	868	1050	683	683
query47	1955	2009	1937	1937
query48	403	430	353	353
query49	732	513	441	441
query50	679	700	426	426
query51	7396	7388	7168	7168
query52	105	100	91	91
query53	220	261	191	191
query54	557	546	484	484
query55	80	78	81	78
query56	260	287	257	257
query57	1318	1284	1203	1203
query58	241	215	232	215
query59	3165	3220	3120	3120
query60	299	285	317	285
query61	130	112	113	112
query62	809	738	715	715
query63	232	199	201	199
query64	3786	1021	664	664
query65	3380	3291	3318	3291
query66	817	405	305	305
query67	16248	15730	15764	15730
query68	7566	820	550	550
query69	493	301	270	270
query70	1173	1136	1096	1096
query71	385	301	261	261
query72	5832	3790	3643	3643
query73	622	737	352	352
query74	10653	9331	9136	9136
query75	3271	3128	2636	2636
query76	3207	1178	777	777
query77	715	361	298	298
query78	10512	10524	9604	9604
query79	3462	876	594	594
query80	639	514	430	430
query81	503	265	218	218
query82	554	117	90	90
query83	171	175	147	147
query84	282	102	83	83
query85	785	369	301	301
query86	362	309	301	301
query87	4310	4289	4209	4209
query88	4641	2400	2388	2388
query89	402	329	298	298
query90	1794	193	191	191
query91	165	135	107	107
query92	65	54	52	52
query93	1519	844	554	554
query94	685	417	312	312
query95	343	285	270	270
query96	478	598	283	283
query97	3238	3271	3208	3208
query98	216	218	199	199
query99	1551	1429	1297	1297
Total cold run time: 294017 ms
Total hot run time: 193166 ms

@doris-robot
Copy link

doris-robot commented Sep 5, 2025

ClickBench: Total hot run time: 28.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 946243b44ad1a02cbd7ea9df2692d13dbc389967, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.11	0.11
query5	0.52	0.52	0.52
query6	1.13	0.74	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.48
query10	0.56	0.54	0.55
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	0.79	0.79	0.79
query15	0.84	0.83	0.82
query16	0.36	0.39	0.38
query17	0.98	1.10	1.09
query18	0.23	0.21	0.21
query19	1.83	1.86	1.81
query20	0.01	0.00	0.01
query21	15.44	0.94	0.60
query22	0.74	0.91	0.72
query23	14.96	1.45	0.52
query24	3.24	0.54	1.56
query25	0.13	0.11	0.11
query26	0.36	0.16	0.13
query27	0.05	0.05	0.05
query28	13.35	1.06	0.45
query29	12.60	4.02	3.26
query30	0.26	0.09	0.07
query31	2.81	0.59	0.38
query32	3.22	0.54	0.47
query33	3.00	3.02	3.04
query34	16.71	5.21	4.53
query35	4.56	4.57	4.60
query36	0.66	0.51	0.47
query37	0.09	0.06	0.05
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.29 s
Total hot run time: 28.63 s

@doris-robot
Copy link

doris-robot commented Sep 5, 2025

BE UT Coverage Report

Increment line coverage 20.62% (33/160) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.55% (12758/28008)
Line Coverage 36.38% (113756/312650)
Region Coverage 34.01% (65057/191282)
Branch Coverage 31.04% (34142/109992)

@morrySnow morrySnow merged commit a1fe9f2 into apache:branch-3.1 Sep 5, 2025
20 of 22 checks passed
@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 5, 2025

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.46% (21062/27547)
Line Coverage 69.76% (217423/311676)
Region Coverage 67.73% (130077/192059)
Branch Coverage 61.25% (67681/110506)

luwei16 added a commit to luwei16/incubator-doris that referenced this pull request Sep 12, 2025
@luwei16
[fix](tde) fix issues related to TDE (apache#55692)
0c9e018
@luwei16 luwei16 mentioned this pull request Sep 12, 2025
Merged
gavinchou pushed a commit that referenced this pull request Sep 14, 2025
@luwei16
[fix](tde) fix issues related to TDE (#55692) (#55950)
1bee89a 
pick 3.1 #55692
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
@airborne12 airborne12 mentioned this pull request Nov 25, 2025
Merged
16 tasks
airborne12 added a commit that referenced this pull request Nov 26, 2025
@airborne12
[fix](inverted index) fix fs set when meet segment corruption (#58317)
88ec7dc 
Related PR: #55692
github-actions bot pushed a commit that referenced this pull request Nov 26, 2025
@airborne12
[fix](inverted index) fix fs set when meet segment corruption (#58317)
26c80f4 
Related PR: #55692
github-actions bot pushed a commit that referenced this pull request Nov 26, 2025
@airborne12
[fix](inverted index) fix fs set when meet segment corruption (#58317)
0900b19 
Related PR: #55692
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@airborne12 @nagisa-kunhah
[fix](inverted index) fix fs set when meet segment corruption (apache…
d1f3fb0 
…#58317)

Related PR: apache#55692
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

@dataroaring dataroaring dataroaring left review comments

Assignees

No one assigned

Labels

dev/3.1.1-merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@luwei16 @hello-stephen @doris-robot @dataroaring @morrySnow