Skip to content

branch-3.1: [fix](mow) Fix mow agg cache version check #55330 #55475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 4, 2025
Merged

branch-3.1: [fix](mow) Fix mow agg cache version check #55330 #55475

morrySnow merged 1 commit into from
Sep 4, 2025
+13 −19

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 29, 2025

Cherry-picked from #55330

@mymeiyi
[fix](mow) Fix mow agg cache version check (#55330)
2c07107 
```
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1756116003 (unix time) try "date -d @1756116003" if you are using GNU date ***
*** Current BE git commitID: 4e63c65 ***
*** SIGSEGV address not mapped to object (@0x190) received by PID 5202 (TID 5685 OR 0x7ff37209a640) from PID 400; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007FF49704E520 in /lib/x86_64-linux-gnu/libc.so.6
 4# doris::TabletMeta::_check_mow_rowset_cache_version_size(unsigned long) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/olap/tablet_meta.cpp:1082
 5# doris::TabletMeta::remove_rowset_delete_bitmap(doris::RowsetId const&, doris::Version const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/olap/tablet_meta.cpp:506
 6# doris::CloudTablet::remove_unused_rowsets() at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/cloud/cloud_tablet.cpp:563
 7# doris::CloudTabletMgr::vacuum_stale_rowsets(doris::CountDownLatch const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/cloud/cloud_tablet_mgr.cpp:286
 8# std::_Function_handler<void (), doris::CloudStorageEngine::start_bg_threads()::$_1>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
 9# doris::Thread::supervise_thread(void*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/util/thread.cpp:499
```

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot requested a review from morrySnow as a code owner August 29, 2025 09:14
@Thearas
Copy link
Contributor

Thearas commented Aug 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Aug 29, 2025
@Thearas
Copy link
Contributor

Thearas commented Aug 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-H: Total hot run time: 32944 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2c071071e2bac88e91cd82bf10904cc9bf246cdc, data reload: false

------ Round 1 ----------------------------------
q1	17591	5485	5470	5470
q2	2007	401	286	286
q3	11701	1265	748	748
q4	10300	890	478	478
q5	9244	2396	2143	2143
q6	196	167	137	137
q7	883	748	616	616
q8	9317	1441	1148	1148
q9	5276	4957	4987	4957
q10	6776	2282	1819	1819
q11	511	281	270	270
q12	347	352	216	216
q13	17787	3636	3039	3039
q14	224	227	211	211
q15	529	474	469	469
q16	434	441	399	399
q17	601	877	366	366
q18	6843	6536	6387	6387
q19	1206	948	550	550
q20	344	349	210	210
q21	3101	2226	2051	2051
q22	1048	1051	974	974
Total cold run time: 106266 ms
Total hot run time: 32944 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5574	5510	5523	5510
q2	238	321	235	235
q3	2237	2599	2313	2313
q4	1363	1857	1336	1336
q5	4391	5025	5011	5011
q6	167	162	133	133
q7	2103	1941	1814	1814
q8	2661	2840	2719	2719
q9	7281	7222	7266	7222
q10	3048	3291	2766	2766
q11	583	505	501	501
q12	670	775	584	584
q13	3359	3740	3162	3162
q14	295	299	268	268
q15	522	464	465	464
q16	429	476	453	453
q17	1213	1747	1290	1290
q18	7716	7392	7302	7302
q19	788	887	1143	887
q20	2000	2054	1881	1881
q21	5407	4910	4645	4645
q22	1085	1057	1002	1002
Total cold run time: 53130 ms
Total hot run time: 51498 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-DS: Total hot run time: 192657 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2c071071e2bac88e91cd82bf10904cc9bf246cdc, data reload: false

query1	964	402	398	398
query2	6237	1998	1907	1907
query3	8679	197	200	197
query4	33695	23660	23427	23427
query5	3658	590	443	443
query6	293	183	170	170
query7	4209	493	312	312
query8	316	251	245	245
query9	9441	2611	2577	2577
query10	443	313	253	253
query11	18148	15409	15150	15150
query12	152	107	102	102
query13	1573	551	398	398
query14	9878	7136	7200	7136
query15	241	191	177	177
query16	8048	684	512	512
query17	1545	796	600	600
query18	2156	430	345	345
query19	248	188	166	166
query20	128	134	119	119
query21	203	129	108	108
query22	4485	4751	4497	4497
query23	35208	34134	34462	34134
query24	7395	2695	2728	2695
query25	519	486	435	435
query26	1018	285	178	178
query27	2020	491	372	372
query28	5150	2197	2178	2178
query29	696	605	458	458
query30	238	208	158	158
query31	1013	914	857	857
query32	86	68	63	63
query33	505	385	322	322
query34	765	849	529	529
query35	789	803	720	720
query36	1011	1090	935	935
query37	106	93	67	67
query38	4052	3977	3962	3962
query39	1544	1498	1449	1449
query40	203	126	111	111
query41	53	53	49	49
query42	123	108	108	108
query43	526	516	493	493
query44	1383	840	859	840
query45	193	178	171	171
query46	895	1054	682	682
query47	1982	1981	1937	1937
query48	402	440	349	349
query49	741	521	401	401
query50	668	684	423	423
query51	7454	7343	7257	7257
query52	100	105	98	98
query53	229	252	188	188
query54	552	544	475	475
query55	85	79	78	78
query56	281	269	260	260
query57	1273	1255	1202	1202
query58	245	218	222	218
query59	3125	3253	3013	3013
query60	313	285	267	267
query61	121	120	117	117
query62	810	766	678	678
query63	244	206	203	203
query64	3668	1005	689	689
query65	3365	3301	3352	3301
query66	1005	403	294	294
query67	16040	15703	15623	15623
query68	7407	825	534	534
query69	493	298	272	272
query70	1227	1131	1137	1131
query71	389	294	260	260
query72	5846	3782	3946	3782
query73	637	738	357	357
query74	10402	9180	9193	9180
query75	3230	3150	2682	2682
query76	3194	1146	743	743
query77	633	362	272	272
query78	10392	10478	9634	9634
query79	3417	892	595	595
query80	766	521	440	440
query81	507	261	221	221
query82	895	116	88	88
query83	165	167	145	145
query84	241	106	77	77
query85	785	381	317	317
query86	411	305	294	294
query87	4323	4310	4246	4246
query88	5179	2388	2392	2388
query89	411	333	299	299
query90	1713	188	189	188
query91	138	138	110	110
query92	66	61	53	53
query93	2886	851	543	543
query94	781	409	321	321
query95	343	278	267	267
query96	486	601	291	291
query97	3213	3237	3162	3162
query98	239	208	196	196
query99	1338	1400	1293	1293
Total cold run time: 295087 ms
Total hot run time: 192657 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

ClickBench: Total hot run time: 29.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2c071071e2bac88e91cd82bf10904cc9bf246cdc, data reload: false

query1	0.04	0.03	0.02
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.54	0.50	0.53
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.58	0.50	0.50
query10	0.55	0.56	0.55
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	0.78	0.83	0.80
query15	0.85	0.84	0.82
query16	0.39	0.39	0.38
query17	0.99	1.07	1.05
query18	0.25	0.23	0.23
query19	1.91	1.81	1.85
query20	0.01	0.01	0.01
query21	15.38	0.90	0.59
query22	0.75	0.79	0.64
query23	15.18	1.48	0.57
query24	2.61	1.31	1.50
query25	0.18	0.20	0.08
query26	0.35	0.15	0.15
query27	0.05	0.06	0.06
query28	13.77	1.01	0.46
query29	12.58	3.98	3.35
query30	0.25	0.09	0.06
query31	2.82	0.59	0.39
query32	3.22	0.54	0.47
query33	3.04	3.02	3.02
query34	16.60	5.17	4.54
query35	4.58	4.53	4.57
query36	0.68	0.51	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.16	0.13	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 103.41 s
Total hot run time: 29.47 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/12) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.49% (12725/27973)
Line Coverage 36.38% (113481/311925)
Region Coverage 34.00% (64945/191003)
Branch Coverage 31.05% (34099/109808)

@morrySnow morrySnow merged commit 2e50f49 into branch-3.1 Sep 4, 2025
21 of 22 checks passed
@morrySnow morrySnow deleted the auto-pick-55330-branch-3.1 branch September 4, 2025 01:58
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Thearas @doris-robot @hello-stephen @morrySnow @dataroaring @mymeiyi