Skip to content

branch-3.0: [fix](mow) Fix mow agg cache version check #55330 #55474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Sep 5, 2025
Merged

branch-3.0: [fix](mow) Fix mow agg cache version check #55330 #55474

dataroaring merged 1 commit into from
Sep 5, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 29, 2025

Cherry-picked from #55330

@mymeiyi
[fix](mow) Fix mow agg cache version check (#55330)
d9580e6 
```
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1756116003 (unix time) try "date -d @1756116003" if you are using GNU date ***
*** Current BE git commitID: 4e63c65 ***
*** SIGSEGV address not mapped to object (@0x190) received by PID 5202 (TID 5685 OR 0x7ff37209a640) from PID 400; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007FF49704E520 in /lib/x86_64-linux-gnu/libc.so.6
 4# doris::TabletMeta::_check_mow_rowset_cache_version_size(unsigned long) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/olap/tablet_meta.cpp:1082
 5# doris::TabletMeta::remove_rowset_delete_bitmap(doris::RowsetId const&, doris::Version const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/olap/tablet_meta.cpp:506
 6# doris::CloudTablet::remove_unused_rowsets() at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/cloud/cloud_tablet.cpp:563
 7# doris::CloudTabletMgr::vacuum_stale_rowsets(doris::CountDownLatch const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/cloud/cloud_tablet_mgr.cpp:286
 8# std::_Function_handler<void (), doris::CloudStorageEngine::start_bg_threads()::$_1>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
 9# doris::Thread::supervise_thread(void*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-second-branch/be/src/util/thread.cpp:499
```

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot requested a review from dataroaring as a code owner August 29, 2025 09:13
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Aug 29, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-H: Total hot run time: 40253 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d9580e655e6e8da264bd5e8ec83395c2be77161a, data reload: false

------ Round 1 ----------------------------------
q1	17732	7001	6738	6738
q2	2056	230	185	185
q3	10549	1172	1194	1172
q4	10456	794	731	731
q5	7753	2897	2880	2880
q6	214	132	136	132
q7	975	621	605	605
q8	9370	1942	2016	1942
q9	6738	6393	6452	6393
q10	7006	2300	2321	2300
q11	471	266	269	266
q12	403	207	223	207
q13	17785	3046	3000	3000
q14	231	207	210	207
q15	499	460	464	460
q16	484	383	379	379
q17	997	633	589	589
q18	7377	6750	6672	6672
q19	1429	1073	1041	1041
q20	480	209	204	204
q21	3888	3181	3190	3181
q22	1106	985	969	969
Total cold run time: 107999 ms
Total hot run time: 40253 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6660	6578	6577	6577
q2	340	237	235	235
q3	2980	2927	2966	2927
q4	2024	1873	1793	1793
q5	5784	5774	5769	5769
q6	210	127	127	127
q7	2272	1816	1840	1816
q8	3407	3559	3582	3559
q9	8821	9054	8977	8977
q10	3577	3574	3554	3554
q11	604	498	497	497
q12	809	571	596	571
q13	7995	3218	3179	3179
q14	301	291	268	268
q15	530	460	467	460
q16	502	448	446	446
q17	1872	1647	1644	1644
q18	8387	7785	7858	7785
q19	1670	1527	1449	1449
q20	2074	1859	1909	1859
q21	5176	5077	5065	5065
q22	1171	1048	1017	1017
Total cold run time: 67166 ms
Total hot run time: 59574 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-DS: Total hot run time: 192748 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d9580e655e6e8da264bd5e8ec83395c2be77161a, data reload: false

query1	939	384	378	378
query2	6271	1956	1896	1896
query3	8675	205	200	200
query4	33806	23662	23538	23538
query5	3484	462	448	448
query6	281	184	185	184
query7	4208	308	316	308
query8	291	228	242	228
query9	9361	2611	2628	2611
query10	486	268	265	265
query11	18307	15149	15490	15149
query12	165	103	101	101
query13	1550	433	429	429
query14	8988	7366	7582	7366
query15	252	176	186	176
query16	8079	507	491	491
query17	1639	613	597	597
query18	2216	332	322	322
query19	339	164	158	158
query20	126	118	118	118
query21	211	109	112	109
query22	4849	4486	4506	4486
query23	35379	34272	33880	33880
query24	11089	2962	2988	2962
query25	675	416	427	416
query26	1269	176	171	171
query27	2363	362	375	362
query28	6962	2179	2188	2179
query29	863	466	454	454
query30	271	178	163	163
query31	1071	856	825	825
query32	104	60	60	60
query33	782	329	317	317
query34	1025	517	511	511
query35	923	738	741	738
query36	1113	958	922	922
query37	138	67	69	67
query38	4057	3942	3975	3942
query39	1505	1463	1453	1453
query40	211	101	103	101
query41	50	49	54	49
query42	120	99	101	99
query43	529	491	494	491
query44	1231	819	826	819
query45	200	171	171	171
query46	1166	711	713	711
query47	2039	1866	1890	1866
query48	469	379	385	379
query49	1069	402	397	397
query50	832	440	435	435
query51	7332	7340	7254	7254
query52	115	92	95	92
query53	265	186	183	183
query54	1216	483	496	483
query55	83	78	76	76
query56	259	258	261	258
query57	1332	1213	1165	1165
query58	229	211	229	211
query59	3231	3055	3094	3055
query60	294	262	247	247
query61	115	109	109	109
query62	851	728	707	707
query63	226	191	185	185
query64	4121	663	669	663
query65	3398	3293	3336	3293
query66	864	305	285	285
query67	16628	15868	15570	15570
query68	4575	595	600	595
query69	439	265	273	265
query70	1169	1099	1104	1099
query71	359	260	248	248
query72	6331	4154	4110	4110
query73	763	348	360	348
query74	10279	8905	9348	8905
query75	3414	2649	2659	2649
query76	2896	1090	1094	1090
query77	424	286	277	277
query78	10723	9515	9579	9515
query79	1904	592	614	592
query80	728	432	430	430
query81	529	221	221	221
query82	593	91	86	86
query83	159	154	148	148
query84	234	73	80	73
query85	1392	301	290	290
query86	454	288	298	288
query87	4354	4165	4198	4165
query88	4045	2409	2396	2396
query89	406	296	286	286
query90	2002	185	183	183
query91	188	150	151	150
query92	62	52	49	49
query93	1636	562	568	562
query94	749	306	273	273
query95	373	262	261	261
query96	606	280	282	280
query97	3303	3106	3158	3106
query98	217	200	194	194
query99	1567	1280	1322	1280
Total cold run time: 299460 ms
Total hot run time: 192748 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

ClickBench: Total hot run time: 29.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d9580e655e6e8da264bd5e8ec83395c2be77161a, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.11	0.11
query5	0.53	0.50	0.52
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.56	0.55	0.58
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.59	0.59
query14	0.78	0.82	0.80
query15	0.83	0.82	0.82
query16	0.40	0.40	0.40
query17	1.00	1.06	1.04
query18	0.25	0.22	0.23
query19	1.90	1.88	1.74
query20	0.02	0.01	0.01
query21	15.40	0.60	0.59
query22	2.35	2.43	1.53
query23	17.10	0.92	0.66
query24	3.10	0.50	0.98
query25	0.13	0.08	0.06
query26	0.49	0.13	0.14
query27	0.05	0.04	0.05
query28	11.01	0.53	0.45
query29	12.61	3.19	3.24
query30	0.25	0.07	0.07
query31	2.85	0.40	0.39
query32	3.24	0.47	0.46
query33	2.98	3.03	3.06
query34	17.14	4.56	4.56
query35	4.57	4.55	4.55
query36	0.68	0.49	0.50
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.32 s
Total hot run time: 29.39 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/12) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 42.08% (11241/26713)
Line Coverage 32.60% (96210/295144)
Region Coverage 30.54% (55252/180914)
Branch Coverage 26.86% (27348/101802)

dataroaring
dataroaring approved these changes Sep 5, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 10ffdbb into branch-3.0 Sep 5, 2025
23 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-55330-branch-3.0 branch September 5, 2025 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @mymeiyi