Skip to content

[fix](statistics)Show table stats need to forward to master use FORWARD_NO_SYNC #48189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 1 commit into from
Feb 21, 2025
Merged

[fix](statistics)Show table stats need to forward to master use FORWARD_NO_SYNC #48189

englefly merged 1 commit into from
Feb 21, 2025

Conversation

@Jibing-Li
Copy link
Contributor

@Jibing-Li Jibing-Li commented Feb 21, 2025

What problem does this PR solve?

Forward 'show table stats' command to master to make regression case stable.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Jibing-Li Jibing-Li marked this pull request as ready for review February 21, 2025 10:19
@Jibing-Li
Copy link
Contributor Author

Jibing-Li commented Feb 21, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 21, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Feb 21, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 21, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Feb 21, 2025

TPC-H: Total hot run time: 31311 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5e5faa1b50dab7c53aa4d2a1b9d69fd87f160934, data reload: false

------ Round 1 ----------------------------------
q1	17792	5381	5058	5058
q2	2047	298	169	169
q3	10420	1224	734	734
q4	10214	1018	539	539
q5	7532	2446	2289	2289
q6	192	163	130	130
q7	883	759	586	586
q8	9295	1287	1175	1175
q9	4813	4578	4744	4578
q10	6809	2325	1891	1891
q11	505	277	253	253
q12	348	345	221	221
q13	17753	3656	3042	3042
q14	234	235	216	216
q15	498	466	456	456
q16	626	598	578	578
q17	563	842	338	338
q18	7084	6177	6172	6172
q19	1206	947	520	520
q20	304	328	188	188
q21	2707	2145	1880	1880
q22	356	323	298	298
Total cold run time: 102181 ms
Total hot run time: 31311 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5525	5083	5131	5083
q2	237	323	230	230
q3	2168	2690	2306	2306
q4	1429	1813	1394	1394
q5	4237	4071	4154	4071
q6	204	165	124	124
q7	1890	1833	1642	1642
q8	2598	2551	2486	2486
q9	7283	7299	7197	7197
q10	3020	3248	2802	2802
q11	574	543	497	497
q12	722	786	647	647
q13	3367	3945	3322	3322
q14	303	301	271	271
q15	501	488	457	457
q16	653	684	618	618
q17	1120	1604	1292	1292
q18	7669	7501	7305	7305
q19	776	774	998	774
q20	2019	2016	1858	1858
q21	5485	5003	4682	4682
q22	592	544	552	544
Total cold run time: 52372 ms
Total hot run time: 49602 ms

@doris-robot
Copy link

doris-robot commented Feb 21, 2025

TPC-DS: Total hot run time: 190966 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5e5faa1b50dab7c53aa4d2a1b9d69fd87f160934, data reload: false

query1	1323	931	971	931
query2	6212	1884	1831	1831
query3	11101	4800	4604	4604
query4	53024	24211	23431	23431
query5	5140	618	509	509
query6	361	226	186	186
query7	5127	542	298	298
query8	333	248	254	248
query9	6119	2576	2574	2574
query10	441	313	256	256
query11	15194	15052	14812	14812
query12	154	108	118	108
query13	1140	520	376	376
query14	10294	7057	6341	6341
query15	211	206	179	179
query16	7093	700	512	512
query17	1125	715	608	608
query18	1536	422	310	310
query19	208	211	166	166
query20	131	130	124	124
query21	217	135	109	109
query22	4424	4544	4489	4489
query23	34236	33391	33334	33334
query24	5736	2432	2473	2432
query25	473	463	405	405
query26	733	301	172	172
query27	1823	501	338	338
query28	2892	2493	2427	2427
query29	568	564	455	455
query30	214	187	168	168
query31	893	869	833	833
query32	75	61	64	61
query33	480	358	307	307
query34	764	888	519	519
query35	825	843	755	755
query36	976	998	922	922
query37	130	104	82	82
query38	4280	4375	4273	4273
query39	1495	1451	1440	1440
query40	215	120	112	112
query41	53	50	48	48
query42	131	123	106	106
query43	512	525	476	476
query44	1335	801	796	796
query45	194	173	164	164
query46	919	1102	672	672
query47	1824	1848	1798	1798
query48	424	442	321	321
query49	697	528	438	438
query50	758	794	434	434
query51	4353	4295	4217	4217
query52	110	107	96	96
query53	245	275	188	188
query54	492	519	451	451
query55	85	88	82	82
query56	274	265	276	265
query57	1175	1173	1107	1107
query58	270	263	247	247
query59	2773	2904	2806	2806
query60	300	303	266	266
query61	126	125	125	125
query62	749	732	664	664
query63	239	202	232	202
query64	2026	1048	692	692
query65	3186	3153	3266	3153
query66	738	396	293	293
query67	15994	15505	15474	15474
query68	5461	803	515	515
query69	535	297	268	268
query70	1222	1093	1125	1093
query71	434	306	260	260
query72	6049	3733	4003	3733
query73	1084	778	360	360
query74	9017	9280	8973	8973
query75	3247	3153	2662	2662
query76	3899	1231	765	765
query77	526	406	281	281
query78	10027	10117	9211	9211
query79	2251	875	617	617
query80	628	560	453	453
query81	505	274	243	243
query82	208	135	96	96
query83	185	183	158	158
query84	288	93	78	78
query85	783	356	366	356
query86	357	288	291	288
query87	4397	4462	4393	4393
query88	3813	2203	2195	2195
query89	445	321	289	289
query90	1648	193	190	190
query91	135	142	112	112
query92	75	63	57	57
query93	1874	1021	599	599
query94	681	421	293	293
query95	354	270	257	257
query96	511	594	288	288
query97	2812	2832	2805	2805
query98	231	211	202	202
query99	1337	1384	1258	1258
Total cold run time: 292717 ms
Total hot run time: 190966 ms

@doris-robot
Copy link

doris-robot commented Feb 21, 2025

ClickBench: Total hot run time: 30.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5e5faa1b50dab7c53aa4d2a1b9d69fd87f160934, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.25	0.06	0.06
query4	1.62	0.10	0.11
query5	0.42	0.43	0.39
query6	1.16	0.65	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.51
query10	0.57	0.58	0.58
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.64	0.61	0.61
query14	2.78	2.71	2.67
query15	0.92	0.85	0.86
query16	0.38	0.37	0.38
query17	1.01	1.03	1.03
query18	0.21	0.19	0.19
query19	1.93	1.75	1.99
query20	0.01	0.00	0.01
query21	15.35	0.87	0.56
query22	0.75	1.14	0.61
query23	15.02	1.41	0.59
query24	7.21	0.93	0.83
query25	0.55	0.22	0.11
query26	0.54	0.16	0.13
query27	0.05	0.05	0.06
query28	9.07	0.86	0.44
query29	12.56	4.00	3.30
query30	0.25	0.08	0.06
query31	2.84	0.58	0.39
query32	3.22	0.54	0.46
query33	2.99	3.13	3.02
query34	15.87	5.18	4.52
query35	4.57	4.61	4.54
query36	0.65	0.48	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.15	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.92 s
Total hot run time: 30.5 s

@englefly englefly merged commit 56b744a into apache:master Feb 21, 2025
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 21, 2025
@Jibing-Li
[fix](statistics)Show table stats need to forward to master use FORWA…
a0d7772 
…RD_NO_SYNC (#48189)

### What problem does this PR solve?

Forward 'show table stats' command to master to make regression case
stable.
@github-actions github-actions bot mentioned this pull request Feb 21, 2025
Merged
@Jibing-Li Jibing-Li deleted the forward branch February 24, 2025 01:49
dataroaring pushed a commit that referenced this pull request Feb 24, 2025
@github-actions @Jibing-Li
branch-3.0: [fix](statistics)Show table stats need to forward to mast…
578c99d 
…er use FORWARD_NO_SYNC #48189 (#48197)

Cherry-picked from #48189

Co-authored-by: James <lijibing@selectdb.com>
seawinde pushed a commit to seawinde/doris that referenced this pull request Feb 28, 2025
@Jibing-Li @seawinde
[fix](statistics)Show table stats need to forward to master use FORWA…
a8cec19 
…RD_NO_SYNC (apache#48189)

### What problem does this PR solve?

Forward 'show table stats' command to master to make regression case
stable.
@Jibing-Li Jibing-Li mentioned this pull request Apr 21, 2025
Merged
16 tasks
Jibing-Li added a commit that referenced this pull request Apr 21, 2025
@Jibing-Li
[fix](statistics)No need to forward show table stats command. (#50207)
66b8e8d 
### What problem does this PR solve?

No need to forward show table stats command. Forwarding to master is to
make sure regression case can pass. But it is not the right way. Because
forward to master makes it impossible to get Observer's table stats.
After this change, we should fix the case if it depends on forward.
Previous pr that forward show table stats to master:
#48189
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Apr 21, 2025
@Jibing-Li
[fix](statistics)No need to forward show table stats command. (apache…
d914d3c 
…#50207)

No need to forward show table stats command. Forwarding to master is to
make sure regression case can pass. But it is not the right way. Because
forward to master makes it impossible to get Observer's table stats.
After this change, we should fix the case if it depends on forward.
Previous pr that forward show table stats to master:
apache#48189
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@Jibing-Li @koarz
[fix](statistics)Show table stats need to forward to master use FORWA…
cf6f842 
…RD_NO_SYNC (apache#48189)

### What problem does this PR solve?

Forward 'show table stats' command to master to make regression case
stable.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@Jibing-Li @koarz
[fix](statistics)No need to forward show table stats command. (apache…
5945426 
…#50207)

### What problem does this PR solve?

No need to forward show table stats command. Forwarding to master is to
make sure regression case can pass. But it is not the right way. Because
forward to master makes it impossible to get Observer's table stats.
After this change, we should fix the case if it depends on forward.
Previous pr that forward show table stats to master:
apache#48189
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.5-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Jibing-Li @Thearas @doris-robot @englefly @morrySnow @dataroaring