Skip to content

[fix](schema-change) Rebuild distribution info according to original order #54024

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gavinchou merged 1 commit into from
Jul 29, 2025
Merged

[fix](schema-change) Rebuild distribution info according to original order #54024

gavinchou merged 1 commit into from
Jul 29, 2025

Conversation

@Hastyshell
Copy link
Collaborator

@Hastyshell Hastyshell commented Jul 29, 2025

What problem does this PR solve?

Problem Summary:

Rebuild distribution info should use the column order from original distribution info, rather than base schema

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jul 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Hastyshell
Copy link
Collaborator Author

Hastyshell commented Jul 29, 2025

run buildall

dataroaring
dataroaring previously approved these changes Jul 29, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 29, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 29, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 29, 2025

PR approved by anyone and no changes requested.

@dataroaring dataroaring added usercase Important user case type label dev/3.0.x dev/3.1.x and removed reviewed labels Jul 29, 2025
@Hastyshell Hastyshell force-pushed the fix-rebuild-distribution branch from b7d5c61 to 3bd3046 Compare July 29, 2025 08:59
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 29, 2025
@Hastyshell
Copy link
Collaborator Author

Hastyshell commented Jul 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 29, 2025

TPC-H: Total hot run time: 33680 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3bd30465179894dbb4baf15ed26a3781f03f64a1, data reload: false

------ Round 1 ----------------------------------
q1	17615	5690	5456	5456
q2	1946	297	191	191
q3	10280	1277	680	680
q4	10214	986	527	527
q5	7527	2342	2196	2196
q6	176	161	130	130
q7	901	758	625	625
q8	9300	1309	1034	1034
q9	6737	5066	5093	5066
q10	6888	2353	1960	1960
q11	473	276	273	273
q12	343	358	219	219
q13	17768	3445	2986	2986
q14	228	231	213	213
q15	515	460	458	458
q16	429	434	369	369
q17	561	835	359	359
q18	7204	7085	6985	6985
q19	2545	985	519	519
q20	316	309	222	222
q21	3331	2428	2228	2228
q22	1047	1048	984	984
Total cold run time: 106344 ms
Total hot run time: 33680 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5902	5687	5683	5683
q2	228	309	212	212
q3	2097	2576	2166	2166
q4	1271	1711	1333	1333
q5	4052	4402	4426	4402
q6	217	180	140	140
q7	2016	1933	1976	1933
q8	2499	2472	2451	2451
q9	7316	7318	7207	7207
q10	3231	3411	2908	2908
q11	536	507	489	489
q12	686	852	658	658
q13	3453	3798	3201	3201
q14	275	315	306	306
q15	500	451	443	443
q16	442	485	440	440
q17	1146	1391	1457	1391
q18	8111	7385	7685	7385
q19	12514	1035	1113	1035
q20	1994	2071	1787	1787
q21	14617	4217	4170	4170
q22	1049	1076	1003	1003
Total cold run time: 74152 ms
Total hot run time: 50743 ms

dataroaring
dataroaring previously approved these changes Jul 29, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jul 29, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 29, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 29, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jul 29, 2025

TPC-DS: Total hot run time: 173922 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3bd30465179894dbb4baf15ed26a3781f03f64a1, data reload: false

============================================
query1	999	386	424	386
query2	6544	1706	1631	1631
query3	6749	229	226	226
query4	27003	23945	23448	23448
query5	4348	637	545	545
query6	326	252	236	236
query7	4642	517	308	308
query8	274	232	214	214
query9	8561	3163	3128	3128
query10	491	345	299	299
query11	16068	15310	14777	14777
query12	194	135	129	129
query13	1654	564	400	400
query14	9397	7708	7674	7674
query15	208	189	177	177
query16	7124	631	496	496
query17	955	756	622	622
query18	1973	445	349	349
query19	230	204	188	188
query20	149	144	152	144
query21	216	127	109	109
query22	4194	3947	3924	3924
query23	34196	34108	34043	34043
query24	7550	2395	2446	2395
query25	517	509	437	437
query26	724	296	162	162
query27	2371	493	353	353
query28	3055	2291	2279	2279
query29	663	604	496	496
query30	284	235	206	206
query31	878	795	717	717
query32	87	79	109	79
query33	535	425	378	378
query34	802	844	522	522
query35	789	824	747	747
query36	1038	1056	953	953
query37	130	109	93	93
query38	3976	4069	4028	4028
query39	1453	1399	1388	1388
query40	245	146	141	141
query41	62	57	53	53
query42	140	121	156	121
query43	508	500	492	492
query44	1395	870	884	870
query45	197	196	188	188
query46	943	1055	677	677
query47	1843	1838	1742	1742
query48	394	427	318	318
query49	677	504	428	428
query50	678	686	427	427
query51	5488	5481	5483	5481
query52	129	140	118	118
query53	260	287	212	212
query54	662	640	576	576
query55	93	91	89	89
query56	362	383	368	368
query57	1205	1205	1168	1168
query58	339	336	351	336
query59	2606	2546	2508	2508
query60	436	416	399	399
query61	152	146	147	146
query62	765	732	661	661
query63	259	218	218	218
query64	2439	1171	848	848
query65	4195	4127	4097	4097
query66	1029	451	397	397
query67	query68	17131	879	955	879
query69	1039	283	292	283
query70	1396	1147	1165	1147
query71	721	318	332	318
query72	9178	2307	2164	2164
query73	3461	678	363	363
query74	9053	8643	8884	8643
query75	7556	3137	2687	2687
query76	8876	1215	792	792
query77	1176	412	345	345
query78	9749	12108	9240	9240
query79	15663	642	582	582
query80	2814	575	479	479
query81	543	278	230	230
query82	553	149	117	117
query83	396	285	273	273
query84	300	95	89	89
query85	1096	371	381	371
query86	367	333	307	307
query87	4308	4374	4174	4174
query88	5633	2311	2336	2311
query89	506	371	312	312
query90	2487	235	235	235
query91	143	139	110	110
query92	92	74	67	67
query93	7005	969	675	675
query94	1232	389	277	277
query95	431	332	324	324
query96	506	600	288	288
query97	2740	2795	2573	2573
query98	279	225	228	225
query99	1478	1358	1277	1277
Total cold run time: 306756 ms
Total hot run time: 173922 ms

@doris-robot
Copy link

doris-robot commented Jul 29, 2025

ClickBench: Total hot run time: 33.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3bd30465179894dbb4baf15ed26a3781f03f64a1, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.41	0.44	0.43
query6	1.18	0.68	0.67
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.55	0.47	0.48
query10	0.54	0.54	0.54
query11	0.15	0.10	0.10
query12	0.14	0.11	0.10
query13	0.65	0.66	0.64
query14	0.91	1.09	1.09
query15	0.90	0.91	0.88
query16	0.40	0.40	0.39
query17	1.05	1.08	1.06
query18	0.22	0.20	0.19
query19	2.06	1.93	1.94
query20	0.02	0.01	0.01
query21	15.36	0.84	0.53
query22	0.79	1.17	0.77
query23	14.83	1.16	0.62
query24	6.88	1.19	0.90
query25	0.51	0.19	0.26
query26	0.61	0.15	0.14
query27	0.06	0.06	0.05
query28	10.24	0.84	0.44
query29	12.57	3.82	3.36
query30	3.06	2.97	2.97
query31	2.82	0.57	0.40
query32	3.23	0.58	0.49
query33	3.02	3.16	3.22
query34	16.04	5.26	4.95
query35	4.81	4.92	5.04
query36	0.69	0.51	0.48
query37	0.10	0.08	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.05	0.03	0.03
Total cold run time: 107.28 s
Total hot run time: 33.54 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 29, 2025

FE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

@Hastyshell Hastyshell force-pushed the fix-rebuild-distribution branch from 3bd3046 to 8407cf1 Compare July 29, 2025 11:26
@Hastyshell Hastyshell force-pushed the fix-rebuild-distribution branch from 8407cf1 to d2c3e92 Compare July 29, 2025 11:27
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 29, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 29, 2025

PR approved by at least one committer and no changes requested.

@gavinchou gavinchou merged commit eac6606 into apache:master Jul 29, 2025
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 29, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
9f37f66 
…order (#54024)

Rebuild distribution info should use the column order from original
distribution info, rather than base schema
@github-actions github-actions bot mentioned this pull request Jul 29, 2025
Closed
github-actions bot pushed a commit that referenced this pull request Jul 29, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
275f467 
…order (#54024)

Rebuild distribution info should use the column order from original
distribution info, rather than base schema
@github-actions github-actions bot mentioned this pull request Jul 29, 2025
Closed
@Hastyshell Hastyshell mentioned this pull request Jul 30, 2025
Merged
16 tasks
This was referenced Jul 30, 2025
Merged
Merged
Merged
dataroaring pushed a commit that referenced this pull request Jul 30, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info should update default …
488b291 
…distribution info (#54072)

### What problem does this PR solve?

Related PR: #54024
github-actions bot pushed a commit that referenced this pull request Jul 30, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info should update default …
56b3e14 
…distribution info (#54072)

### What problem does this PR solve?

Related PR: #54024
github-actions bot pushed a commit that referenced this pull request Jul 30, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info should update default …
7d684fe 
…distribution info (#54072)

### What problem does this PR solve?

Related PR: #54024
dataroaring pushed a commit that referenced this pull request Jul 30, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
6bb8596 
…order (#54024) (#54072) (#54109)

### What problem does this PR solve?

cherry-pick: #54024 #54072
morrySnow pushed a commit that referenced this pull request Jul 31, 2025
@Hastyshell
branch-3.1: [fix](schema-change) Rebuild distribution info according …
fc0bbd0 
…to original order #54024 #54072 (#54110)

Cherry-pick: #54024 #54072
yiguolei pushed a commit that referenced this pull request Aug 1, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
c9e1435 
…order (#54024) (#54072) (#54111)

### What problem does this PR solve?

Cherry-pick: #54024 #54072

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@bobhan1 bobhan1 mentioned this pull request Aug 5, 2025
Merged
16 tasks
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Aug 8, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
61a20a1 
…order apache#54024 apache#54072  (apache#4233)

pick: 
apache#54024
apache#54072
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Open
Gabriel39 pushed a commit to Gabriel39/incubator-doris that referenced this pull request Sep 25, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
7a1429b 
…order apache#54024 apache#54072 (apache#4231)

pick: 
apache#54024
apache#54072
deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Dec 19, 2025
@Hastyshell
[fix](schema-change) Rebuild distribution info according to original …
f37a834 
…order (apache#54024) (apache#54072) (apache#54109) (apache#380)

### What problem does this PR solve?

cherry-pick: apache#54024 apache#54072
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged p0_w reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@Hastyshell @Thearas @doris-robot @hello-stephen @gavinchou @dataroaring @yiguolei @morrySnow