Skip to content

[feature](function) support utf8 input in initcap #49846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
BiteTheDDDDt merged 3 commits into from
Apr 10, 2025
Merged

[feature](function) support utf8 input in initcap #49846

BiteTheDDDDt merged 3 commits into from
Apr 10, 2025

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Apr 7, 2025 •
edited
Loading

What problem does this PR solve?

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Contributor Author

Mryange commented Apr 7, 2025

run buildall

@Mryange
Copy link
Contributor Author

Mryange commented Apr 7, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 7, 2025

TPC-H: Total hot run time: 36067 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8802b611ee54a82e5ee46375936cd56784b898b, data reload: false

------ Round 1 ----------------------------------
q1	26156	5277	5284	5277
q2	2081	300	204	204
q3	10356	1402	737	737
q4	10231	1096	559	559
q5	7568	2626	2732	2626
q6	206	173	135	135
q7	1011	797	614	614
q8	9318	1566	1249	1249
q9	6817	5300	5282	5282
q10	6991	2382	1912	1912
q11	495	322	269	269
q12	357	393	224	224
q13	17763	3792	3136	3136
q14	248	231	206	206
q15	561	485	485	485
q16	630	625	581	581
q17	603	915	373	373
q18	7702	7219	7164	7164
q19	1220	1098	601	601
q20	343	356	234	234
q21	4458	3749	3236	3236
q22	1072	1039	963	963
Total cold run time: 116187 ms
Total hot run time: 36067 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5491	5493	5458	5458
q2	248	349	231	231
q3	2205	2746	2515	2515
q4	1613	2089	1545	1545
q5	4573	4340	4453	4340
q6	256	176	125	125
q7	2084	1925	1794	1794
q8	2874	2971	2879	2879
q9	7118	7152	7168	7152
q10	3112	3294	2827	2827
q11	645	518	493	493
q12	687	742	568	568
q13	3644	4041	3255	3255
q14	282	316	293	293
q15	551	487	470	470
q16	651	703	639	639
q17	1194	1777	1409	1409
q18	7856	7620	7372	7372
q19	859	799	817	799
q20	1959	2029	1826	1826
q21	5348	4756	4804	4756
q22	1082	1047	935	935
Total cold run time: 54332 ms
Total hot run time: 51681 ms

@doris-robot
Copy link

doris-robot commented Apr 7, 2025

TPC-DS: Total hot run time: 186167 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8802b611ee54a82e5ee46375936cd56784b898b, data reload: false

query1	1036	499	479	479
query2	6548	1894	1896	1894
query3	6762	226	214	214
query4	26014	23876	23016	23016
query5	4360	653	490	490
query6	305	213	203	203
query7	4625	498	285	285
query8	319	261	253	253
query9	8652	2586	2581	2581
query10	490	332	266	266
query11	15892	15104	14817	14817
query12	176	115	113	113
query13	1674	523	411	411
query14	9711	6276	6176	6176
query15	193	180	165	165
query16	7400	594	479	479
query17	1207	724	577	577
query18	1974	416	299	299
query19	190	175	163	163
query20	122	121	120	120
query21	215	123	102	102
query22	4100	4233	4032	4032
query23	33797	32965	32737	32737
query24	8431	2412	2390	2390
query25	531	452	393	393
query26	1234	262	152	152
query27	2751	505	345	345
query28	4318	2410	2405	2405
query29	724	566	425	425
query30	289	221	191	191
query31	944	866	762	762
query32	78	65	64	64
query33	571	368	311	311
query34	796	842	520	520
query35	794	825	738	738
query36	952	964	867	867
query37	119	104	82	82
query38	4215	4385	4209	4209
query39	1449	1403	1422	1403
query40	217	119	109	109
query41	56	55	53	53
query42	118	107	105	105
query43	489	516	475	475
query44	1297	797	792	792
query45	178	173	167	167
query46	873	1027	627	627
query47	1763	1794	1750	1750
query48	379	427	298	298
query49	771	513	429	429
query50	652	677	408	408
query51	4152	4151	4080	4080
query52	133	112	108	108
query53	226	255	184	184
query54	586	574	519	519
query55	82	81	81	81
query56	301	297	319	297
query57	1135	1162	1057	1057
query58	279	259	257	257
query59	2637	2771	2552	2552
query60	334	320	302	302
query61	132	126	130	126
query62	808	748	655	655
query63	238	188	190	188
query64	4248	1017	792	792
query65	4320	4215	4222	4215
query66	1151	441	332	332
query67	15900	15369	15482	15369
query68	7607	906	530	530
query69	469	312	274	274
query70	1208	1122	1068	1068
query71	451	326	300	300
query72	5682	4685	4746	4685
query73	674	637	346	346
query74	9132	9065	8696	8696
query75	3808	3221	2706	2706
query76	3465	1184	748	748
query77	802	375	288	288
query78	9831	10362	9463	9463
query79	2000	824	574	574
query80	584	513	446	446
query81	508	254	233	233
query82	477	128	98	98
query83	265	257	232	232
query84	254	106	84	84
query85	828	359	327	327
query86	336	280	295	280
query87	4556	4556	4304	4304
query88	3676	2227	2222	2222
query89	391	325	278	278
query90	1942	227	207	207
query91	155	152	111	111
query92	81	59	59	59
query93	1515	987	594	594
query94	695	405	315	315
query95	380	291	288	288
query96	484	564	274	274
query97	3131	3254	3200	3200
query98	215	200	210	200
query99	1354	1391	1281	1281
Total cold run time: 274369 ms
Total hot run time: 186167 ms

@doris-robot
Copy link

doris-robot commented Apr 7, 2025

ClickBench: Total hot run time: 31.08 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8802b611ee54a82e5ee46375936cd56784b898b, data reload: false

query1	0.03	0.04	0.03
query2	0.12	0.10	0.10
query3	0.26	0.19	0.20
query4	1.60	0.19	0.19
query5	0.55	0.55	0.54
query6	1.21	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.50
query10	0.61	0.58	0.59
query11	0.16	0.10	0.11
query12	0.14	0.11	0.12
query13	0.62	0.60	0.61
query14	2.82	2.72	2.85
query15	0.94	0.85	0.86
query16	0.39	0.39	0.40
query17	1.07	1.03	1.05
query18	0.22	0.20	0.20
query19	1.98	1.91	1.84
query20	0.02	0.01	0.01
query21	15.38	0.90	0.53
query22	0.75	1.19	0.65
query23	14.95	1.42	0.66
query24	7.02	1.11	0.82
query25	0.54	0.11	0.05
query26	0.54	0.16	0.13
query27	0.06	0.05	0.05
query28	9.94	0.90	0.42
query29	12.54	4.07	3.29
query30	0.25	0.09	0.06
query31	2.84	0.57	0.40
query32	3.22	0.55	0.48
query33	3.04	3.05	3.01
query34	15.86	5.11	4.49
query35	4.49	4.51	4.51
query36	0.67	0.50	0.48
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.96 s
Total hot run time: 31.08 s

@doris-robot
Copy link

doris-robot commented Apr 7, 2025

BE UT Coverage Report

Increment line coverage 100.00% (39/39) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.27% (14025/26834)
Line Coverage 41.05% (121089/294980)
Region Coverage 39.79% (61635/154894)
Branch Coverage 34.46% (30855/89548)

yiguolei
yiguolei previously approved these changes Apr 8, 2025
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 8, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 8, 2025

PR approved by anyone and no changes requested.

@Mryange
Copy link
Contributor Author

Mryange commented Apr 8, 2025

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 8, 2025
@doris-robot
Copy link

doris-robot commented Apr 8, 2025

TPC-H: Total hot run time: 34289 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1563d8135fb7283f641de5a901eb1577a18f01d0, data reload: false

------ Round 1 ----------------------------------
q1	26241	5620	5086	5086
q2	2086	295	198	198
q3	10361	1253	697	697
q4	10245	1012	536	536
q5	7545	2378	2293	2293
q6	194	164	138	138
q7	915	746	608	608
q8	9320	1318	1093	1093
q9	6798	5060	5110	5060
q10	6805	2290	1882	1882
q11	489	293	270	270
q12	354	350	224	224
q13	17753	3697	3161	3161
q14	223	238	203	203
q15	529	490	495	490
q16	618	631	583	583
q17	606	871	375	375
q18	7544	7230	7025	7025
q19	1232	940	566	566
q20	351	333	226	226
q21	4535	3698	2594	2594
q22	1065	1005	981	981
Total cold run time: 115809 ms
Total hot run time: 34289 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5231	5101	5287	5101
q2	241	337	233	233
q3	2176	2618	2276	2276
q4	1452	1890	1493	1493
q5	4525	4458	4408	4408
q6	208	164	127	127
q7	2005	1920	1786	1786
q8	2621	2643	2558	2558
q9	7100	7133	7025	7025
q10	2973	3158	2735	2735
q11	577	501	492	492
q12	692	798	587	587
q13	3626	3912	3256	3256
q14	283	307	283	283
q15	511	457	466	457
q16	654	708	651	651
q17	1152	1561	1353	1353
q18	7754	7544	7453	7453
q19	859	822	887	822
q20	1992	2062	1852	1852
q21	5232	4782	4740	4740
q22	1076	1004	971	971
Total cold run time: 52940 ms
Total hot run time: 50659 ms

@doris-robot
Copy link

doris-robot commented Apr 8, 2025

TPC-DS: Total hot run time: 185713 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1563d8135fb7283f641de5a901eb1577a18f01d0, data reload: false

query1	988	500	472	472
query2	6536	1917	1942	1917
query3	6764	217	218	217
query4	26272	23636	22951	22951
query5	4344	635	488	488
query6	302	217	197	197
query7	4623	485	290	290
query8	311	251	240	240
query9	8600	2541	2528	2528
query10	476	314	246	246
query11	15375	14942	14823	14823
query12	153	104	106	104
query13	1631	516	384	384
query14	8688	6166	6144	6144
query15	217	193	174	174
query16	7148	615	473	473
query17	939	714	592	592
query18	1972	401	307	307
query19	193	184	179	179
query20	126	116	119	116
query21	216	134	111	111
query22	4045	4309	4119	4119
query23	33611	32708	32768	32708
query24	8455	2363	2378	2363
query25	564	468	450	450
query26	1228	276	160	160
query27	2754	517	335	335
query28	4352	2398	2397	2397
query29	778	584	446	446
query30	290	222	248	222
query31	920	853	762	762
query32	71	67	63	63
query33	591	371	323	323
query34	806	860	519	519
query35	793	804	756	756
query36	945	981	872	872
query37	119	101	76	76
query38	4074	4117	3981	3981
query39	1451	1414	1394	1394
query40	221	125	106	106
query41	53	53	59	53
query42	124	114	107	107
query43	497	491	465	465
query44	1311	768	789	768
query45	177	174	170	170
query46	824	1006	606	606
query47	1750	1804	1740	1740
query48	371	393	293	293
query49	777	506	422	422
query50	648	673	414	414
query51	4226	4198	4099	4099
query52	109	106	95	95
query53	228	253	191	191
query54	569	557	492	492
query55	85	82	78	78
query56	309	298	285	285
query57	1125	1145	1056	1056
query58	263	259	259	259
query59	2601	2745	2695	2695
query60	330	314	319	314
query61	137	169	130	130
query62	801	729	657	657
query63	235	201	197	197
query64	4326	1046	675	675
query65	4295	4194	4204	4194
query66	1166	412	313	313
query67	15855	15595	15392	15392
query68	8065	876	504	504
query69	466	314	276	276
query70	1189	1161	1075	1075
query71	478	322	286	286
query72	5807	4834	4859	4834
query73	740	675	350	350
query74	8823	8914	8867	8867
query75	3811	3197	2703	2703
query76	3610	1186	745	745
query77	792	397	285	285
query78	9892	10051	9270	9270
query79	6730	803	530	530
query80	710	513	440	440
query81	460	263	221	221
query82	384	132	100	100
query83	296	262	252	252
query84	294	118	91	91
query85	819	347	309	309
query86	334	282	271	271
query87	4496	4557	4255	4255
query88	2826	2242	2297	2242
query89	464	310	276	276
query90	2052	215	214	214
query91	145	150	110	110
query92	78	63	57	57
query93	3797	927	566	566
query94	676	409	298	298
query95	379	300	291	291
query96	486	572	276	276
query97	3122	3201	3120	3120
query98	222	211	195	195
query99	1400	1417	1239	1239
Total cold run time: 278977 ms
Total hot run time: 185713 ms

@doris-robot
Copy link

doris-robot commented Apr 8, 2025

ClickBench: Total hot run time: 31.92 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1563d8135fb7283f641de5a901eb1577a18f01d0, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.11
query3	0.24	0.20	0.20
query4	1.59	0.20	0.20
query5	0.62	0.58	0.58
query6	1.19	0.72	0.73
query7	0.03	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.53	0.52
query10	0.60	0.59	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.59	0.60
query14	2.69	2.81	2.83
query15	0.92	0.85	0.84
query16	0.38	0.38	0.38
query17	1.06	1.00	1.02
query18	0.21	0.20	0.19
query19	2.00	1.96	1.89
query20	0.02	0.01	0.01
query21	15.36	0.91	0.55
query22	0.75	1.33	0.98
query23	14.72	1.36	0.66
query24	6.79	1.70	1.01
query25	0.47	0.21	0.27
query26	0.59	0.16	0.14
query27	0.05	0.05	0.04
query28	10.15	0.82	0.43
query29	12.53	3.96	3.35
query30	0.26	0.09	0.07
query31	2.85	0.57	0.38
query32	3.23	0.53	0.46
query33	2.95	3.02	3.09
query34	15.79	5.06	4.46
query35	4.52	4.48	4.48
query36	0.66	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.41 s
Total hot run time: 31.92 s

@Mryange
Copy link
Contributor Author

Mryange commented Apr 8, 2025

run cloud_p0

@doris-robot
Copy link

doris-robot commented Apr 8, 2025

BE UT Coverage Report

Increment line coverage 100.00% (39/39) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.27% (14036/26852)
Line Coverage 41.04% (121083/295034)
Region Coverage 39.78% (61629/154909)
Branch Coverage 34.45% (30850/89538)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 10, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 10, 2025

PR approved by at least one committer and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit 8f53532 into apache:master Apr 10, 2025
24 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 10, 2025
@Mryange
[feature](function) support utf8 input in initcap (#49846)
9f38d20 
### What problem does this PR solve?


Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Apr 10, 2025
Merged
Mryange added a commit to Mryange/doris that referenced this pull request Apr 11, 2025
@Mryange
[feature](function) support utf8 input in initcap (apache#49846)
d7bac2b 
Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Mryange added a commit to Mryange/doris that referenced this pull request Apr 11, 2025
@Mryange
[feature](function) support utf8 input in initcap (apache#49846)
eb902a8 
Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
yiguolei pushed a commit that referenced this pull request Apr 11, 2025
@Mryange
branch-2.1: [feature](function) support utf8 input in initcap #49846 (#…
d8a2742 
…49977)
dataroaring pushed a commit that referenced this pull request Apr 22, 2025
@github-actions @Mryange
branch-3.0: [feature](function) support utf8 input in initcap #49846 (#…
35548b4 
…49933)

Cherry-picked from #49846

Co-authored-by: Mryange <yanxuecheng@selectdb.com>
@yiguolei yiguolei mentioned this pull request May 13, 2025
Closed
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@Mryange @koarz
[feature](function) support utf8 input in initcap (apache#49846)
306f5d7 
### What problem does this PR solve?


Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@Gabriel39 Gabriel39 Gabriel39 approved these changes

@yiguolei yiguolei yiguolei left review comments

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.6-merged p0_w reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Mryange @hello-stephen @doris-robot @BiteTheDDDDt @yiguolei @Gabriel39 @dataroaring