Skip to content

branch-3.0: [fix](mem) heap-buffer-overflow for function convert_to #46405 #46483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jan 17, 2025
Merged

branch-3.0: [fix](mem) heap-buffer-overflow for function convert_to #46405 #46483

dataroaring merged 1 commit into from
Jan 17, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 6, 2025

Cherry-picked from #46405

@cambyzju
[fix](mem) heap-buffer-overflow for function convert_to (#46405)
1753172 
### What problem does this PR solve?

Reproduce SQL with ASAN version:
`select convert('装装装装装' using gbk);`

Then be crashed:
```
=================================================================
==1830466==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606002aeec20 at pc 0x560826fb3e66 bp 0x7fc3816a5890 sp 0x7fc3816a5058

WRITE of size 10 at 0x606002aeec20 thread T711 (brpc_light)
```

But if we use release version, we found the result is not correct, and
the memory maybe already corrupted:
```
> select convert('装装装装装' using gbk);
+---------------------------------------------------------+
| convert_to('装装装装装', 'gbk')                         |
+---------------------------------------------------------+
| ~zhuangdang~zhuangdang~zhuangdang~zhuangdang~zhu        |
+---------------------------------------------------------+
1 row in set (0.04 sec)
```

The correct answer should be:
```
> select convert('装装装装装' using gbk);
+--------------------------------------+
| convert_to('装装装装装', 'gbk')      |
+--------------------------------------+
| ~zhuang~zhuang~zhuang~zhuang~zhuang  |
+--------------------------------------+
1 row in set (0.06 sec)
```
@Thearas
Copy link
Contributor

Thearas commented Jan 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 6, 2025
@dataroaring dataroaring reopened this Jan 6, 2025
@Thearas
Copy link
Contributor

Thearas commented Jan 6, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 6, 2025

TPC-H: Total hot run time: 41065 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1753172f892fb17e73b07f68ee7eaeb911afd0a1, data reload: false

------ Round 1 ----------------------------------
q1	17576	7344	7258	7258
q2	2057	170	163	163
q3	10707	1160	1219	1160
q4	10674	766	672	672
q5	7899	2906	2920	2906
q6	236	148	147	147
q7	977	626	618	618
q8	9638	1992	2037	1992
q9	6808	6546	6517	6517
q10	8611	2251	2273	2251
q11	461	270	272	270
q12	411	215	205	205
q13	17805	3007	2999	2999
q14	249	209	205	205
q15	570	523	517	517
q16	706	618	608	608
q17	982	619	541	541
q18	7284	6637	6686	6637
q19	1400	1024	1075	1024
q20	472	199	196	196
q21	4220	3206	3305	3206
q22	1090	973	976	973
Total cold run time: 110833 ms
Total hot run time: 41065 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7230	7215	7307	7215
q2	332	231	223	223
q3	3131	3051	2970	2970
q4	2046	1772	1827	1772
q5	5693	5721	5738	5721
q6	217	137	136	136
q7	2210	1816	1738	1738
q8	3382	3524	3510	3510
q9	8918	8901	8895	8895
q10	3634	3506	3504	3504
q11	602	504	497	497
q12	813	605	584	584
q13	6213	3169	3133	3133
q14	316	265	278	265
q15	552	506	504	504
q16	704	673	662	662
q17	1778	1569	1555	1555
q18	7739	7331	7351	7331
q19	1654	1417	1476	1417
q20	2064	1821	1796	1796
q21	5396	5138	5159	5138
q22	1087	1021	985	985
Total cold run time: 65711 ms
Total hot run time: 59551 ms

@doris-robot
Copy link

doris-robot commented Jan 6, 2025

TPC-DS: Total hot run time: 191692 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1753172f892fb17e73b07f68ee7eaeb911afd0a1, data reload: false

query1	968	371	370	370
query2	6519	2116	2095	2095
query3	6717	219	223	219
query4	34151	23528	23588	23528
query5	4260	438	452	438
query6	279	169	170	169
query7	4623	308	315	308
query8	277	218	207	207
query9	9703	2672	2655	2655
query10	445	255	254	254
query11	17941	15053	15332	15053
query12	152	103	102	102
query13	1657	421	414	414
query14	8679	7235	7165	7165
query15	273	171	180	171
query16	8163	452	465	452
query17	1667	586	560	560
query18	2135	315	324	315
query19	386	157	156	156
query20	114	106	106	106
query21	206	103	101	101
query22	4645	4356	4416	4356
query23	34733	33959	34201	33959
query24	11595	2814	2816	2814
query25	691	420	417	417
query26	1653	174	170	170
query27	2874	350	349	349
query28	8030	2446	2417	2417
query29	1007	449	438	438
query30	333	170	161	161
query31	1020	811	797	797
query32	102	60	60	60
query33	792	290	312	290
query34	927	498	535	498
query35	864	711	709	709
query36	1078	954	945	945
query37	146	79	76	76
query38	3985	3798	3885	3798
query39	1480	1437	1465	1437
query40	289	102	102	102
query41	54	50	52	50
query42	113	105	100	100
query43	529	492	498	492
query44	1243	811	804	804
query45	190	171	169	169
query46	1172	688	707	688
query47	1943	1886	1864	1864
query48	471	385	397	385
query49	1250	398	396	396
query50	831	409	411	409
query51	7278	7171	7106	7106
query52	106	97	96	96
query53	257	184	185	184
query54	1313	479	470	470
query55	83	81	78	78
query56	281	248	256	248
query57	1239	1101	1090	1090
query58	240	218	214	214
query59	3339	2974	2927	2927
query60	319	253	262	253
query61	115	113	116	113
query62	873	662	662	662
query63	214	189	190	189
query64	5281	657	639	639
query65	3275	3194	3180	3180
query66	1440	319	322	319
query67	15959	15524	15694	15524
query68	4625	566	575	566
query69	449	263	264	263
query70	1205	1130	1144	1130
query71	416	256	257	256
query72	6445	4024	4065	4024
query73	750	341	344	341
query74	10162	9008	9122	9008
query75	3362	2671	2639	2639
query76	2900	1128	951	951
query77	429	273	277	273
query78	10520	9657	9501	9501
query79	1095	594	604	594
query80	766	431	432	431
query81	513	238	238	238
query82	1313	116	114	114
query83	242	148	144	144
query84	238	84	79	79
query85	1120	299	305	299
query86	316	297	298	297
query87	4464	4264	4244	4244
query88	3540	2414	2366	2366
query89	403	294	287	287
query90	2088	187	181	181
query91	183	152	151	151
query92	59	51	52	51
query93	1037	557	552	552
query94	823	286	301	286
query95	360	258	249	249
query96	668	280	282	280
query97	3360	3156	3123	3123
query98	217	210	193	193
query99	1544	1301	1362	1301
Total cold run time: 300579 ms
Total hot run time: 191692 ms

@doris-robot
Copy link

doris-robot commented Jan 6, 2025

ClickBench: Total hot run time: 33.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1753172f892fb17e73b07f68ee7eaeb911afd0a1, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.53	0.50	0.52
query6	1.14	0.72	0.71
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.56	0.55	0.54
query11	0.15	0.10	0.10
query12	0.14	0.12	0.11
query13	0.61	0.62	0.59
query14	3.06	3.06	3.04
query15	0.89	0.83	0.82
query16	0.42	0.38	0.38
query17	1.07	1.07	0.99
query18	0.22	0.22	0.21
query19	1.93	1.83	1.97
query20	0.01	0.01	0.01
query21	15.36	0.57	0.59
query22	2.70	2.51	2.87
query23	17.06	0.96	0.80
query24	3.58	1.97	1.19
query25	0.29	0.32	0.06
query26	0.46	0.13	0.13
query27	0.04	0.05	0.04
query28	9.56	1.10	1.06
query29	12.58	3.24	3.22
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.26	0.46	0.46
query33	2.96	3.03	3.05
query34	16.98	4.51	4.50
query35	4.51	4.51	4.48
query36	0.68	0.51	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 106.93 s
Total hot run time: 33.87 s

@dataroaring dataroaring merged commit a3cc265 into branch-3.0 Jan 17, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-46405-branch-3.0 branch January 17, 2025 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @dataroaring @cambyzju