[feature](inverted index) multi_match function add #37722

zzzxl1993 · 2024-07-12T10:28:13Z

Proposed changes

select * from tbl where multi_match(c1, 'c2, c3, c4', 'phrase_prefix', 'xxx');
multi_match performs a match_phrase_prefix query on the columns specified in the first and second parameters. The value to be searched is specified in the fourth parameter.

doris-robot · 2024-07-12T10:28:18Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

airborne12

LGTM

zzzxl1993 · 2024-07-15T15:24:22Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2024-07-15T15:31:11Z

be/src/vec/functions/function_multi_match.cpp

+ return Status::OK();
+}
+
+Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,


warning: function 'eval_inverted_index' exceeds recommended size/complexity thresholds [readability-function-size]

Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context, ^

Additional context

be/src/vec/functions/function_multi_match.cpp:94: 85 lines including whitespace and comments (threshold 80)

Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context, ^

github-actions · 2024-07-15T15:31:12Z

be/src/vec/functions/function_multi_match.h

+
+#pragma once
+
+#include <boost/algorithm/string/split.hpp>


warning: 'boost/algorithm/string/split.hpp' file not found [clang-diagnostic-error]

#include <boost/algorithm/string/split.hpp> ^

zzzxl1993 · 2024-07-16T02:31:38Z

run buildall

doris-robot · 2024-07-16T03:47:24Z

TPC-H: Total hot run time: 39750 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false

------ Round 1 ----------------------------------
q1	17607	4949	4288	4288
q2	2022	194	184	184
q3	10540	1232	1141	1141
q4	10202	842	781	781
q5	7549	2789	2613	2613
q6	217	134	136	134
q7	961	594	603	594
q8	9341	2090	2081	2081
q9	8806	6616	6596	6596
q10	8711	3843	3783	3783
q11	437	231	238	231
q12	410	222	217	217
q13	17771	2954	2965	2954
q14	275	245	250	245
q15	516	494	488	488
q16	513	404	376	376
q17	981	659	656	656
q18	8046	7514	7333	7333
q19	4903	1319	1465	1319
q20	699	333	315	315
q21	4916	3151	3134	3134
q22	348	301	287	287
Total cold run time: 115771 ms
Total hot run time: 39750 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4214	4280	4214
q2	382	274	273	273
q3	3015	2863	2851	2851
q4	2005	1723	1678	1678
q5	5710	5522	5556	5522
q6	224	137	131	131
q7	2219	1893	1870	1870
q8	3300	3428	3440	3428
q9	8826	8788	8939	8788
q10	4072	3878	3809	3809
q11	605	518	510	510
q12	828	629	641	629
q13	17303	3139	3209	3139
q14	331	286	299	286
q15	535	496	494	494
q16	500	438	466	438
q17	1817	1497	1506	1497
q18	8301	7804	7833	7804
q19	2261	1513	1569	1513
q20	2182	1899	1842	1842
q21	10280	4641	4740	4641
q22	603	511	509	509
Total cold run time: 79694 ms
Total hot run time: 55866 ms

doris-robot · 2024-07-16T03:58:36Z

TPC-DS: Total hot run time: 174312 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false

query1	915	372	371	371
query2	6455	1859	1846	1846
query3	6634	204	216	204
query4	23521	17355	17143	17143
query5	3770	481	475	475
query6	267	164	162	162
query7	4588	295	282	282
query8	244	187	188	187
query9	8399	2469	2479	2469
query10	443	293	279	279
query11	11072	10148	10150	10148
query12	114	99	84	84
query13	1648	391	364	364
query14	10094	8026	7806	7806
query15	238	165	167	165
query16	7692	327	323	323
query17	1788	567	544	544
query18	1310	294	285	285
query19	202	158	160	158
query20	91	86	82	82
query21	204	129	127	127
query22	4297	4119	4145	4119
query23	34010	33724	33758	33724
query24	10827	2917	3085	2917
query25	598	406	384	384
query26	704	150	148	148
query27	2239	285	279	279
query28	6031	2098	2089	2089
query29	884	646	622	622
query30	257	154	155	154
query31	958	760	770	760
query32	96	52	56	52
query33	705	295	301	295
query34	897	490	488	488
query35	692	586	598	586
query36	1117	985	976	976
query37	157	87	86	86
query38	3006	2929	2853	2853
query39	911	863	866	863
query40	215	124	129	124
query41	54	45	44	44
query42	116	103	102	102
query43	506	451	467	451
query44	1106	712	730	712
query45	197	159	159	159
query46	1077	723	750	723
query47	1861	1794	1781	1781
query48	389	292	287	287
query49	838	401	416	401
query50	772	396	395	395
query51	6866	6733	6743	6733
query52	110	99	94	94
query53	356	281	288	281
query54	902	451	446	446
query55	72	72	73	72
query56	303	263	262	262
query57	1129	1058	1061	1058
query58	237	241	246	241
query59	2829	2613	2641	2613
query60	293	280	273	273
query61	94	94	120	94
query62	790	649	640	640
query63	314	285	283	283
query64	9096	2214	1680	1680
query65	3164	3115	3120	3115
query66	761	325	320	320
query67	15504	14954	15065	14954
query68	4986	537	540	537
query69	694	484	335	335
query70	1182	1122	1129	1122
query71	485	281	275	275
query72	9046	5656	6128	5656
query73	771	327	325	325
query74	6228	5638	5689	5638
query75	4429	2699	2735	2699
query76	3612	959	924	924
query77	748	323	308	308
query78	11964	9333	9035	9035
query79	5855	517	519	517
query80	936	483	474	474
query81	583	225	218	218
query82	675	133	135	133
query83	377	166	171	166
query84	281	86	89	86
query85	704	383	291	291
query86	431	321	321	321
query87	3388	3115	3103	3103
query88	4708	2447	2472	2447
query89	480	375	375	375
query90	1961	198	193	193
query91	129	100	103	100
query92	62	49	58	49
query93	3773	504	495	495
query94	1218	212	218	212
query95	404	316	314	314
query96	607	290	276	276
query97	3206	3023	3049	3023
query98	226	204	196	196
query99	1556	1232	1233	1232
Total cold run time: 283731 ms
Total hot run time: 174312 ms

doris-robot · 2024-07-16T04:04:08Z

ClickBench: Total hot run time: 31.17 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.66	0.08	0.08
query5	0.50	0.49	0.49
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.53	0.50	0.49
query10	0.55	0.54	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.77	0.79	0.77
query15	0.87	0.82	0.82
query16	0.34	0.37	0.36
query17	0.97	1.03	0.96
query18	0.22	0.22	0.21
query19	1.74	1.70	1.82
query20	0.01	0.01	0.01
query21	15.41	0.76	0.67
query22	4.22	7.10	2.45
query23	18.31	1.34	1.26
query24	2.09	0.24	0.22
query25	0.15	0.09	0.08
query26	0.29	0.21	0.20
query27	0.46	0.23	0.23
query28	13.26	1.01	1.00
query29	12.60	3.30	3.32
query30	0.25	0.06	0.05
query31	2.86	0.39	0.39
query32	3.29	0.48	0.47
query33	2.94	2.90	2.92
query34	16.95	4.32	4.39
query35	4.47	4.44	4.41
query36	0.65	0.49	0.51
query37	0.18	0.16	0.16
query38	0.15	0.14	0.15
query39	0.05	0.03	0.03
query40	0.15	0.13	0.11
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.52 s
Total hot run time: 31.17 s

zzzxl1993 · 2024-07-16T11:00:15Z

run buildall

doris-robot · 2024-07-16T12:09:25Z

TPC-H: Total hot run time: 40212 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false

------ Round 1 ----------------------------------
q1	17595	4394	4381	4381
q2	2009	192	193	192
q3	10444	1269	1073	1073
q4	10190	793	813	793
q5	7591	2719	2728	2719
q6	217	135	138	135
q7	970	607	609	607
q8	9221	2099	2123	2099
q9	8746	6610	6624	6610
q10	8837	3860	3778	3778
q11	482	242	246	242
q12	429	236	236	236
q13	17765	3027	2996	2996
q14	276	231	232	231
q15	533	495	497	495
q16	509	393	393	393
q17	985	664	621	621
q18	8103	7497	7441	7441
q19	6770	1407	1355	1355
q20	702	316	317	316
q21	5088	3219	3242	3219
q22	354	280	294	280
Total cold run time: 117816 ms
Total hot run time: 40212 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4273	4248	4248
q2	388	267	277	267
q3	3046	2959	2941	2941
q4	2112	1716	1767	1716
q5	5616	5619	5518	5518
q6	220	137	139	137
q7	2309	1828	1841	1828
q8	3307	3447	3438	3438
q9	8816	8917	8834	8834
q10	4139	3829	3944	3829
q11	601	503	491	491
q12	844	650	631	631
q13	16051	3169	3213	3169
q14	337	281	288	281
q15	531	488	498	488
q16	496	434	434	434
q17	1821	1523	1488	1488
q18	8137	8029	7849	7849
q19	1793	1660	1479	1479
q20	2079	1885	1871	1871
q21	5237	4728	4885	4728
q22	597	514	542	514
Total cold run time: 72860 ms
Total hot run time: 56179 ms

doris-robot · 2024-07-16T12:20:32Z

TPC-DS: Total hot run time: 174442 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false

query1	914	388	370	370
query2	6419	1869	1875	1869
query3	6631	205	225	205
query4	28116	17566	17249	17249
query5	3767	485	480	480
query6	288	156	176	156
query7	4579	303	285	285
query8	235	185	187	185
query9	8563	2472	2427	2427
query10	454	285	272	272
query11	12433	10013	10168	10013
query12	113	86	82	82
query13	1641	365	361	361
query14	10158	7683	7634	7634
query15	216	165	166	165
query16	7719	315	321	315
query17	1753	565	545	545
query18	1966	291	280	280
query19	195	150	150	150
query20	91	80	83	80
query21	205	131	131	131
query22	4206	4204	4062	4062
query23	33879	33774	33699	33699
query24	10701	2936	2951	2936
query25	579	389	396	389
query26	706	152	160	152
query27	2329	281	284	281
query28	6086	2134	2090	2090
query29	896	648	652	648
query30	249	151	150	150
query31	976	765	755	755
query32	91	58	56	56
query33	725	323	327	323
query34	880	519	521	519
query35	683	627	588	588
query36	1135	999	981	981
query37	148	86	85	85
query38	2971	2871	2848	2848
query39	893	875	841	841
query40	202	118	115	115
query41	49	40	40	40
query42	116	98	103	98
query43	503	469	480	469
query44	1089	730	754	730
query45	192	159	160	159
query46	1082	747	736	736
query47	1943	1799	1808	1799
query48	374	301	309	301
query49	858	451	410	410
query50	791	386	389	386
query51	6863	6812	6793	6793
query52	102	90	94	90
query53	352	289	287	287
query54	906	454	451	451
query55	75	76	74	74
query56	290	265	275	265
query57	1104	1079	1048	1048
query58	255	248	252	248
query59	2828	2590	2570	2570
query60	306	269	276	269
query61	97	97	113	97
query62	783	660	632	632
query63	322	291	284	284
query64	9183	2296	1673	1673
query65	3182	3150	3135	3135
query66	748	329	331	329
query67	15523	15237	14936	14936
query68	4503	562	552	552
query69	564	403	346	346
query70	1132	1124	1155	1124
query71	372	279	278	278
query72	7029	5695	5866	5695
query73	764	331	326	326
query74	6188	5749	5786	5749
query75	3343	2676	2716	2676
query76	2122	924	952	924
query77	431	302	304	302
query78	9866	9202	9190	9190
query79	2119	535	525	525
query80	1335	486	458	458
query81	602	222	221	221
query82	667	132	136	132
query83	243	167	166	166
query84	249	90	85	85
query85	1739	310	304	304
query86	474	333	303	303
query87	3308	3137	3093	3093
query88	4035	2397	2385	2385
query89	465	386	386	386
query90	1733	190	192	190
query91	126	99	104	99
query92	63	48	49	48
query93	2196	515	518	515
query94	1051	218	212	212
query95	406	322	319	319
query96	587	283	271	271
query97	3213	3040	3078	3040
query98	211	201	195	195
query99	1533	1252	1282	1252
Total cold run time: 277376 ms
Total hot run time: 174442 ms

doris-robot · 2024-07-16T12:26:03Z

ClickBench: Total hot run time: 30.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.49	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.55	0.51	0.50
query10	0.57	0.55	0.55
query11	0.14	0.12	0.12
query12	0.14	0.12	0.12
query13	0.58	0.59	0.58
query14	0.77	0.76	0.79
query15	0.85	0.81	0.81
query16	0.36	0.36	0.36
query17	0.99	0.96	0.96
query18	0.23	0.22	0.22
query19	1.81	1.70	1.70
query20	0.01	0.01	0.01
query21	15.39	0.74	0.65
query22	4.11	7.12	1.99
query23	18.24	1.51	1.30
query24	2.08	0.26	0.21
query25	0.15	0.09	0.08
query26	0.28	0.20	0.22
query27	0.45	0.24	0.22
query28	13.26	1.03	1.00
query29	12.57	3.28	3.29
query30	0.25	0.06	0.07
query31	2.84	0.40	0.39
query32	3.26	0.48	0.47
query33	2.88	2.86	2.94
query34	17.19	4.46	4.41
query35	4.45	4.42	4.42
query36	0.66	0.49	0.47
query37	0.20	0.15	0.16
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.15	0.12	0.11
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.03
Total cold run time: 109.54 s
Total hot run time: 30.71 s

zzzxl1993 · 2024-07-19T12:50:51Z

run buildall

doris-robot · 2024-07-19T14:06:41Z

TPC-H: Total hot run time: 39945 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false

------ Round 1 ----------------------------------
q1	17611	4625	4336	4336
q2	2020	190	183	183
q3	10448	1272	1078	1078
q4	10197	778	689	689
q5	7561	2686	2687	2686
q6	223	136	138	136
q7	958	602	598	598
q8	9223	2089	2101	2089
q9	8583	6571	6544	6544
q10	8827	3752	3819	3752
q11	478	234	232	232
q12	403	224	224	224
q13	18844	2968	2957	2957
q14	275	229	228	228
q15	540	482	504	482
q16	506	385	371	371
q17	973	655	722	655
q18	8192	7529	7439	7439
q19	6125	1425	1411	1411
q20	661	313	323	313
q21	5246	3326	3269	3269
q22	339	273	288	273
Total cold run time: 118233 ms
Total hot run time: 39945 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4425	4250	4300	4250
q2	383	267	258	258
q3	3003	2883	2933	2883
q4	1991	1696	1747	1696
q5	5627	5510	5484	5484
q6	220	137	136	136
q7	2186	1876	1840	1840
q8	3277	3399	3456	3399
q9	8786	8812	8842	8812
q10	4058	3813	3750	3750
q11	596	495	500	495
q12	832	626	632	626
q13	15972	3189	3157	3157
q14	338	279	293	279
q15	521	492	481	481
q16	487	451	435	435
q17	1851	1524	1523	1523
q18	8139	8018	7900	7900
q19	1805	1691	1505	1505
q20	2822	1873	1844	1844
q21	9319	5061	4703	4703
q22	583	536	479	479
Total cold run time: 77221 ms
Total hot run time: 55935 ms

doris-robot · 2024-07-19T14:17:51Z

TPC-DS: Total hot run time: 172760 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false

query1	917	395	376	376
query2	6445	1857	1753	1753
query3	6631	211	217	211
query4	28153	17427	17282	17282
query5	3675	466	475	466
query6	254	179	159	159
query7	4548	283	290	283
query8	237	195	189	189
query9	8441	2404	2380	2380
query10	418	271	282	271
query11	11199	10043	10083	10043
query12	114	83	85	83
query13	1589	357	363	357
query14	10110	7431	7654	7431
query15	211	162	160	160
query16	7592	475	450	450
query17	1117	558	517	517
query18	1951	280	274	274
query19	186	147	146	146
query20	86	83	76	76
query21	204	130	127	127
query22	4310	4146	3991	3991
query23	34003	33777	33578	33578
query24	10968	2984	2917	2917
query25	575	393	397	393
query26	717	146	151	146
query27	2289	280	277	277
query28	6041	2049	2049	2049
query29	888	619	632	619
query30	258	148	149	148
query31	1016	749	754	749
query32	104	54	59	54
query33	662	368	330	330
query34	911	494	501	494
query35	884	742	747	742
query36	1147	992	973	973
query37	140	77	89	77
query38	3018	2872	2821	2821
query39	935	837	879	837
query40	214	118	122	118
query41	47	44	42	42
query42	113	99	97	97
query43	490	448	449	448
query44	1133	726	718	718
query45	194	165	160	160
query46	1093	713	743	713
query47	1868	1771	1784	1771
query48	359	295	298	295
query49	844	417	422	417
query50	789	396	403	396
query51	6927	6766	6787	6766
query52	112	88	97	88
query53	351	286	287	286
query54	881	453	456	453
query55	77	76	73	73
query56	307	280	292	280
query57	1124	1049	1054	1049
query58	256	265	255	255
query59	2835	2679	2484	2484
query60	313	319	292	292
query61	113	116	113	113
query62	815	638	667	638
query63	312	292	285	285
query64	9621	2325	1656	1656
query65	3153	3119	3139	3119
query66	755	336	321	321
query67	15359	14900	15103	14900
query68	4561	533	542	533
query69	615	392	352	352
query70	1208	1120	1124	1120
query71	439	283	275	275
query72	8791	5080	5859	5080
query73	756	323	346	323
query74	6073	5697	5642	5642
query75	4024	2700	2664	2664
query76	3013	972	816	816
query77	690	316	306	306
query78	9725	9166	8945	8945
query79	2910	538	515	515
query80	1399	471	479	471
query81	600	219	222	219
query82	947	139	131	131
query83	359	164	166	164
query84	269	91	86	86
query85	1780	311	299	299
query86	459	332	325	325
query87	3251	3107	3097	3097
query88	4208	2367	2384	2367
query89	488	382	382	382
query90	1874	187	196	187
query91	129	98	97	97
query92	60	51	49	49
query93	3747	498	501	498
query94	1259	284	287	284
query95	414	385	320	320
query96	605	272	285	272
query97	3176	3033	3020	3020
query98	226	203	200	200
query99	1479	1259	1261	1259
Total cold run time: 282933 ms
Total hot run time: 172760 ms

doris-robot · 2024-07-19T14:23:22Z

ClickBench: Total hot run time: 30.9 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.54	0.48	0.49
query6	1.13	0.73	0.72
query7	0.01	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.53	0.53	0.55
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.60	0.60	0.58
query14	0.76	0.78	0.77
query15	0.85	0.80	0.82
query16	0.36	0.37	0.35
query17	1.03	0.99	0.99
query18	0.24	0.23	0.22
query19	1.93	1.81	1.69
query20	0.01	0.00	0.01
query21	15.39	0.79	0.65
query22	4.04	6.53	2.18
query23	18.29	1.37	1.35
query24	2.07	0.23	0.24
query25	0.16	0.09	0.08
query26	0.31	0.21	0.21
query27	0.45	0.24	0.23
query28	13.26	1.02	1.00
query29	12.63	3.41	3.36
query30	0.25	0.06	0.06
query31	2.88	0.39	0.38
query32	3.26	0.48	0.46
query33	2.91	2.92	2.86
query34	17.08	4.36	4.40
query35	4.39	4.39	4.46
query36	0.64	0.46	0.46
query37	0.18	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.06
query42	0.05	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.63 s
Total hot run time: 30.9 s

zzzxl1993 · 2024-07-19T14:50:16Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2024-07-19T14:57:31Z

be/src/vec/functions/function_multi_match.cpp

+ return Status::OK();
+}
+
+Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,


warning: function 'eval_inverted_index' exceeds recommended size/complexity thresholds [readability-function-size]

Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context, ^

Additional context

be/src/vec/functions/function_multi_match.cpp:94: 86 lines including whitespace and comments (threshold 80)

Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context, ^

doris-robot · 2024-07-19T16:06:48Z

TPC-H: Total hot run time: 39971 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false

------ Round 1 ----------------------------------
q1	17625	4439	4312	4312
q2	2010	191	184	184
q3	10472	1249	1168	1168
q4	10184	777	753	753
q5	7561	2713	2708	2708
q6	227	139	134	134
q7	962	595	587	587
q8	9218	2072	2118	2072
q9	8698	6556	6599	6556
q10	8846	3768	3795	3768
q11	450	235	233	233
q12	434	221	215	215
q13	17777	2985	2969	2969
q14	280	235	238	235
q15	525	490	499	490
q16	499	389	376	376
q17	969	709	737	709
q18	8233	7504	7430	7430
q19	6819	1458	1345	1345
q20	684	323	332	323
q21	4903	3121	3992	3121
q22	344	290	283	283
Total cold run time: 117720 ms
Total hot run time: 39971 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4406	4230	4261	4230
q2	373	274	273	273
q3	3002	2944	2913	2913
q4	1952	1677	1708	1677
q5	5659	5562	5486	5486
q6	222	128	129	128
q7	2188	1862	1854	1854
q8	3280	3426	3489	3426
q9	8790	8865	8850	8850
q10	4076	3807	3838	3807
q11	596	521	512	512
q12	808	622	612	612
q13	17146	3124	3172	3124
q14	325	298	279	279
q15	545	504	483	483
q16	498	443	431	431
q17	1823	1552	1529	1529
q18	8321	8015	7871	7871
q19	1861	1743	1619	1619
q20	2096	1903	1860	1860
q21	9202	4707	4748	4707
q22	581	511	495	495
Total cold run time: 77750 ms
Total hot run time: 56166 ms

doris-robot · 2024-07-19T16:18:06Z

TPC-DS: Total hot run time: 173229 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false

query1	900	373	371	371
query2	6427	1941	1841	1841
query3	6655	214	224	214
query4	28361	17275	17552	17275
query5	3704	467	475	467
query6	276	169	166	166
query7	4582	288	285	285
query8	244	202	192	192
query9	8554	2497	2447	2447
query10	430	295	280	280
query11	11141	9984	10092	9984
query12	113	88	84	84
query13	1640	372	371	371
query14	10325	7211	7658	7211
query15	213	167	170	167
query16	7677	493	468	468
query17	1571	572	541	541
query18	1928	285	278	278
query19	202	155	157	155
query20	93	87	85	85
query21	207	126	127	126
query22	4208	4090	4035	4035
query23	34141	33841	33508	33508
query24	11085	2972	2909	2909
query25	640	415	411	411
query26	1157	155	158	155
query27	2866	285	294	285
query28	7413	2110	2102	2102
query29	1002	659	638	638
query30	264	159	155	155
query31	1004	744	724	724
query32	103	52	55	52
query33	780	345	330	330
query34	976	491	501	491
query35	868	759	726	726
query36	1109	983	959	959
query37	156	82	81	81
query38	2969	2898	2838	2838
query39	912	849	845	845
query40	199	120	114	114
query41	47	43	43	43
query42	118	99	101	99
query43	515	470	468	468
query44	1185	744	727	727
query45	192	159	162	159
query46	1089	745	746	745
query47	1847	1740	1758	1740
query48	361	294	287	287
query49	836	411	434	411
query50	776	402	392	392
query51	6900	6794	6770	6770
query52	100	94	91	91
query53	352	290	298	290
query54	853	451	448	448
query55	77	73	75	73
query56	308	263	266	263
query57	1128	1059	1038	1038
query58	242	250	261	250
query59	2944	2570	2537	2537
query60	306	271	273	271
query61	93	98	92	92
query62	802	661	653	653
query63	313	296	288	288
query64	9460	2224	1662	1662
query65	3157	3078	3079	3078
query66	770	315	327	315
query67	15391	14850	14835	14835
query68	5340	552	544	544
query69	718	454	359	359
query70	1217	1112	1146	1112
query71	486	280	279	279
query72	9234	5700	5525	5525
query73	784	324	326	324
query74	6083	5670	5623	5623
query75	4300	2690	2641	2641
query76	4007	989	908	908
query77	674	319	311	311
query78	9727	8999	10368	8999
query79	3763	533	530	530
query80	2294	480	482	480
query81	604	225	219	219
query82	1322	138	132	132
query83	309	163	164	163
query84	278	93	87	87
query85	1579	315	301	301
query86	462	303	312	303
query87	3281	3128	3116	3116
query88	4070	2395	2428	2395
query89	469	372	382	372
query90	1913	194	192	192
query91	135	100	99	99
query92	62	49	50	49
query93	2807	503	517	503
query94	1300	291	266	266
query95	404	308	310	308
query96	602	271	278	271
query97	3252	2994	3019	2994
query98	217	199	191	191
query99	1496	1288	1242	1242
Total cold run time: 290446 ms
Total hot run time: 173229 ms

doris-robot · 2024-07-19T16:23:38Z

ClickBench: Total hot run time: 30.54 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.08	0.09
query5	0.49	0.50	0.50
query6	1.12	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.50
query10	0.53	0.55	0.55
query11	0.16	0.11	0.11
query12	0.15	0.13	0.12
query13	0.61	0.58	0.58
query14	0.76	0.77	0.78
query15	0.86	0.82	0.82
query16	0.36	0.37	0.35
query17	1.03	1.04	1.00
query18	0.22	0.22	0.21
query19	1.92	1.74	1.84
query20	0.01	0.00	0.01
query21	15.42	0.80	0.66
query22	3.78	7.85	1.96
query23	18.25	1.33	1.24
query24	2.13	0.23	0.24
query25	0.16	0.09	0.09
query26	0.29	0.21	0.20
query27	0.46	0.23	0.22
query28	13.20	1.02	1.00
query29	12.61	3.34	3.26
query30	0.25	0.06	0.05
query31	2.88	0.39	0.38
query32	3.27	0.49	0.48
query33	2.91	2.83	2.94
query34	16.90	4.30	4.34
query35	4.41	4.44	4.39
query36	0.66	0.48	0.48
query37	0.19	0.15	0.16
query38	0.16	0.14	0.14
query39	0.05	0.03	0.04
query40	0.16	0.13	0.12
query41	0.09	0.04	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.18 s
Total hot run time: 30.54 s

zzzxl1993 · 2024-07-22T05:10:17Z

run buildall

doris-robot · 2024-07-22T06:08:37Z

TPC-H: Total hot run time: 40355 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false

------ Round 1 ----------------------------------
q1	18118	4713	4472	4472
q2	2008	194	199	194
q3	10443	1284	1053	1053
q4	10193	837	796	796
q5	7558	2700	2778	2700
q6	216	139	140	139
q7	958	602	625	602
q8	9227	2126	2087	2087
q9	8750	6568	6615	6568
q10	8737	3858	3825	3825
q11	466	244	251	244
q12	395	233	236	233
q13	17753	2987	2989	2987
q14	287	242	250	242
q15	533	486	476	476
q16	484	384	386	384
q17	992	612	703	612
q18	8232	7514	7546	7514
q19	7018	1407	1513	1407
q20	682	323	332	323
q21	5046	3207	3344	3207
q22	352	291	290	290
Total cold run time: 118448 ms
Total hot run time: 40355 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4401	4255	4364	4255
q2	383	272	276	272
q3	3033	2764	2790	2764
q4	1904	1670	1614	1614
q5	5291	5358	5356	5356
q6	215	133	132	132
q7	2141	1702	1749	1702
q8	3208	3369	3328	3328
q9	8505	8395	8423	8395
q10	3931	3684	3665	3665
q11	573	467	486	467
q12	781	635	584	584
q13	16839	2983	3037	2983
q14	299	272	283	272
q15	518	473	485	473
q16	484	438	422	422
q17	1792	1490	1496	1490
q18	7706	7592	7455	7455
q19	1706	1588	1524	1524
q20	2013	1786	1782	1782
q21	4799	4624	4690	4624
q22	573	506	504	504
Total cold run time: 71095 ms
Total hot run time: 54063 ms

doris-robot · 2024-07-22T06:19:52Z

TPC-DS: Total hot run time: 174180 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false

query1	920	380	378	378
query2	6451	1918	1894	1894
query3	6653	203	219	203
query4	23729	17685	17391	17391
query5	4204	495	506	495
query6	276	202	194	194
query7	4601	296	290	290
query8	249	199	202	199
query9	8369	2450	2419	2419
query10	443	275	272	272
query11	10634	10201	9994	9994
query12	131	84	86	84
query13	1631	366	385	366
query14	9062	7959	7492	7492
query15	209	170	165	165
query16	7584	495	517	495
query17	1321	532	528	528
query18	1768	277	283	277
query19	205	147	150	147
query20	94	84	84	84
query21	212	138	132	132
query22	4276	4147	4073	4073
query23	33838	34665	33372	33372
query24	12142	2972	2848	2848
query25	529	370	369	369
query26	1235	151	151	151
query27	2977	270	276	270
query28	7376	2016	2021	2016
query29	875	625	631	625
query30	283	149	147	147
query31	941	744	739	739
query32	96	53	55	53
query33	793	388	337	337
query34	900	490	498	490
query35	858	734	731	731
query36	1110	919	909	909
query37	218	80	80	80
query38	2840	2745	2753	2745
query39	894	814	825	814
query40	281	119	124	119
query41	47	46	44	44
query42	117	100	100	100
query43	507	476	475	475
query44	1210	735	738	735
query45	199	165	164	164
query46	1094	747	730	730
query47	1869	1788	1790	1788
query48	378	297	286	286
query49	1168	405	434	405
query50	783	413	411	411
query51	6899	6698	6735	6698
query52	110	90	97	90
query53	359	287	294	287
query54	898	459	454	454
query55	75	77	78	77
query56	292	269	266	266
query57	1141	1047	1048	1047
query58	256	249	249	249
query59	2962	2574	2699	2574
query60	298	278	289	278
query61	104	100	96	96
query62	840	642	650	642
query63	354	300	293	293
query64	10423	2197	1687	1687
query65	3165	3342	3168	3168
query66	1269	363	325	325
query67	15386	14936	15018	14936
query68	4562	533	552	533
query69	468	359	354	354
query70	1137	1147	1089	1089
query71	448	283	288	283
query72	7139	5675	5731	5675
query73	757	337	333	333
query74	6176	5670	5686	5670
query75	3381	2698	2671	2671
query76	2819	1037	942	942
query77	461	326	332	326
query78	12375	10558	9050	9050
query79	6576	532	545	532
query80	1223	526	514	514
query81	582	225	238	225
query82	452	146	137	137
query83	275	180	173	173
query84	283	90	96	90
query85	778	380	350	350
query86	498	312	325	312
query87	3303	3141	3139	3139
query88	4092	2451	2460	2451
query89	472	387	393	387
query90	1940	211	201	201
query91	140	116	114	114
query92	69	55	52	52
query93	2707	514	507	507
query94	1132	309	313	309
query95	436	338	339	338
query96	613	417	279	279
query97	3241	3028	3051	3028
query98	226	197	196	196
query99	1727	1315	1228	1228
Total cold run time: 283899 ms
Total hot run time: 174180 ms

doris-robot · 2024-07-22T06:25:24Z

ClickBench: Total hot run time: 31.13 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.48
query6	1.12	0.74	0.74
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.54	0.48	0.49
query10	0.54	0.56	0.53
query11	0.16	0.12	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.59
query14	0.76	0.78	0.79
query15	0.85	0.82	0.82
query16	0.37	0.36	0.37
query17	1.03	1.04	0.97
query18	0.24	0.23	0.23
query19	1.92	1.74	1.83
query20	0.01	0.01	0.01
query21	15.42	0.77	0.66
query22	4.50	6.90	2.21
query23	18.33	1.43	1.27
query24	2.11	0.23	0.23
query25	0.17	0.09	0.08
query26	0.29	0.22	0.21
query27	0.45	0.23	0.24
query28	13.32	1.02	0.99
query29	12.62	3.33	3.30
query30	0.25	0.06	0.06
query31	2.89	0.38	0.38
query32	3.30	0.49	0.48
query33	2.95	2.89	2.93
query34	16.76	4.43	4.42
query35	4.48	4.41	4.47
query36	0.65	0.49	0.49
query37	0.19	0.16	0.15
query38	0.16	0.14	0.15
query39	0.04	0.03	0.04
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.12 s
Total hot run time: 31.13 s

airborne12

LGTM

github-actions · 2024-07-22T09:07:59Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-22T09:08:01Z

PR approved by anyone and no changes requested.

1. select * from tbl where multi_match(c1, 'c2, c3, c4', 'phrase_prefix', 'xxx'); 2. multi_match performs a match_phrase_prefix query on the columns specified in the first and second parameters. The value to be searched is specified in the fourth parameter.

…opn (#38870) pick #37722

1. select * from tbl where multi_match(c1, 'c2, c3, c4', 'phrase_prefix', 'xxx'); 2. multi_match performs a match_phrase_prefix query on the columns specified in the first and second parameters. The value to be searched is specified in the fourth parameter.

## Proposed changes 1. select * from tbl where multi_match(c1, 'c2, c3, c4', 'phrase_prefix', 'xxx'); 2. multi_match performs a match_phrase_prefix query on the columns specified in the first and second parameters. The value to be searched is specified in the fourth parameter.

1. select * from tbl where multi_match(c1, 'c2, c3, c4', 'phrase_prefix', 'xxx'); 2. multi_match performs a match_phrase_prefix query on the columns specified in the first and second parameters. The value to be searched is specified in the fourth parameter.

… (#38877)

airborne12 previously approved these changes Jul 13, 2024

View reviewed changes

zzzxl1993 dismissed airborne12’s stale review via 1a438ae July 15, 2024 15:24

zzzxl1993 force-pushed the multi_match branch from d5e25c6 to 1a438ae Compare July 15, 2024 15:24

github-actions bot reviewed Jul 15, 2024

View reviewed changes

zzzxl1993 force-pushed the multi_match branch from 1a438ae to 718b166 Compare July 16, 2024 02:31

zzzxl1993 force-pushed the multi_match branch from 718b166 to ed9f6e7 Compare July 16, 2024 11:00

zzzxl1993 force-pushed the multi_match branch 3 times, most recently from 7087e1b to b45cfd3 Compare July 19, 2024 12:50

zzzxl1993 force-pushed the multi_match branch from b45cfd3 to a926b4c Compare July 19, 2024 14:50

github-actions bot reviewed Jul 19, 2024

View reviewed changes

multi match function add

fee667f

zzzxl1993 force-pushed the multi_match branch from a926b4c to fee667f Compare July 22, 2024 05:10

airborne12 approved these changes Jul 22, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024

github-actions bot added the reviewed label Jul 22, 2024

qidaye approved these changes Jul 22, 2024

View reviewed changes

qidaye merged commit 47dfd41 into apache:master Jul 22, 2024
26 of 29 checks passed

xiaokang added usercase Important user case type label dev/2.1.x labels Jul 31, 2024

zzzxl1993 mentioned this pull request Aug 5, 2024

[opt](inverted index) Optimization of the initialization process in topn #38870

Merged

zzzxl1993 mentioned this pull request Aug 5, 2024

[feature](inverted index) Add multi_match function (#37722) #38877

Merged

yiguolei pushed a commit that referenced this pull request Aug 5, 2024

[opt](inverted index) Optimization of the initialization process in t…

9d5af7f

…opn (#38870) pick #37722

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Aug 5, 2024

dataroaring added the dev/3.0.1-merged label Aug 6, 2024

xiaokang pushed a commit that referenced this pull request Aug 10, 2024

[feature](inverted index) Add multi_match function #37722 #38931 #39149…

5e1e725

… (#38877)

gavinchou mentioned this pull request Aug 19, 2024

Release Note 3.0.1 #39570

Open

yiguolei mentioned this pull request Sep 5, 2024

Release Note 2.1.6 #40443

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature](inverted index) multi_match function add #37722

[feature](inverted index) multi_match function add #37722

zzzxl1993 commented Jul 12, 2024 •

edited

Loading

doris-robot commented Jul 12, 2024

airborne12 left a comment

zzzxl1993 commented Jul 15, 2024

github-actions bot left a comment

github-actions bot Jul 15, 2024

github-actions bot Jul 15, 2024

zzzxl1993 commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

zzzxl1993 commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

zzzxl1993 commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

zzzxl1993 commented Jul 19, 2024

github-actions bot left a comment

github-actions bot Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

zzzxl1993 commented Jul 22, 2024

doris-robot commented Jul 22, 2024

doris-robot commented Jul 22, 2024

doris-robot commented Jul 22, 2024

airborne12 left a comment

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

[feature](inverted index) multi_match function add #37722

[feature](inverted index) multi_match function add #37722

Conversation

zzzxl1993 commented Jul 12, 2024 • edited Loading

Proposed changes

doris-robot commented Jul 12, 2024

airborne12 left a comment

Choose a reason for hiding this comment

zzzxl1993 commented Jul 15, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Jul 15, 2024

Choose a reason for hiding this comment

github-actions bot Jul 15, 2024

Choose a reason for hiding this comment

zzzxl1993 commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

zzzxl1993 commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

doris-robot commented Jul 16, 2024

zzzxl1993 commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

zzzxl1993 commented Jul 19, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Jul 19, 2024

Choose a reason for hiding this comment

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

doris-robot commented Jul 19, 2024

zzzxl1993 commented Jul 22, 2024

doris-robot commented Jul 22, 2024

doris-robot commented Jul 22, 2024

doris-robot commented Jul 22, 2024

airborne12 left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

zzzxl1993 commented Jul 12, 2024 •

edited

Loading