Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](inverted index) multi match distinguishes the inverted index v1 and v2 #39149

Merged
merged 1 commit into from
Aug 9, 2024
Merged

[fix](inverted index) multi match distinguishes the inverted index v1 and v2 #39149

merged 1 commit into from
Aug 9, 2024

Conversation

zzzxl1993
Copy link
Contributor

Proposed changes

  1. reversed Index v1 and v2 have different column names

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the doing label Aug 9, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39645 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6892bc8e80ff270c8df766cc52905432c7ec402, data reload: false

------ Round 1 ----------------------------------
q1	17614	4382	4341	4341
q2	2016	180	176	176
q3	10524	1193	1034	1034
q4	10148	761	691	691
q5	7474	2508	2495	2495
q6	232	144	142	142
q7	969	594	586	586
q8	9223	1921	1926	1921
q9	8549	6559	6500	6500
q10	7046	2184	2209	2184
q11	463	239	242	239
q12	521	224	226	224
q13	18547	2971	2959	2959
q14	274	233	234	233
q15	521	484	499	484
q16	502	388	391	388
q17	960	668	729	668
q18	8008	7454	7490	7454
q19	4487	1079	1027	1027
q20	666	323	326	323
q21	5440	4559	4588	4559
q22	1118	1041	1017	1017
Total cold run time: 115302 ms
Total hot run time: 39645 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4455	4288	4257	4257
q2	373	259	257	257
q3	2839	2625	2709	2625
q4	1967	1770	1729	1729
q5	5505	5493	5447	5447
q6	223	137	134	134
q7	2142	1762	1820	1762
q8	3284	3409	3444	3409
q9	8733	8666	8859	8666
q10	3501	3312	3274	3274
q11	572	489	500	489
q12	773	621	638	621
q13	15961	3214	3143	3143
q14	328	280	292	280
q15	542	493	486	486
q16	490	452	446	446
q17	1839	1538	1505	1505
q18	8057	8159	7777	7777
q19	1753	1677	1626	1626
q20	2379	1896	1881	1881
q21	9812	5311	5386	5311
q22	1123	1040	1033	1033
Total cold run time: 76651 ms
Total hot run time: 56158 ms

@zzzxl1993
Copy link
Contributor Author

run buildall

airborne12
airborne12 approved these changes Aug 9, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 9, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 9, 2024

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39790 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 24866fed18f90d8d332b60081a1e57421d49412f, data reload: false

------ Round 1 ----------------------------------
q1	17991	4422	4418	4418
q2	2924	181	176	176
q3	11141	1178	1119	1119
q4	10714	781	783	781
q5	8007	2530	2561	2530
q6	226	147	142	142
q7	1002	599	589	589
q8	9227	1921	1893	1893
q9	8692	6576	7014	6576
q10	7049	2290	2199	2199
q11	456	245	248	245
q12	404	223	222	222
q13	17754	2974	3008	2974
q14	276	234	240	234
q15	531	481	482	481
q16	496	404	386	386
q17	966	710	671	671
q18	8099	7570	7474	7474
q19	4251	1019	960	960
q20	664	332	328	328
q21	5337	4390	4421	4390
q22	1116	1002	1035	1002
Total cold run time: 117323 ms
Total hot run time: 39790 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4410	4253	4256	4253
q2	386	287	271	271
q3	2885	2625	2616	2616
q4	1932	1650	1652	1650
q5	5243	5248	5248	5248
q6	221	131	129	129
q7	2063	1687	1646	1646
q8	3146	3341	3341	3341
q9	8424	8352	8385	8352
q10	3393	3168	3141	3141
q11	596	498	502	498
q12	778	605	634	605
q13	16576	3022	2968	2968
q14	303	279	281	279
q15	521	475	474	474
q16	476	419	427	419
q17	1764	1481	1516	1481
q18	7734	7575	7420	7420
q19	1736	1382	1553	1382
q20	2021	1817	1766	1766
q21	5244	5059	5086	5059
q22	1102	991	997	991
Total cold run time: 70954 ms
Total hot run time: 53989 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201296 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 24866fed18f90d8d332b60081a1e57421d49412f, data reload: false

query1	920	373	361	361
query2	6455	1845	1742	1742
query3	6646	209	214	209
query4	34348	23297	23230	23230
query5	4229	500	498	498
query6	267	175	171	171
query7	4591	300	306	300
query8	247	200	198	198
query9	8657	2471	2454	2454
query10	530	471	462	462
query11	16677	15006	14935	14935
query12	153	103	100	100
query13	1655	376	361	361
query14	10217	7658	7679	7658
query15	262	215	235	215
query16	7654	511	463	463
query17	1714	554	543	543
query18	1879	269	274	269
query19	187	134	158	134
query20	118	108	108	108
query21	200	101	102	101
query22	4431	4121	4077	4077
query23	34192	33225	35342	33225
query24	11996	2577	2539	2539
query25	503	369	375	369
query26	1110	155	156	155
query27	2878	286	285	285
query28	7622	2040	2023	2023
query29	682	402	407	402
query30	305	150	146	146
query31	982	758	764	758
query32	94	52	53	52
query33	728	278	284	278
query34	955	453	471	453
query35	940	778	769	769
query36	1078	938	923	923
query37	218	79	77	77
query38	4347	4227	4172	4172
query39	1423	1372	1359	1359
query40	280	114	113	113
query41	49	45	44	44
query42	115	99	100	99
query43	506	450	457	450
query44	1206	734	724	724
query45	236	208	217	208
query46	1089	732	738	732
query47	1869	1781	1787	1781
query48	376	302	294	294
query49	1201	422	423	422
query50	791	404	407	404
query51	6846	6806	6635	6635
query52	104	87	94	87
query53	253	187	186	186
query54	862	450	451	450
query55	77	73	74	73
query56	274	249	250	249
query57	1142	1071	1060	1060
query58	240	228	230	228
query59	2819	2832	2857	2832
query60	305	259	262	259
query61	100	99	97	97
query62	812	639	657	639
query63	225	185	178	178
query64	10407	2259	1768	1768
query65	3211	3187	3125	3125
query66	1231	342	358	342
query67	15430	14806	14668	14668
query68	4613	556	572	556
query69	422	386	394	386
query70	1102	1063	1105	1063
query71	416	278	268	268
query72	17809	16812	16695	16695
query73	761	325	324	324
query74	9128	8790	8794	8790
query75	3360	2722	2664	2664
query76	2802	938	982	938
query77	461	315	326	315
query78	11443	9411	8967	8967
query79	2970	523	515	515
query80	1065	483	494	483
query81	586	226	223	223
query82	643	134	139	134
query83	263	159	148	148
query84	273	80	76	76
query85	1202	279	280	279
query86	498	314	296	296
query87	4712	4600	4498	4498
query88	4323	2562	2494	2494
query89	417	288	290	288
query90	1907	203	194	194
query91	122	94	96	94
query92	69	53	50	50
query93	4052	549	534	534
query94	821	288	302	288
query95	354	263	261	261
query96	608	273	271	271
query97	3228	3032	3023	3023
query98	221	201	200	200
query99	1601	1288	1260	1260
Total cold run time: 316398 ms
Total hot run time: 201296 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 24866fed18f90d8d332b60081a1e57421d49412f, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.47	0.48
query6	1.13	0.73	0.74
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.50	0.49
query10	0.55	0.54	0.54
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.60	0.59
query14	0.75	0.78	0.78
query15	0.88	0.82	0.82
query16	0.37	0.36	0.37
query17	1.04	1.05	1.05
query18	0.24	0.23	0.22
query19	1.84	1.71	1.69
query20	0.01	0.02	0.01
query21	15.41	0.77	0.67
query22	4.19	6.70	2.51
query23	18.29	1.38	1.22
query24	2.07	0.22	0.22
query25	0.15	0.09	0.08
query26	0.31	0.21	0.22
query27	0.46	0.23	0.23
query28	13.33	1.02	1.02
query29	12.58	3.32	3.30
query30	0.24	0.05	0.05
query31	2.89	0.40	0.38
query32	3.28	0.49	0.47
query33	2.89	2.88	2.91
query34	17.04	4.31	4.36
query35	4.41	4.42	4.51
query36	0.65	0.46	0.49
query37	0.19	0.15	0.16
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.04	0.04
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.79 s
Total hot run time: 31.25 s

qidaye
qidaye approved these changes Aug 9, 2024
View reviewed changes
Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qidaye qidaye merged commit 34f7c66 into apache:master Aug 9, 2024
27 of 30 checks passed
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 9, 2024
@zzzxl1993
[fix](inverted index) multi match distinguishes the inverted index v1…
87e7494 
… and v2 (apache#39149)

## Proposed changes

1. reversed Index v1 and v2 have different column names
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 9, 2024
@zzzxl1993
[fix](inverted index) multi match distinguishes the inverted index v1…
ff3378d 
… and v2 (apache#39149)

## Proposed changes

1. reversed Index v1 and v2 have different column names
@zzzxl1993 zzzxl1993 mentioned this pull request Aug 9, 2024
Merged
xiaokang pushed a commit that referenced this pull request Aug 10, 2024
@zzzxl1993
[feature](inverted index) Add multi_match function #37722 #38931 #39149…
5e1e725 
… (#38877)
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@zzzxl1993 @dataroaring
[fix](inverted index) multi match distinguishes the inverted index v1…
eb87fe0 
… and v2 (#39149)

## Proposed changes

1. reversed Index v1 and v2 have different column names
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
@zzzxl1993 @wyxxxcat
[fix](inverted index) multi match distinguishes the inverted index v1…
a653945 
… and v2 (apache#39149)

## Proposed changes

1. reversed Index v1 and v2 have different column names
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@zzzxl1993 @dataroaring
[fix](inverted index) multi match distinguishes the inverted index v1…
6677024 
… and v2 (#39149)

## Proposed changes

1. reversed Index v1 and v2 have different column names
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@airborne12 airborne12 airborne12 approved these changes

@qidaye qidaye qidaye approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zzzxl1993 @doris-robot @airborne12 @qidaye @xiaokang @dataroaring