Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](routine load) fix show routine load task result incorrect #38523

Merged
merged 1 commit into from
Aug 1, 2024
Merged

[fix](routine load) fix show routine load task result incorrect #38523

merged 1 commit into from
Aug 1, 2024

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Jul 30, 2024

Bug report:

Create a job:

CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);

show routine load task:

SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";

result:

ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled

Solution

Do not use toLowerCase method;

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@sollhui
Copy link
Contributor Author

sollhui commented Jul 30, 2024

run buildall

@sollhui
Copy link
Contributor Author

sollhui commented Jul 30, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 42246 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e168993ad838f71491ca9cb8912f94692d6e73a0, data reload: false

------ Round 1 ----------------------------------
q1	17855	4127	4072	4072
q2	2022	200	198	198
q3	10862	1320	1376	1320
q4	11322	833	958	833
q5	7650	3033	2987	2987
q6	222	141	145	141
q7	1052	615	613	613
q8	9435	1880	1910	1880
q9	8532	6646	6630	6630
q10	8799	3864	3853	3853
q11	429	250	251	250
q12	431	236	228	228
q13	17777	3106	3097	3097
q14	278	261	251	251
q15	520	489	499	489
q16	534	430	420	420
q17	962	907	968	907
q18	9799	7875	7601	7601
q19	3737	1080	1225	1080
q20	547	332	339	332
q21	5317	4783	4807	4783
q22	349	281	286	281
Total cold run time: 118431 ms
Total hot run time: 42246 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4097	4051	4026	4026
q2	344	230	227	227
q3	3130	3179	3144	3144
q4	2031	2007	2001	2001
q5	5471	5411	5458	5411
q6	223	133	136	133
q7	2172	1806	1800	1800
q8	3311	3365	3351	3351
q9	8640	8750	8675	8675
q10	3980	3940	4030	3940
q11	560	464	464	464
q12	764	546	654	546
q13	16311	3121	3094	3094
q14	309	263	273	263
q15	531	492	482	482
q16	464	424	421	421
q17	1761	1712	1726	1712
q18	8171	7816	7674	7674
q19	1911	1722	1707	1707
q20	2055	1834	1827	1827
q21	5922	5389	5438	5389
q22	555	469	482	469
Total cold run time: 72713 ms
Total hot run time: 56756 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170174 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e168993ad838f71491ca9cb8912f94692d6e73a0, data reload: false

query1	906	372	364	364
query2	6452	1763	1804	1763
query3	6647	212	229	212
query4	19541	17381	17318	17318
query5	3668	498	497	497
query6	279	179	163	163
query7	4609	304	291	291
query8	257	202	193	193
query9	8523	2370	2364	2364
query10	426	270	270	270
query11	10502	9959	9953	9953
query12	129	89	94	89
query13	1638	376	371	371
query14	9780	7702	7888	7702
query15	227	166	164	164
query16	6996	461	440	440
query17	976	592	559	559
query18	1922	300	290	290
query19	197	151	149	149
query20	94	85	90	85
query21	213	105	105	105
query22	4291	3937	3981	3937
query23	34099	33567	33140	33140
query24	10368	3181	3097	3097
query25	672	394	397	394
query26	1768	150	150	150
query27	2844	283	288	283
query28	7340	2004	1960	1960
query29	1277	432	439	432
query30	231	164	158	158
query31	933	794	762	762
query32	98	54	54	54
query33	679	311	311	311
query34	903	506	501	501
query35	867	774	785	774
query36	1049	914	884	884
query37	305	92	84	84
query38	3011	2817	2819	2817
query39	911	828	808	808
query40	256	109	113	109
query41	46	42	44	42
query42	118	101	104	101
query43	492	441	446	441
query44	1174	723	729	723
query45	213	176	175	175
query46	1084	787	766	766
query47	1814	1711	1718	1711
query48	360	289	314	289
query49	939	412	408	408
query50	908	426	430	426
query51	6784	6894	6681	6681
query52	97	87	92	87
query53	247	181	179	179
query54	650	446	447	446
query55	77	73	75	73
query56	269	262	254	254
query57	1126	1041	1029	1029
query58	265	286	282	282
query59	2616	2613	2488	2488
query60	286	287	263	263
query61	98	95	123	95
query62	893	675	670	670
query63	216	175	180	175
query64	5579	1899	1890	1890
query65	3172	3091	3075	3075
query66	1303	332	336	332
query67	15518	15005	14811	14811
query68	4608	548	562	548
query69	557	322	284	284
query70	1102	1064	1105	1064
query71	464	276	274	274
query72	7022	2736	2561	2561
query73	780	326	325	325
query74	6002	5690	5636	5636
query75	3339	2737	2732	2732
query76	2688	1324	1380	1324
query77	414	303	307	303
query78	9398	8923	8903	8903
query79	1900	523	534	523
query80	1145	508	508	508
query81	563	230	222	222
query82	1064	131	130	130
query83	239	177	166	166
query84	275	77	78	77
query85	1411	318	310	310
query86	449	297	315	297
query87	3249	3090	3073	3073
query88	2955	2411	2415	2411
query89	370	284	299	284
query90	1875	195	194	194
query91	126	104	101	101
query92	63	49	51	49
query93	1616	599	612	599
query94	911	292	296	292
query95	376	264	275	264
query96	596	292	286	286
query97	3218	3037	3051	3037
query98	226	204	192	192
query99	1657	1296	1305	1296
Total cold run time: 262886 ms
Total hot run time: 170174 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e168993ad838f71491ca9cb8912f94692d6e73a0, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.68	0.06	0.05
query5	0.48	0.48	0.49
query6	1.13	0.71	0.71
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.56	0.49	0.51
query10	0.59	0.56	0.56
query11	0.15	0.12	0.12
query12	0.15	0.12	0.13
query13	0.60	0.60	0.61
query14	0.78	0.79	0.80
query15	0.90	0.86	0.87
query16	0.36	0.35	0.35
query17	0.99	0.99	0.98
query18	0.22	0.22	0.21
query19	1.82	1.78	1.72
query20	0.02	0.01	0.01
query21	15.40	0.76	0.66
query22	3.52	5.90	2.81
query23	18.03	1.41	1.33
query24	2.26	0.22	0.21
query25	0.18	0.08	0.08
query26	0.30	0.22	0.22
query27	0.46	0.24	0.23
query28	13.23	1.00	0.96
query29	12.53	3.35	3.34
query30	0.26	0.06	0.05
query31	2.85	0.42	0.40
query32	3.23	0.49	0.49
query33	2.93	2.91	2.91
query34	15.45	4.27	4.27
query35	4.27	4.29	4.28
query36	0.69	0.48	0.47
query37	0.19	0.16	0.15
query38	0.17	0.15	0.16
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.19 s
Total hot run time: 31.51 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 30, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 2e6cead into apache:master Aug 1, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Aug 4, 2024
…) (#38826)

pick (#38523)

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
dataroaring pushed a commit that referenced this pull request Aug 4, 2024
### Bug report:
Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

### Solution
Do not  use `toLowerCase` method;
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
…he#38523)

### Bug report:
Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

### Solution
Do not  use `toLowerCase` method;
dataroaring pushed a commit that referenced this pull request Aug 19, 2024
…) (#39529)

pick (#38523)

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…he#38523) (apache#39529)

pick (apache#38523)

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants