Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](routine load) fix show routine load task result incorrect (#38523) #39529

Merged
merged 1 commit into from
Aug 19, 2024
Merged

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Aug 18, 2024

pick (#38523)

Create a job:

CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);

show routine load task:

SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";

result:

ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled

Do not use toLowerCase method;

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@sollhui
Copy link
Contributor Author

sollhui commented Aug 18, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50107 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eca70970d1da4fde1bb4f3df81ad4d4c1633f6e9, data reload: false

------ Round 1 ----------------------------------
q1	18299	4378	4373	4373
q2	2082	196	186	186
q3	10361	1888	1951	1888
q4	10350	1287	1345	1287
q5	8371	3868	3883	3868
q6	274	149	143	143
q7	2050	1651	1617	1617
q8	9326	2775	2737	2737
q9	10613	10356	10253	10253
q10	8651	3536	3559	3536
q11	442	266	285	266
q12	494	329	331	329
q13	18383	3934	4040	3934
q14	375	350	348	348
q15	560	509	501	501
q16	711	606	616	606
q17	1150	990	959	959
q18	7211	6840	6947	6840
q19	1780	1580	1553	1553
q20	552	351	336	336
q21	4506	4144	4115	4115
q22	537	432	451	432
Total cold run time: 117078 ms
Total hot run time: 50107 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4321	4311	4316	4311
q2	354	264	264	264
q3	4219	4197	4148	4148
q4	2750	2757	2759	2757
q5	7188	7135	7116	7116
q6	252	141	138	138
q7	3217	2920	2907	2907
q8	4457	4530	4550	4530
q9	16884	16787	16817	16787
q10	4240	4327	4261	4261
q11	788	705	706	705
q12	1053	860	885	860
q13	7142	3725	3783	3725
q14	483	447	450	447
q15	555	496	507	496
q16	745	718	706	706
q17	3871	3840	3895	3840
q18	8813	8694	8909	8694
q19	1757	1770	1710	1710
q20	2446	2215	2163	2163
q21	8623	8527	8620	8527
q22	1082	979	1024	979
Total cold run time: 85240 ms
Total hot run time: 80071 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 209060 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eca70970d1da4fde1bb4f3df81ad4d4c1633f6e9, data reload: false

query1	983	417	409	409
query2	6776	2299	2355	2299
query3	6946	228	227	227
query4	20325	18231	18323	18231
query5	19929	6805	6810	6805
query6	324	262	277	262
query7	4190	341	357	341
query8	484	459	477	459
query9	3278	2789	2759	2759
query10	497	379	356	356
query11	11474	10983	10884	10884
query12	167	110	113	110
query13	5667	740	739	739
query14	18594	13944	13492	13492
query15	410	280	248	248
query16	6499	328	303	303
query17	1649	1468	937	937
query18	2320	482	496	482
query19	251	184	198	184
query20	117	115	113	113
query21	246	152	150	150
query22	5166	5136	4984	4984
query23	32709	32342	32343	32342
query24	7039	6593	6598	6593
query25	580	477	481	477
query26	555	198	198	198
query27	1806	334	335	334
query28	6042	2457	2392	2392
query29	2896	2801	2746	2746
query30	289	212	208	208
query31	1019	848	843	843
query32	101	94	80	80
query33	478	339	351	339
query34	893	561	542	542
query35	1189	1038	943	943
query36	1287	1238	1173	1173
query37	122	89	92	89
query38	3130	2936	2931	2931
query39	1485	1449	1432	1432
query40	252	148	150	148
query41	156	159	152	152
query42	109	116	113	113
query43	818	651	683	651
query44	1242	790	787	787
query45	283	272	270	270
query46	1264	985	1037	985
query47	2181	1825	1871	1825
query48	1066	745	758	745
query49	856	584	599	584
query50	955	722	689	689
query51	4805	4721	4731	4721
query52	116	105	110	105
query53	514	378	377	377
query54	2749	2559	2548	2548
query55	112	102	107	102
query56	330	299	283	283
query57	1253	1218	1167	1167
query58	323	310	310	310
query59	3530	3330	3407	3330
query60	298	270	285	270
query61	155	157	167	157
query62	932	522	577	522
query63	542	398	410	398
query64	2717	1691	1614	1614
query65	3643	3617	3593	3593
query66	1248	818	817	817
query67	16201	18048	14696	14696
query68	8289	694	700	694
query69	641	420	402	402
query70	1760	1365	1502	1365
query71	464	365	368	365
query72	6610	3590	3583	3583
query73	809	374	371	371
query74	6439	5895	6080	5895
query75	4744	3854	3892	3854
query76	4638	1215	1227	1215
query77	712	431	423	423
query78	12845	11508	11832	11508
query79	8962	703	695	695
query80	2852	547	548	547
query81	581	284	277	277
query82	1704	134	130	130
query83	272	222	218	218
query84	293	98	98	98
query85	1495	409	412	409
query86	403	327	336	327
query87	3279	3039	3053	3039
query88	5399	2533	2548	2533
query89	445	342	333	333
query90	1821	263	269	263
query91	192	162	160	160
query92	95	86	87	86
query93	6017	620	612	612
query94	943	255	249	249
query95	1172	1112	1110	1110
query96	677	346	341	341
query97	6603	6422	6493	6422
query98	228	220	208	208
query99	3042	933	992	933
Total cold run time: 320988 ms
Total hot run time: 209060 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eca70970d1da4fde1bb4f3df81ad4d4c1633f6e9, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.26	0.06	0.06
query4	1.82	0.07	0.07
query5	0.54	0.53	0.52
query6	1.29	0.63	0.62
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.49
query10	0.55	0.54	0.55
query11	0.14	0.10	0.10
query12	0.12	0.10	0.10
query13	0.63	0.63	0.62
query14	0.79	0.81	0.79
query15	0.81	0.78	0.78
query16	0.37	0.39	0.37
query17	1.03	1.00	1.03
query18	0.22	0.27	0.24
query19	1.96	1.85	1.85
query20	0.02	0.02	0.01
query21	15.45	0.56	0.55
query22	2.01	2.19	1.72
query23	17.55	1.12	0.86
query24	5.52	1.06	0.71
query25	0.39	0.09	0.06
query26	0.59	0.15	0.16
query27	0.05	0.04	0.04
query28	7.81	0.78	0.72
query29	12.70	2.22	2.30
query30	0.79	0.72	0.72
query31	2.83	0.40	0.38
query32	3.37	0.52	0.52
query33	3.08	3.09	3.03
query34	15.28	4.80	4.82
query35	4.85	4.84	4.84
query36	1.10	1.02	1.03
query37	0.07	0.06	0.06
query38	0.05	0.03	0.06
query39	0.04	0.04	0.03
query40	0.19	0.16	0.18
query41	0.08	0.03	0.04
query42	0.04	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 105.13 s
Total hot run time: 30.96 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit eca70970d1da4fde1bb4f3df81ad4d4c1633f6e9 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@dataroaring dataroaring merged commit 71b33f0 into apache:branch-2.0 Aug 19, 2024
22 of 25 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…he#38523) (apache#39529)

pick (apache#38523)

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants