Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][api] Upgrade from 1.2.0 to 1.3.0,Run the spark task, the error is "spark task params is not valid" #3397

Closed
xingchun-chen opened this issue Aug 4, 2020 · 0 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@xingchun-chen
Copy link
Contributor

xingchun-chen commented Aug 4, 2020

1.Upgrade from 1.2.0 to 1.3.0
2.Run the spark task, the error is "spark task params is not valid"
image

The error message is as follows:

[INFO] 2020-08-04 15:54:20.988 - [taskAppId=TASK-2-31-42]:[72] - spark task params {"mainArgs":"/chenxingchun/words.txt /chenxingchun/out_c","driverMemory":"512M","executorMemory":"2G","programType":"SCALA","mainClass":"com.journey.spark.WordCount","driverCores":1,"deployMode":"cluster","executorCores":2,"mainJar":{"res":"bigdata-1.0-SNAPSHOT.jar"},"numExecutors":2,"localParams":[],"others":"","resourceList":[{"res":"test.sh"},{"res":"udf-demo-SNAPSHOT.jar"}]}
[ERROR] 2020-08-04 15:54:20.989 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[140] - task scheduler failure
java.lang.RuntimeException: spark task params is not valid
at org.apache.dolphinscheduler.server.worker.task.spark.SparkTask.init(SparkTask.java:77)
at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:126)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO] 2020-08-04 15:54:21.996 org.apache.dolphinscheduler.service.log.LogClientService:[100] - view log path /data1/dolphinscheduler_1.3.2/logs/2/31/42.log

Which version of Dolphin Scheduler:
-[1.3.2-release]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants