Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] Which version of Flink is used in DolphinScheduler #2226

Closed
oceanos opened this issue Mar 18, 2020 · 4 comments
Closed

[QUESTION] Which version of Flink is used in DolphinScheduler #2226

oceanos opened this issue Mar 18, 2020 · 4 comments
Labels
question Further information is requested

Comments

@oceanos
Copy link
Contributor

oceanos commented Mar 18, 2020

Describe
Cmd:
flink run -m yarn-cluster -ys 1 -yn 1 -yjm 1G -ytm 1G -d -c org.apache.flink.examples.java.wordcount.WordCount WordCount.jar

Error:
Caused by: java.io.FileNotFoundException: JAR file does not exist: -ys
at org.apache.flink.client.cli.CliFrontend.getJarFile(CliFrontend.java:819)
at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:796)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:196)

Version of DolphinScheduler:
-[1.2.0]

Version of Flink:
-[1.9.0]
-[1.10.0]

Requirement or improvement

  • Some flink command line arguments or args order may not be compatible with the new version.
  • Please add more infomation about flink task in DolphinScheduler, such as the adaptation version.
@oceanos oceanos added the question Further information is requested label Mar 18, 2020
@davidzollo
Copy link
Contributor

@LoveEvenWong could you answer this issue?

@davidzollo
Copy link
Contributor

#711

@davidzollo
Copy link
Contributor

@oceanos
because you use shell task , you can set FLINK_HOME under the conf/env/dolphinscheduler-env.sh, you can use any version if you like

@Kyofin
Copy link
Contributor

Kyofin commented Apr 3, 2020

I use flink 1.10 test is ok!

  1. I remove -yn 1 in the command (maybe -yn dosen't work in flink 1.10)
  2. I add below in the conf/env/dolphinscheduler-env.sh file
export HADOOP_CLASSPATH=`${HADOOP_HOME}/bin/hadoop classpath`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants