开源了不能用，flink on yarn 根本无法启动，你们可以吗

[js17741166]

No description provided.

[js17741166]

flinkx版本tag r1.5.0 依然报错

./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop

Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms

[lijiangbo]

flinkx版本tag r1.5.0 依然报错

./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop

Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms

yarn上有没有启动一个flink session

[lijiangbo]

flinkx版本tag r1.5.0 依然报错
./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop
Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms

yarn上有没有启动一个flink session

目前用yarn方式提交只支持session模式，需要预先启动一个flink session，名称为默认的"Flink session cluster"才可以，后面会支持PerJob模式在yarn上提交任务

[kanata163]

已支持PerJob模式