log4j conflict

[lianxiaobao]

请在提交你的 issue 前，请回答以下问题，这有助于社区快速定位问题，谢谢！ 🙏

Environment Description（运行环境描述）

• StreamX Version: 1.1.1
• JVM version (java -version): 1.8
• OS version (uname -a if on a Unix-like system): centos 7

Bug Description（Bug 描述）

After starting any task, the start log reports an error

java.lang.reflect.UndeclaredThrowableException
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1944)
	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at com.streamxhub.streamx.flink.submit.impl.YarnApplicationSubmit$.doSubmit(YarnApplicationSubmit.scala:66)
	at com.streamxhub.streamx.flink.submit.trait.FlinkSubmitTrait$class.submit(FlinkSubmitTrait.scala:71)
	at com.streamxhub.streamx.flink.submit.impl.YarnApplicationSubmit$.submit(YarnApplicationSubmit.scala:50)
...
Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn Application Cluster
	at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:465)
	at com.streamxhub.streamx.flink.submit.impl.YarnApplicationSubmit$$anon$1.call(YarnApplicationSubmit.scala:82)
	at com.streamxhub.streamx.flink.submit.impl.YarnApplicationSubmit$$anon$1.call(YarnApplicationSubmit.scala:66)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
	... 147 more
Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. 
Diagnostics from YARN: Application application_1639901608638_2359 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1639901608638_2359_000001 exited with  exitCode: 1
Failing this attempt.Diagnostics: Exception from container-launch.
Container id: container_e25_1639901608638_2359_01_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:972)
	at org.apache.hadoop.util.Shell.run(Shell.java:869)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1170)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:236)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:305)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1
For more detailed output, check the application tracking page: http://bmr-edap-edshswqx-0:8088/cluster/app/application_1639901608638_2359 Then click on links to logs of each attempt.
. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1639901608638_2359
	at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:1201)
	at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:593)
	at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:458)
	... 152 more

yarn logs

How to Reproduce（如何重现这个 Bug）

重现 Bug 现场的步骤，比如：

1. The packaged UDF contains log4j
2. Submit the code to flash lib, flink/lib. or streamx/jars/ of HDFS
3. run

methods of resolution

Dependency hierarchy, see where the log conflicts are

fix udf log4j conflict

Requirement or improvement（诉求 & 改进建议）

• add java parameter
• Start the job of Flink job add： -Dlog4j.ignoreTC=true