Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After updating to Azul OpenJDK 8.46 (~=8u252) the client fails with java.net.SocketException: Software caused connection abort: socket write error #2145

Closed
jgoeres opened this issue Apr 20, 2020 · 6 comments · Fixed by #2176
Assignees
Labels

Comments

@jgoeres
Copy link

jgoeres commented Apr 20, 2020

Hi,
we recently updated our Azul Java 8 JRE from 8.44.0.12 (which corresponds to regular 8u242) to 8.46.0.20 (which corresponds to regular 8u252). We now consistently get exceptions when doing any Kubernetes API operations through the kubernetes client, both when running stuff inside the cluster and when running it outside (in the IDE), see below.
We updated from 4.7.1 to 4.9.1 but that didn't change anything.

Regards

J

io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list]  for kind: [SparkApplication]  with name: [null]  in namespace: [foobar]  failed.
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:155) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:621) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:70) ~[kubernetes-client-4.9.1.jar:?]
	at x.y.z.spark.shim.SparkK8sShim.deleteAllSparkApplicationsInStates(SparkK8sShim.java:184) ~[classes/:?]
	at x.y.z.services.k8s.K8sSparkApplicationService.deleteCompletedAndFailedSparkApplications(K8sSparkApplicationService.java:116) ~[classes/:?]
	at x.y.z.services.BackgroundCleanupService.runBackgroundCleanupOfSparkJobs(BackgroundCleanupService.java:116) ~[classes/:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_252]
	at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) [?:1.8.0_252]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java) [?:1.8.0_252]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_252]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_252]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
Caused by: java.net.SocketException: Software caused connection abort: socket write error
	at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:1.8.0_252]
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) ~[?:1.8.0_252]
	at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[?:1.8.0_252]
	at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431) ~[?:1.8.0_252]
	at sun.security.ssl.OutputRecord.write(OutputRecord.java:417) ~[?:1.8.0_252]
	at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:894) ~[?:1.8.0_252]
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:865) ~[?:1.8.0_252]
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[?:1.8.0_252]
	at okio.Okio$1.write(Okio.java:79) ~[okio-1.15.0.jar:?]
	at okio.AsyncTimeout$1.write(AsyncTimeout.java:180) ~[okio-1.15.0.jar:?]
	at okio.RealBufferedSink.flush(RealBufferedSink.java:224) ~[okio-1.15.0.jar:?]
	at okhttp3.internal.http2.Http2Writer.windowUpdate(Http2Writer.java:262) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http2.Http2Connection.start(Http2Connection.java:517) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http2.Http2Connection.start(Http2Connection.java:504) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.RealConnection.startHttp2(RealConnection.java:299) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:288) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:169) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:134) ~[kubernetes-client-4.9.1.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68) ~[kubernetes-client-4.9.1.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:114) ~[kubernetes-client-4.9.1.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.6.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.6.jar:?]
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257) ~[okhttp-3.12.6.jar:?]
	at okhttp3.RealCall.execute(RealCall.java:93) ~[okhttp-3.12.6.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:469) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:430) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:412) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:151) ~[kubernetes-client-4.9.1.jar:?]
	... 13 more
@rohanKanojia
Copy link
Member

I moved to 1.8.0_251 last week and I'm also seeing similar issues:

Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Operation: [create]  for kind: [CustomResourceDefinition]  with name: [null]  in namespace: [null]  failed.
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:338)
	at io.fabric8.CustomResourceDefinitionEx.main(CustomResourceDefinitionEx.java:11)
Caused by: java.net.SocketException: Broken pipe (Write failed)
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
	at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
	at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
	at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:894)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:865)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
	at okio.Okio$1.write(Okio.java:79)

@jgoeres
Copy link
Author

jgoeres commented Apr 20, 2020

Yes, the SocketException with "broken pipe" is another we get, the actual cause seems a bit random, we also have the cause shown below.
BTW: While we use Azul JRE, we can also reproduce it with Oracle u251.

io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list]  for kind: [SparkApplication]  with name: [null]  in namespace: [foobar]  failed.
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:155) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:621) ~[kubernetes-client-4.9.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:70) ~[kubernetes-client-4.9.1.jar:?]
	at x.y.z.spark.shim.SparkK8sShim.deleteAllSparkApplicationsInStates(SparkK8sShim.java:184) ~[classes/:?]
	at x.y.z.services.k8s.K8sSparkApplicationService.deleteCompletedAndFailedSparkApplications(K8sSparkApplicationService.java:116) ~[classes/:?]
	at x.y.z.services.BackgroundCleanupService.runBackgroundCleanupOfSparkJobs(BackgroundCleanupService.java:116) ~[classes/:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_251]
	at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) [?:1.8.0_251]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java) [?:1.8.0_251]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_251]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_251]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_251]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_251]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251]
Caused by: java.net.SocketException: Software caused connection abort: socket write error
	at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:1.8.0_251]
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) ~[?:1.8.0_251]
	at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[?:1.8.0_251]
	at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431) ~[?:1.8.0_251]
	at sun.security.ssl.OutputRecord.write(OutputRecord.java:417) ~[?:1.8.0_251]
	at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:894) ~[?:1.8.0_251]
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:865) ~[?:1.8.0_251]
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[?:1.8.0_251]

@manusa manusa added the bug label Apr 22, 2020
@manusa
Copy link
Member

manusa commented Apr 23, 2020

Getting this with AdoptOpenJDK too:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.252-b09, mixed mode)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable (KubernetesClientException.java:64)
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable (KubernetesClientException.java:53)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShiftAPIGroups (OpenshiftAdapterSupport.java:80)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShift (OpenshiftAdapterSupport.java:55)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isAdaptable (OpenshiftAdapterSupport.java:43)
    at io.fabric8.kubernetes.client.BaseClient.isAdaptable (BaseClient.java:118)
    at org.eclipse.jkube.kit.common.util.OpenshiftHelper.isOpenShift (OpenshiftHelper.java:55)
    at org.eclipse.jkube.maven.plugin.mojo.build.ApplyMojo.executeInternal (ApplyMojo.java:237)
    at org.eclipse.jkube.maven.plugin.mojo.build.AbstractJKubeMojo.execute (AbstractJKubeMojo.java:68)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: okhttp3.internal.http2.ConnectionShutdownException
    at okhttp3.internal.http2.Http2Connection.newStream (Http2Connection.java:247)
    at okhttp3.internal.http2.Http2Connection.newStream (Http2Connection.java:230)
    at okhttp3.internal.http2.Http2Codec.writeRequestHeaders (Http2Codec.java:117)
    at okhttp3.internal.http.CallServerInterceptor.intercept (CallServerInterceptor.java:50)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.connection.ConnectInterceptor.intercept (ConnectInterceptor.java:45)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at okhttp3.internal.cache.CacheInterceptor.intercept (CacheInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at okhttp3.internal.http.BridgeInterceptor.intercept (BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept (RetryAndFollowUpInterceptor.java:127)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept (BackwardsCompatibilityInterceptor.java:134)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept (ImpersonatorInterceptor.java:68)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3 (HttpClientUtils.java:114)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
    at okhttp3.RealCall.getResponseWithInterceptorChain (RealCall.java:257)
    at okhttp3.RealCall.execute (RealCall.java:93)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShiftAPIGroups (OpenshiftAdapterSupport.java:71)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShift (OpenshiftAdapterSupport.java:55)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isAdaptable (OpenshiftAdapterSupport.java:43)
    at io.fabric8.kubernetes.client.BaseClient.isAdaptable (BaseClient.java:118)
    at org.eclipse.jkube.kit.common.util.OpenshiftHelper.isOpenShift (OpenshiftHelper.java:55)
    at org.eclipse.jkube.maven.plugin.mojo.build.ApplyMojo.executeInternal (ApplyMojo.java:237)
    at org.eclipse.jkube.maven.plugin.mojo.build.AbstractJKubeMojo.execute (AbstractJKubeMojo.java:68)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

@manusa
Copy link
Member

manusa commented Apr 23, 2020

Relates to: square/okhttp#5970 square/okhttp#5977 square/okhttp#5976

@drametoid
Copy link

drametoid commented May 2, 2020

So I'm facing the same issue as mentioned by @jgoeres here. For me it failed at:
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Operation: [create] for kind: [Pod] with name: [null] in namespace: [default] failed.

After doing the suggested workaround of downgrading the java version (from 8u252-b09-1-18.04 to 8u162-b12-1), I am no longer seeing the error but now it fails at:
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Pod] with name: [spark-pi-1587656135431-driver] in namespace: [default] failed.

Also, the second "Caused by" in my stack trace is always one of these two:

- Caused by: java.net.SocketException: Broken pipe (Write failed)
- Caused by: okhttp3.internal.http2.ConnectionShutdownException [currently seeing this one]

I have met with dead-ends so far in trying to resolve this issue and have been stuck here quite a while so any help/work around here would be really appreciated.

Attaching two logs and a stackoverflow question I recently posted for your reference:
Log #1
Log #2
StackOverflow Question

Thanks.

@LeonardAukea
Copy link

Facing similar issues. Not sure how to proceed kubeflow/spark-operator#1277

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants