Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot exec from one pod to another #180

Closed
ceeaspb opened this issue Feb 7, 2018 · 10 comments
Closed

cannot exec from one pod to another #180

ceeaspb opened this issue Feb 7, 2018 · 10 comments

Comments

@ceeaspb
Copy link
Contributor

ceeaspb commented Feb 7, 2018

The exec call worked via the api when calling the api from outside of k8s but not when running from within a pod. Other api calls like set label, watch pod/namespace work.

Also exec as the user from command line works:

$ kubectl --as=system:serviceaccount:namespace:custom-account exec podname -ti -- sh -c "ls /"
bin    etc    lib    media  opt    root   sbin   sys    usr
dev    home   logs   mnt    proc   run    srv    tmp    var

But from the api :

        final Exec exec = new Exec();
        exec.exec(namespace, pod, cmd, true, false);

java.net.ProtocolException: Expected HTTP 101 response but was '403 Forbidden'
	at com.squareup.okhttp.ws.WebSocketCall.createWebSocket(WebSocketCall.java:123)
	at com.squareup.okhttp.ws.WebSocketCall.access$000(WebSocketCall.java:40)
	at com.squareup.okhttp.ws.WebSocketCall$1.onResponse(WebSocketCall.java:98)
	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:177)
	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

If I enable debug on the client via client.setDebugging(true)
we see

--> GET https://10.96.0.1/api/v1/namespaces/default/pods/server-0/exec?stdin=true&tty=false&command=sh&command=-c&command=blah HTTP/1.1
X-Stream-Protocol-Version: v4.channel.k8s.io,v3.channel.k8s.io,v2.channel.k8s.io,channel.k8s.io
User-Agent: Swagger-Codegen/1.0-SNAPSHOT/java
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: pJBCpoidqwWw3x30yFHbig==
Sec-WebSocket-Version: 13
--> END GET
<-- HTTP/1.1 403 Forbidden (54ms)
Content-Type: application/json
X-Content-Type-Options: nosniff
Date: Wed, 07 Feb 2018 19:00:44 GMT
Content-Length: 271
OkHttp-Sent-Millis: 1518030044051
OkHttp-Received-Millis: 1518030044052
Exception in thread "OkHttp Dispatcher" java.lang.NullPointerException
	at com.squareup.okhttp.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:223)
	at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:232)
	at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
	at com.squareup.okhttp.Call.access$100(Call.java:35)
	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:171)
	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Compared with a list pods it is missing the Bearer token.
The ProtocolException is not being logged presumably due to the NullPointerException.

This change seems to have worked around for me.

$ git diff util/src/main/java/io/kubernetes/client/util/WebSockets.java
diff --git a/util/src/main/java/io/kubernetes/client/util/WebSockets.java b/util/src/main/java/io/kubernetes/client/util/WebSockets.java
index 6bc088c..26b0382 100644
--- a/util/src/main/java/io/kubernetes/client/util/WebSockets.java
+++ b/util/src/main/java/io/kubernetes/client/util/WebSockets.java
@@ -94,7 +94,8 @@ public class WebSockets {
         headers.put(HttpHeaders.CONNECTION, HttpHeaders.UPGRADE);
         headers.put(HttpHeaders.UPGRADE, SPDY_3_1);
 
-        Request request = client.buildRequest(path, method, queryParams, new ArrayList<Pair>(), null, headers, new HashMap<String, Object>(), new String[0], null);
+        String[] localVarAuthNames = new String[] { "BearerToken" };
+        Request request = client.buildRequest(path, method, queryParams, new ArrayList<Pair>(), null, headers, new HashMap<String, Object>(), localVarAuthNames, null);
         streamRequest(request, client, listener);
     }
@brendandburns
Copy link
Contributor

Can you send a PR with that change?

Thanks

@karthikkondapally karthikkondapally mentioned this issue Feb 22, 2018
Closed
@ceeaspb
Copy link
Contributor Author

ceeaspb commented Feb 22, 2018

will get to this shortly

@brendandburns
Copy link
Contributor

brendandburns commented Feb 22, 2018 via email

brendandburns added a commit that referenced this issue Feb 27, 2018
@brendandburns
Merge pull request #202 from pushtechnology/master
dfc9d39 
set BearerToken in buildRequest fixes #180
@rjeberhard rjeberhard mentioned this issue Mar 14, 2018
Closed
@greglanthier
Copy link
Contributor

For those who stumble onto this issue while googling as I did I think this problem also impacts port-forwarding.

--> GET https://169.46.7.238:22285/api/v1/namespaces/waca-perf-test/pods/waca-perf-test/portforward?ports=8080&ports=9300 HTTP/1.1
X-Stream-Protocol-Version: v4.channel.k8s.io,v3.channel.k8s.io,v2.channel.k8s.io,channel.k8s.io
User-Agent: Swagger-Codegen/1.0-SNAPSHOT/java
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: Xkr3fLWGMIG4Z/iAZB0vZw==
Sec-WebSocket-Version: 13
--> END GET
<-- HTTP/1.1 403 Forbidden (357ms)
Content-Type: text/plain
X-Content-Type-Options: nosniff
Date: Mon, 19 Mar 2018 01:58:58 GMT
Content-Length: 86
OkHttp-Sent-Millis: 1521424738263
OkHttp-Received-Millis: 1521424738321
Exception in thread "OkHttp Dispatcher" java.lang.NullPointerException
	at com.squareup.okhttp.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:223)
	at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:232)
	at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
	at com.squareup.okhttp.Call.access$100(Call.java:35)
	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:171)
	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I'll try to run a test using the edit proposed in #180 (comment) in the morning.

I looked around but could not find a io.kubernetes:client-java:1.0.0-beta3-SNAPSHOT in a repo anywhere. Are SNAPSHOT builds of the io.kubernetes:client-java library produced / available?

@brendandburns
Copy link
Contributor

brendandburns commented Mar 19, 2018 via email

@greglanthier
Copy link
Contributor

Thanks @brendanburns!

I did run my test this afternoon with a local build from HEAD and confirmed that the changes from PR #202 corrects the problem I was seeing.

@brendandburns brendandburns mentioned this issue Jan 11, 2019
Closed
@eshengit
Copy link

I still have the same problem even I use 5.0.0-SNAPSHOT
Sec-WebSocket-Protocol: v4.channel.k8s.io
User-Agent: Swagger-Codegen/1.0-SNAPSHOT/java
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: Hmm7aq1wnQvNp4yF/aGIkw==
Sec-WebSocket-Version: 13
--> END GET

<-- HTTP/1.1 403 Forbidden (62ms)
Audit-Id: 815889bf-7dd6-4b86-b965-adcf51fbdc7d
Content-Type: application/json
X-Content-Type-Options: nosniff
Date: Thu, 28 Mar 2019 23:02:52 GMT
Content-Length: 306
OkHttp-Sent-Millis: 1553814172952
OkHttp-Received-Millis: 1553814172962Exception in thread "OkHttp Dispatcher" java.lang.NullPointerException
at com.squareup.okhttp.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:223)
at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:232)
at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
at com.squareup.okhttp.Call.access$100(Call.java:35)
at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:171)
at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

@eshengit
Copy link

could anyone help?

@anonymous19890806
Copy link

Facing the same error :

java.net.ProtocolException: Expected HTTP 101 response but was '403 Forbidden' at com.squareup.okhttp.ws.WebSocketCall.createWebSocket(WebSocketCall.java:123) at com.squareup.okhttp.ws.WebSocketCall.access$000(WebSocketCall.java:40) at com.squareup.okhttp.ws.WebSocketCall$1.onResponse(WebSocketCall.java:98) at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:177) at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
--

Any help would be much appreciated!!

@malli31
Copy link

malli31 commented Mar 6, 2020 •
edited
Loading

I am trying to copy from one pod to another using InputStream dataStream = copy.copyFileFromPod(namespacename, podname, filePathToCopy);
I am able to run this from my system, but when running from pods I am facing this issue,

If My application is running from my system, then only one pod comes into picture, my application running locally in my laptop is able to read the file from pod.

if I convert my application into container and run as pod the communication breaks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@greglanthier @eshengit @brendandburns @ceeaspb @malli31 @anonymous19890806