Multiuser-OpenShift on OpenShift behind Proxy not starting Workspaces #14052
Comments
sunix
added
kind/bug
|
@wgbeckmann I cannot see any evidence thats' a proxy problem: the fact that che-server is not able to contact the workspace may be caused by different problems. It would be useful if you could retrieve theia containers logs anyway or look for any failure in the events or any container that is restarted in the workspace pod. Not saying that's not a proxy problem but it may be something else as well. Another thing is that you are using @skabashnyuk is @wgbeckmann configuring the proxy settings correctly? |
l0rd
added
status/info-needed
|
Hmm, I cannot find en error in the logs. Theia: VS-Code LWT-Proxy |
|
@l0rd can you tell me how to install eclipse che with |
|
Perhaps I forgot to mention. I use multi-user mode. |
|
Ok, i tried |
|
Ok, I just install the single-user version there it runs. (Hmm, but we need multiuser) |
wgbeckmann
changed the title
|
No, also the single-user version is not working. The workspace comes up, but you cannot start a terminal or task. --> There are problems with the communication... |
|
Now I added to the CHE_WORKSPACE_NO__PROXY the ip 172.30.0.1 and restartet the workspace and tada, there is a Terminal... |
|
It looks like you have solved your problem @wgbeckmann We can add that to the documentation but it's still not clear what's the platform where you have deployed Che: Kubernetes, OpenShift or Docker Desktop ? |
|
No, I just tried to find out what works and what do not work. Single User Eclipse Che is working behind a proxy on OpenShift The platform is deployed on OpenShift installed with Ansible (so it is not a minishift installation) |
|
I have also encountered the same issue as described in the original post. I'm also trying to set up Eclipse Che multiuser behind a proxy. Has a fix been found? |
|
No, I'm using singleuser mode at the moment, but also only after playing arround with proxy-settings at multiple places. |
https://github.com/eclipse/che-jwtproxy might be an issue. |
|
I have the same issue when running in multiuser mode on an Openshift deployment behind a corporate proxy. I was able to get past the server ‘theia’ in container not available error, this was down to jwtproxy, it was attempting to go through the corporate proxy even though no_proxy was set, I removed the proxy settings entirely just for that container and it was then able to start Theia - clearly not a sufficient workaround as changes are lost for each new workspace. It looks as though Go in jwtproxy may not be respecting the no_proxy property correctly. Unfortunately there is then an additional issue, once in Theia it is not possible to retrieve the workspace configuration, this time axios seems to be mis-handling the no_proxy configuration. |
|
Hey @andy316x , thanks for the pointer. I'll try that given the chance. |
|
@AmitChameides, I have set the wildcard domain of my openshift router, and used both variations of no proxy syntax, .my.router.domain and *.my.router.domain. I also have the AWS nodes domain suffixes, I.e. .compute.internal. - not sure it matters for Che but added anyway. |
skabashnyuk
added
the
severity/P1
|
Alright @andy316x. Thanks. |
|
@AmitChameides fyi #14335 |
|
Oh, cool! |
|
I hope so. |
|
Just try to start a Workspace with 7.2.0-SNAPSHOT. |
|
@wgbeckmann good to hear you also managed to bypass the JWTProxy issue. |
|
@AmitChameides, I did try the same trick with the Theia container of removing the proxy configuration, it was then able to load the workspace configuration but caused other issues, I expect because Theia legitimately needs access to the internet through the proxy. So either one of the components does not respect the no_proxy parameter or I have misconfigured the no_proxy parameter, not sure which one it is at the minute. Great to see the jwtproxy fix, hopefully will get merged soon. |
|
Fix is merged, please let us know if it works for you now, thanks. |
|
I created a new Eclipse Che OCP project to test this out (I wanted to make sure it's getting the most recent version and using modified images as little as possible), and unfortunately the error persisted. I was still getting the Theia error, and removing the proxy settings from the JwtProxy continued to bypass this issue (Only to cause more issues later on). |
|
@mshaposhnik to test it, I try to deploy the "che" deployment-config in my Openshift installtion tomorrow . That would pull the nightly build Image eclipse/che-server:nightly. Is that right? |
|
@wgbeckmann yes, but make sure that image pull policy is set to |
|
@mshaposhnik Ok, I tested ist and the workspace comes up! |
|
@wgbeckmann is this issue actually resolved for you? The workspace now loads for me with the JWTProxy fix, as was the case when I manually removed the proxy environment variable from the JWTProxy container, however I still get the problem where it tells me the workspace configuration cannot be loaded, so I still don't think Che works behind a corporate proxy on OpenShift? I did some digging on the subsequent issue, I believe it happens when the che-theia container attempts to call the workspace api using the workspace-client package. This package uses axios to make the REST request, the problem is the version of axios that is used is 0.18.0, which will accept http_proxy/https_proxy config, but not no_proxy - this is added in 0.19.0. I think the che workspace client needs to update to axios v0.19.0. If you could let me know how you are getting on that would really help. |
@andy316x make sure you are running che7(devfile based) workspaces on che7 |
|
@skabashnyuk thanks for your response. I am running the latest nightly che so I assume that is che7 and choosing the default workspace that is selected when creating a workspace, so I presume that is devfile based. I believe it is a proxy problem but can’t be certain. I tried removing the proxy environment variables from the che-theia container and it seems then to be able to load the workspace configuration. I then get subsequent errors which I have not investigated further, but I presume this could be because Theia legitimately needs to go through the proxy for some subsequent calls. I think this demonstrates Theia is not respecting the no_proxy property, which would seem to make sense as the version of a axios used by che/workspace-client does not seem to support no_proxy. Any thoughts on this would be greatly appreciated, I have been stuck with this particular problem for a long time now with no known workaround. I am really keen to adopt che within our organisation as I think it will add a huge amount of value. |
|
@skabashnyuk The error was "Failed to load workspace configuration" - apologies |
|
Just saying I have the same situation as @andy316x . We would love to use Che, but I haven't been able to deploy Che due to all the various proxy-caused problems (I also have the 'Failed to load workspace configuration' error) |
|
@andy316x @Clicksurfer I'm sorry that you have such UX. Can you create a separate issue and w will try to provide you some help/inputs? |
|
@andy316x @Clicksurfer it's hard to figure out if your problems are related and if they are really related to this issue. We need the following details (preferably in a new issue):
|
Bug
In OpenShift, behind a Proxy Eclipse Che cannot start a workspace.
In the Che-Window I can see:
and then, after some time:
Additionaly I tried to add java-proxy options to the docker-container by setting the java opts in the environment via openshift:
-Dhttp.proxyHost=a.b.c.d -Dhttp.proxyPort=8080 -Dhttps.proxyHost=a.b.c.d -Dhttps.proxyPort=8080 -Dhttp.nonProxyHosts='localhost|127.0.0.1|*.my-company.de|vm-openshift-02|*.vm-openshift-02|*.vm-openshift-02.my-company.de|172.30.*|10.*'I can see in the log of the che-pod that the options are used, but it didn't help.
Che version
7.0.0-rc-5.0-SNAPSHOT
Steps to reproduce
./deploy_che.sh --multiuser --project=eclipse-che-mu.....
Expected behavior
a workspace is running ;-)
Runtime
Installation method
deploy_che.sh
Environment
CentOS Linux release 7.6.1810 (Core)
and a Proxy
There are so many docker container. From which one do you need the log?
I looked thrue the logs and I cann not see anything special.
So I think, that the communication between the workspace and the core do not work, because the sender thinks he has to use the proxy ... (but it is only a guess...)
The text was updated successfully, but these errors were encountered: