[workspace-controller] Need to investigate how to support Che >7.9.0 changes

[amisevsk]

Is your enhancement related to a problem? Please describe.

Since Che has moved the JWT proxy pod within the workspace deployment, many plugins only listen on localhost and depend on proxied requests from the JWT proxy pod.

In the workspace controller, it's possible to deploy using basic routing, in which case no local proxy is present. We will likely need to mirror something like #16227 to support this case.

[amisevsk]

Note that using oauth proxy as a passthrough won't work for our case as we likely need to proxy multiple ports. Using oauth-proxy for this would mean one container per exposed endpoint.

[sleshchenko]

it's the comment I'm going to share issues I faced and some thoughts about their fixes:

1. in the latest plugin registry processes are configured to listen to only localhost and to make it available some proxy is needed( as Angel mentioned).
2. Theia Webview started requiring TLS, so it's became quite a requirement to use self-signed certificates by default. And OpenShift provides us a mechanism to get that certificates pretty easy, but for k8s the solution is not obvious.
   We may require (on k8s) wildcard certificate to start with and then use it for every endpoint... It makes sense to take a look at the single-host approach platform team is working on.

[sleshchenko]

From discussion on the latest prioritization:

1. We don't care about Single User host, so the solution with passthrough proxy where no authentication is required has low priority.
2. Due plan to switch Che Server to use devworkspace controller which currently supports kubernetes, we need a solution for k8s as well.

About TLS:

• Kubernetes: there is no out-of-the-box solution. Devworkspace controller should do the same as Che operator does - use configured secret or generate certificates via the job we have, and then propagate that to every namespace where workspaces are created. The dedicated issue for that [devworkspace] Support TLS routing on kubernetes #17853;
• OpenShift: has an out-of-the-box solution and we just need to enable TLS for the workspace routes;

Authentication:

• Kubernetes: there is no embedded authentication in the vanilla K8s. The solution we can propose - implement keycloak routing that will authentication user's via keycloak instance... Open question - how to link keycloak users with cluster authority. It's needed if we need to use webhooks to protect data in the workspaces pods.
• OpenShift: in the POC we have openshift-oauth-proxy routing that seems to work fine with a few limitations:
  • it does not have an ability to restrict access to the endpoint by user's id. Again, if we want to provide creator-only access, we would need that. I've created an issue on OpenShift OAuth Proxy side for discussion and there is no any activity =( Implement an ability to specify authenticated userIDs openshift/oauth-proxy#168
  • OpenShift OAuth Proxy supports only proxying one container port, which means each secure workspace server needs its own os-oauth-proxy container...
  • every host needs its own authentication to be done. So, when the user starts a workspace - they need to authenticate at least 3 times - theia, theia webview, che-machine-exec(not sure). Probably we would need some auto authentication mechanism something we implemented for jwtproxy + loader.js that automatically sets cookies with machine_token if user is authenticated on the Che Server host (where loader.js lives) and authorized to use the requested workspace.
    * the single-host mode powered by traefik could help to solve this issue

So, about OpenShift authentication: after taking into account the shared limitation we need:

• contribute into or fork and implement just for us needed features
• or find another solution that will satisfy our needs.

^ the issue to investigate possible authentication/authorization we could use #17856

Probably there some other aspects which are missed but TLS and authentication are the main issues we should address. Keeping this issue opened until I created separate issues for the described directions.

[amisevsk]

The above comment is an excellent summary of the current understanding. Two additional considerations I'd like to add:

• The way we implement authentication will eventually have to work well with the Che server. I'm presuming that the Che server will continue to be responsible for authenticating users, storing workspaces, serving the dashboard, etc. -- if this is the case, we'll need to figure out how to link workspaces created from CRs with users in Che's user DB. This suggests keycloak as a good approach, since we use that in Che currently, and the main question would be managing cluster identities.
• We're currently doing more in a security sense than upstream Che, as I don't believe we have any restriction on modifying workspace resources or creating execs into workspaces. We should consider whether we want to relax our webhooks for non-web-terminal devworkspaces, to align with Che's current purposes. One of the reasons we are considering the openshift oauth proxy is that it enables getting a user's OpenShift token into a workspace, but if our goal is to match functionality with Che and integrate with it, we may consider punting on this feature for now.

Overall I don't think we have a clear path forward on the authentication question.

[sleshchenko]

#16568 (comment) now has links to new issues