Improve isolation of Che theia and che-machine-exec components

[skabashnyuk]

Is your task related to a problem? Please describe.

Under some conditions, there is a possibility to reach the port of one workspace from another workspace. To improve the isolation of the major Eclipse Che components we would like to.

Describe the solution you'd like

1. Use single pod for jwt proxy and other workspace containers.
2. make machine-exec, theia-remote-runtime and theia endpoints listening to localhost only

Describe alternatives you've considered

n/a

Additional context

n/a

[amisevsk]

How does this affect #11476? Would each pod in #11476 require a JWT container?

[skabashnyuk]

How does this affect #11476? Would each pod in #11476 require a JWT container?

I don't know how it affects it. At this moment all our tooling(theia and che-machine-exec) runs in a single pod - no?

[amisevsk]

At this moment all our tooling(theia and che-machine-exec) runs in a single pod - no?

Currently yes, but the linked issue is a plan to split workspaces into multiple pods (provided RWX volumes are available) -- some thought should be given how this change would affect future plans. If JWT proxy continues to require ~128Mi of memory, we could be looking at 300-500Mi of overhead if workspaces are split.

[skabashnyuk]

@amisevsk you are right. That is a good and nice topic. However, I prefer not to mix them together because #11476 can require some time to implement it. And we would like to fix/improve what we have now already implemented.

[davidfestal]

I would say that anyway we should at least allow the JWT proxy to be run inside the POD as a default option.

[metlos]

I could make this work by moving the jwtproxy to the workspace pod and make it proxy the secure servers by resending the traffic to 127.0.0.1 and appropriate secure server port.

While this works, it has at least two consequences:

• doesn't prevent the supposedly secure servers from listening on 0.0.0.0 or any other interface/IP address which would make them accept traffic not proxied by jwtproxy
• The secure servers need to listen on 127.0.0.1 which has not been the requirement so far. In 7.8.0 and prior, a secure server merely needs to listen on the pod IP address.

This is solvable in two ways IMHO:

1. We just document that if an endpoint is secure: true, the backing server needs to listen solely on 127.0.0.1 and listening on any more IP addresses essentially enables unsecured access to the backing server.
2. We add a new attribute to the endpoint, something like private: true which tells Che to proxy access to 127.0.0.1. If such attribute was false, Che would deploy a service for the backing server and put a jwtproxy in front of that service, just like it does at the moment. Having private: false leaves the backing server open to unsecured access, which would have to be documented. We'd keep the current behavior though, so plugins/devfiles could opt in to this feature gradually.

@skabashnyuk @sleshchenko @l0rd WDYT?

[sleshchenko]

We add a new attribute to the endpoint, something like private: true which tells Che to proxy access to 127.0.0.1.

private: true does not solve an issue you described with processes which listen to 0.0.0.0

doesn't prevent the supposedly secure servers from listening on 0.0.0.0 or any other interface/IP address which would make them accept traffic not proxied by jwtproxy

right?

we add a new attribute to the endpoint, something like private: true

It's a bit confusing to have public and private attibutes at the same time, like

  attributes:
    public: true // I need URL for this endpoint
    secure: true // I need to make sure that nobody accesses my server without token with URL
    privates: true // I need to make sure that nobody accesses my server without token withing the cluster

So, my personal +1 for

We just document that if an endpoint is secure: true, the backing server needs to listen solely on 127.0.0.1 and listening on any more IP addresses essentially enables unsecured access to the backing server.

in case private: true does not help to solve issues with processes which listen to 0.0.0.0....

[l0rd]

I am +1 for solution n.1

[metlos]

Implemented in #15890, eclipse-che/che-plugin-registry#378 and eclipse-che/che-theia#626.

[metlos]

Note that the PR in che-docs that clarifies the assumptions about the secure servers is still open: eclipse-che/che-docs#1075

[nickboldt]

Based on discussion today, current status is:

• fixed/working for multiuser mode, will be in 7.9.0 release.
• broken/ not working for singleuser mode, which does not impact CRW 2.1 (no singleuser mode available)
• Once 7.9.0 is live, fix will be reverted in master and a more complete solution will be done to support both single and multiuser modes in Che 7.10, with backport (if maintenance release needed) to 7.9.1 and included in CRW2.1

[nickboldt]

This issue is "closed" and has label "in progress".

So I'm reopening it.

[metlos]

#16053 is implemented so this is now complete IMHO.