[registry-facade] Support handover daemonSet

[csweichel]

For the time being daemonSet's do not support surge rollouts. To make use of registry-facade's newfound handover capability, we need to deploy a "handover" daemonSet alongside registry facade. This daemonSet is:

• a helm chart under dev/charts/registryFacadeHandover, maybe a single kubernetes resource that can be used with kubectl apply -f suffices
• relies on roles and PSPs from the Gitpod helm chart
• just starts registry-facade handover

During a deployment that requires continuous operation (e.g. gitpod.io) we'd deploy this chart first, then do the update, and remove the handover daemonSet again.

[csweichel]

The handover mechanism itself works, but it requires registry-facade to run with hostNetwork: true. In that case we cannot access ws-manager anymore to ask for the image spec. Possible ways forward are:

• revive the ws-daemon TCP proxy//etc/hosts trick that we originally built for registry-facade itself, except now use it for ws-manager
• push the config to all registry-facade's by talking to them directly akin to talking to ws-daemon (incl. the same problems [ws-daemon] Offer services not just on a nodePort #2956). At least we could push before the pull happens, simply by pushing before creating the pod

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.