question: sync a controller pod to physical cluster

[vincent-pli]

Just want to know if sync pod of a controller from ws to physic cluster, how could it work?
I saw KUBERNETES_SERVICE_HOST was set as hostname of upstream kcp cluster rather than the url of that ws.

[ncdc]

@vincent-pli kcp does not include the pods resource by default. You could create a SyncTarget and syncer (see https://github.com/kcp-dev/kcp/blob/main/docs/syncer.md) and configure the syncer to sync pods. This would result in the pods resource being added to kcp for anyone binding to the syncer's auto-created kubernetes APIExport. At this point, you could create a Pod in your workspace and it would be synced...

However 😄

I'm curious what your use case is for syncing pods?

[sttts]

Pods will come to kcp soon via inverse syncing. But then the source of truth is the physical cluster. As Andy writes, am curious too whether you have a usecase in mind where kcp should be the source of thruth.

Background: technically kcp might support that: PVs will probably be like that. They are inversely synced to kcp, and a kcp coordination controller for storage might decide to "disconnect" a PV from a physical cluster and attach it on another physical cluster. In that case, for a period of time kcp will the source of thruth.

[vincent-pli]

Thanks @sttts and @ncdc
I'm still trying figure out the design philosophy of KCP. I mean for example: creating a controller in workspace, then the deploy will be sync to physic cluster and parameters like KUBERNETES_SERVICE_HOST, token of SA should be rewrite to let the controller talk to ws' api endpoint, correct me, thanks.

So with the assumption, if the controller is tekton or argo workflow, the controller will create pods in it's reconcile, so we have to sync pod from ws to physic cluster.

As I mentioned, I still not figure out the overview design of kcp, I saw there is project: https://github.com/kcp-dev/controller-runtime-example, so maybe kcp will treat controller as shared functionality provider or whatever, will shared by all user/workspace..., just my guess, need clarification, thanks

[ncdc]

You are correct - when you create a Deployment in a kcp Workspace and it gets synced to a physical cluster via a SyncTarget/the syncer, there is a translation that happens. The resulting Deployment in the physical cluster is modified so the "in-cluster config" that uses KUBERNETES_SERVICE_HOST points back to the kcp apiserver.

Many controllers should be able to operate ok in this manner. They primarily manipulate CRs for their own CRDs, and maybe other things like ConfigMaps and Secrets.

Then there is a class of controllers that need to manipulate things like Pods and Services. Tekton and Argo Worklflows fall into this class, and we aren't really at a point yet where we have full explorations and examples of how to use them with kcp.