How to add labels and annotations to all pods of a cluster?

[Pieerot]

I'm trying write a custom addon.
Now I want to add some labels and annotations to all pods of database cluster.
For example, I want to deploy two database engines A and B. I hope all pods of A have a label of "mylabels/grouplabel=A", and all pods of B have a label of "mylabels/grouplabel=B". Similarly, all pods of A have a annotations of "myannouncements/groupannouncements=A", and all pods of B have a annotations of "myannouncements/groupannouncements=B"
How can i achieve this?

[shanshanying]

It depends on which version of KubeBlocks you are working with.

Prior to Release 0.8

say you are working with ClusterDefinition
you can put your customized lable to clusterdefinition.spec.componentDefs.customLabelSpecs

 k explain clusterdefinition.spec.componentDefs.customLabelSpecs 
GROUP:      apps.kubeblocks.io
KIND:       ClusterDefinition
VERSION:    v1alpha1

FIELD: customLabelSpecs <[]Object>

DESCRIPTION:
    customLabelSpecs is used for custom label tags which you want to add to the
    component resources.

FIELDS:
  key	<string> -required-
    key name of label

  resources	<[]Object>
    resources defines the resources to be labeled.

  value	<string> -required-
    value of label

An example could be found at: https://github.com/apecloud/kubeblocks-addons/blob/ae54e3901285f297ee0d3fcf0c4821f2fcfde8d7/addons/postgresql/templates/clusterdefinition.yaml#L19-L20

  componentDefs:
    - name: postgresql
      workloadType: Replication
      characterType: postgresql
      customLabelSpecs:
        - key: apps.kubeblocks.postgres.patroni/scope
          value: "$(KB_CLUSTER_NAME)-$(KB_COMP_NAME)-patroni$(KB_CLUSTER_UID_POSTFIX_8)"
          resources:
            - gvk: "v1/Pod"
              selector:
                app.kubernetes.io/managed-by: kubeblocks
            - gvk: "apps/v1/StatefulSet"
              selector:
                app.kubernetes.io/managed-by: kubeblocks

Release 0.8 and above

KubeBlocks supports Component, and you can add a cusztomied label using API cmpd.spec.labels

k explain cmpd.spec.labels
GROUP:      apps.kubeblocks.io
KIND:       ComponentDefinition
VERSION:    v1alpha1

FIELD: labels <map[string]string>

DESCRIPTION:
    Specifies static labels that will be patched to all Kubernetes resources
    created for the Component.
     Note: If a label key in the `labels` field conflicts with any system labels
    or user-specified labels, it will be silently ignored to avoid overriding
    higher-priority labels.
     This field is immutable.

An example could be found in Oceanbase ComponentDefinition:
https://github.com/apecloud/kubeblocks-addons/blob/6c83b627a9d93d4bbd30848dea9f053bc3f73810/addons/oceanbase-ce/templates/componentdefinition.yaml#L354

    labels: &oblabels
      kubeblocks.io/ready-without-primary: "true"

And likewise, you can customize you annotaion using cmpd.spec.annotations API,

k explain cmpd.spec.annotations                                             
GROUP:      apps.kubeblocks.io
KIND:       ComponentDefinition
VERSION:    v1alpha1

FIELD: annotations <map[string]string>

DESCRIPTION:
    Specifies static annotations that will be patched to all Kubernetes
    resources created for the Component.
     Note: If an annotation key in the `annotations` field conflicts with any
    system annotations or user-specified annotations, it will be silently
    ignored to avoid overriding higher-priority annotations.
     This field is immutable.

[Pieerot]

Could you tell us why it is necessary to label and annotate Pods when creating a Cluster? It would be great if you could provide some examples. We hope to improve the API design based on these scenarios. Thank you very much.

I have other operators who will filter pods based on custom labels and do some thing, so I hope to specify the value of custom labels in the cluster scope when creating a cluster to distinguish them. When I am not using Kubeblocks, most of the time I manually specify custom labels and values in helm values.yaml when deploying a new database cluster.
Thank you for your concern. I will wait for the release of v0.9 version.

[weicao]

I see. Actually, KubeBlocks labels the pods it creates with a set of pre-defined lables such as "app.kubernetes.io/managed-by: kubeblocks", "app.kubernetes.io/instance: ${clusterName}", "apps.kubeblocks.io/component-name: ${dbEngineName}".
Do these labels work for you?

[weicao]

Here is an example:

$ kubectl get pod osier48-mongodb-0 -o yaml
apiVersion: v1
kind: Pod
metadata:
...
labels:
app.kubernetes.io/component: mongodb
app.kubernetes.io/instance: osier48
app.kubernetes.io/managed-by: kubeblocks
app.kubernetes.io/name: mongodb
app.kubernetes.io/version: ""
apps.kubeblocks.io/cluster-uid: 53745b1c-6f4a-4988-b8c2-6ca08f9a0e89
apps.kubeblocks.io/component-name: mongodb
clusterdefinition.kubeblocks.io/name: mongodb
clusterversion.kubeblocks.io/name: mongodb-5.0
controller-revision-hash: osier48-mongodb-8648bdc789
kubeblocks.io/role: secondary
rsm.workloads.kubeblocks.io/access-mode: Readonly
statefulset.kubernetes.io/pod-name: osier48-mongodb-0

[Pieerot]

I see. Actually, KubeBlocks labels the pods it creates with a set of pre-defined lables such as "app.kubernetes.io/managed-by: kubeblocks", "app.kubernetes.io/instance: ${clusterName}", "apps.kubeblocks.io/component-name: ${dbEngineName}". Do these labels work for you?

I have noticed the pre-defined labels in Kubeblocks. These will be very useful for subsequent development and maintenance. But in my case, I have already developed other operators and defined some custom label and annotations, so I could not use these pre-defined labels.

[weicao]

Got it, here is a pr(#7593 ) to set user-provided labels on all Pods in a Cluster.
It would be merged to both 0.8.4 and 0.9 releases.