Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for sa token volume projection in the admission charts #380

Merged
merged 10 commits into from
Feb 15, 2022
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ roleRef:
kind: ClusterRole
name: {{ include "name" . }}
subjects:
{{- if and .Values.global.virtualGarden.enabled .Values.global.virtualGarden.user.name }}
- apiGroup: rbac.authorization.k8s.io
kind: User
name: {{ .Values.global.virtualGarden.user.name }}
{{- else }}
- kind: ServiceAccount
name: {{ include "name" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
{{- if and .Values.global.virtualGarden.enabled ( not .Values.global.virtualGarden.user.name ) }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "name" . }}
namespace: {{ .Release.Namespace }}
labels:
{{ include "labels" . | indent 4 }}
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,15 @@ spec:
labels:
{{ include "labels" . | indent 8 }}
spec:
{{- if not .Values.global.virtualGarden.enabled }}
serviceAccountName: {{ include "name" . }}
rfranzke marked this conversation as resolved.
Show resolved Hide resolved
{{- else if and .Values.global.virtualGarden.enabled .Values.global.virtualGarden.user.name }}
{{- if .Values.global.serviceAccountTokenVolumeProjection.enabled }}
serviceAccountName: {{ include "name" . }}
{{- end }}
{{- end }}
{{- if .Values.global.kubeconfig }}
automountServiceAccountToken: false
{{- else }}
serviceAccountName: {{ include "name" . }}
{{- end }}
containers:
- name: {{ include "name" . }}
Expand Down Expand Up @@ -54,6 +59,11 @@ spec:
mountPath: /etc/gardener-extension-admission-gcp/kubeconfig
readOnly: true
{{- end }}
{{- if .Values.global.serviceAccountTokenVolumeProjection.enabled }}
- name: service-account-token
mountPath: /var/run/secrets/projected/serviceaccount
readOnly: true
{{- end }}
volumes:
- name: gardener-extension-admission-gcp-cert
secret:
Expand All @@ -65,3 +75,14 @@ spec:
secretName: gardener-extension-admission-gcp-kubeconfig
defaultMode: 420
{{- end }}
{{- if .Values.global.serviceAccountTokenVolumeProjection.enabled }}
- name: service-account-token
projected:
sources:
- serviceAccountToken:
path: token
expirationSeconds: {{ .Values.global.serviceAccountTokenVolumeProjection.expirationSeconds }}
{{- if .Values.global.serviceAccountTokenVolumeProjection.audience }}
audience: {{ .Values.global.serviceAccountTokenVolumeProjection.audience }}
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{{- if not .Values.global.virtualGarden.enabled }}
apiVersion: v1
rfranzke marked this conversation as resolved.
Show resolved Hide resolved
kind: ServiceAccount
metadata:
name: {{ include "name" . }}
namespace: {{ .Release.Namespace }}
labels:
{{ include "labels" . | indent 4 }}
{{- else if and .Values.global.virtualGarden.enabled .Values.global.virtualGarden.user.name }}
{{- if .Values.global.serviceAccountTokenVolumeProjection.enabled }}
apiVersion: v1
dkistner marked this conversation as resolved.
Show resolved Hide resolved
kind: ServiceAccount
metadata:
name: {{ include "name" . }}
namespace: {{ .Release.Namespace }}
labels:
{{ include "labels" . | indent 4 }}
{{- end }}
{{- end }}
7 changes: 7 additions & 0 deletions charts/gardener-extension-admission-gcp/values.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
global:
virtualGarden:
enabled: false
user:
name: ""
image:
repository: eu.gcr.io/gardener-project/gardener/extensions/admission-gcp
tag: latest
Expand Down Expand Up @@ -32,3 +34,8 @@ global:
-----END RSA PRIVATE KEY-----
# Kubeconfig to the target cluster. In-cluster configuration will be used if not specified.
kubeconfig:

serviceAccountTokenVolumeProjection:
rfranzke marked this conversation as resolved.
Show resolved Hide resolved
enabled: false
expirationSeconds: 43200
audience: ""
84 changes: 84 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Deployment of the GCP provider extension

**Disclaimer:** This document is NOT a step by step installation guide for the GCP provider extension and only contains some configuration specifics regarding the installation of different components via the helm charts residing in the GCP provider extension [repository](https://github.com/gardener/gardener-extension-provider-gcp).

## gardener-extension-admission-gcp

### Authentication against the Garden cluster
There are several authentication possibilities depending on whether or not [the concept of *Virtual Garden*](https://github.com/gardener/garden-setup#concept-the-virtual-cluster) is used.

#### *Virtual Garden* is not used, i.e., the `runtime` Garden cluster is also the `target` Garden cluster.

**Automounted Service Account Token**
The easiest way to deploy the `gardener-extension-admission-gcp` component will be to not provide `kubeconfig` at all. This way in-cluster configuration and an automounted service account token will be used. The drawback of this approach is that the automounted token will not be automatically rotated.

**Service Account Token Volume Projection**
Another solution will be to use [Service Account Token Volume Projection](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection) combined with a `kubeconfig` referencing a token file (see example below).
```yaml
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority-data: <CA-DATA>
server: https://default.kubernetes.svc.cluster.local
name: garden
contexts:
- context:
cluster: garden
user: garden
name: garden
current-context: garden
users:
- name: garden
user:
tokenFile: /var/run/secrets/projected/serviceaccount/token
```

This will allow for automatic rotation of the service account token by the `kubelet`. The configuration can be achieved by setting both `.Values.global.serviceAccountTokenVolumeProjection.enabled: true` and `.Values.global.kubeconfig` in the respective chart's `values.yaml` file.

#### *Virtual Garden* is used, i.e., the `runtime` Garden cluster is different from the `target` Garden cluster.

**Service Account**
The easiest way to setup the authentication will be to create a service account and the respective roles will be bound to this service account in the `target` cluster. Then use the generated service account token and craft a `kubeconfig` which will be used by the workload in the `runtime` cluster. This approach does not provide a solution for the rotation of the service account token. However, this setup can be achieved by setting `.Values.global.virtualGarden.enabled: true` and following these steps:

1. Deploy the `application` part of the charts in the `target` cluster.
2. Get the service account token and craft the `kubeconfig`.
3. Set the crafted `kubeconfig` and deploy the `runtime` part of the charts in the `runtime` cluster.

**Client Certificate**
Another solution will be to bind the roles in the `target` cluster to a `User` subject instead of a service account and use a client certificate for authentication. This approach does not provide a solution for the client certificate rotation. However, this setup can be achieved by setting both `.Values.global.virtualGarden.enabled: true` and `.Values.global.virtualGarden.user.name`, then following these steps:

1. Generate a client certificate for the `target` cluster for the respective user.
2. Deploy the `application` part of the charts in the `target` cluster.
3. Craft a `kubeconfig` using the already generated client certificate.
4. Set the crafted `kubeconfig` and deploy the `runtime` part of the charts in the `runtime` cluster.

**Projected Service Account Token**
This approach requires an already deployed and configured [oidc-webhook-authenticator](https://github.com/gardener/oidc-webhook-authenticator) for the `target` cluster. Also the `runtime` cluster should be registered as a trusted identity provider in the `target` cluster. Then projected service accounts tokens from the `runtime` cluster can be used to authenticate against the `target` cluster. The needed steps are as follows:

1. Deploy [OWA](https://github.com/gardener/oidc-webhook-authenticator) and establish the needed trust.
2. Set `.Values.global.virtualGarden.enabled: true` and `.Values.global.virtualGarden.user.name`. **Note:** username value will depend on the trust configuration, e.g., `<prefix>:system:serviceaccount:<namespace>:<serviceaccount>`
3. Set `.Values.global.serviceAccountTokenVolumeProjection.enabled: true` and `.Values.global.serviceAccountTokenVolumeProjection.audience`. **Note:** audience value will depend on the trust configuration, e.g., `<cliend-id-from-trust-config>`.
4. Craft a kubeconfig (see example below).
5. Deploy the `application` part of the charts in the `target` cluster.
6. Deploy the `runtime` part of the charts in the `runtime` cluster.

```yaml
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority-data: <CA-DATA>
server: https://virtual-garden.api
name: virtual-garden
contexts:
- context:
cluster: virtual-garden
user: virtual-garden
name: virtual-garden
current-context: virtual-garden
users:
- name: virtual-garden
user:
tokenFile: /var/run/secrets/projected/serviceaccount/token
```