Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add securityContext for KES pods in Tenant CRD #2010

Merged
merged 3 commits into from
Mar 5, 2024
Merged

add securityContext for KES pods in Tenant CRD #2010

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
19c4461
add securityContext for KES pods in Tenant CRD
pjuarezd Mar 2, 2024
9995d85
bugfix: wrong key name
pjuarezd Mar 2, 2024
69b038f
Kes not nil
pjuarezd Mar 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 28 additions & 4 deletions docs/job_crd.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
[id="{p}-api-reference"]
== API Reference

:minio-image: https://hub.docker.com/r/minio/minio/tags[minio/minio:RELEASE.2023-11-15T20-43-25Z]
:kes-image: https://hub.docker.com/r/minio/kes/tags[minio/kes:2023-11-10T10-44-28Z]
:minio-image: https://hub.docker.com/r/minio/minio/tags[minio/minio:RELEASE.2024-02-09T21-25-16Z]
:kes-image: https://hub.docker.com/r/minio/kes/tags[minio/kes:2024-01-11T13-09-29Z]


[id="{anchor_prefix}-job-min-io-v1alpha1"]
Expand Down Expand Up @@ -73,6 +73,30 @@ CommandStatus Status of MinioJob command execution
|===


[id="{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-execution"]
==== Execution (string)

Execution is the MinIO Job level execution policy

.Appears In:
****
- xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-miniojobspec[$$MinIOJobSpec$$]
****



[id="{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-failurestrategy"]
==== FailureStrategy (string)

FailureStrategy is the failure strategy at MinIO Job level

.Appears In:
****
- xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-miniojobspec[$$MinIOJobSpec$$]
****



[id="{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-miniojob"]
==== MinIOJob

Expand Down Expand Up @@ -122,10 +146,10 @@ MinIOJobSpec (`spec`) defines the configuration of a MinIOJob object. +
|*Required* +
TenantRef Reference for minio Tenant to eun the jobs against

|*`execution`* __Execution__
|*`execution`* __xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-execution[$$Execution$$]__
|Execution order of the jobs, either `parallel` or `sequential`. Defaults to `parallel` if not provided.

|*`failureStrategy`* __FailureStrategy__
|*`failureStrategy`* __xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-failurestrategy[$$FailureStrategy$$]__
|FailureStrategy is the forward plan in case of the failure of one or more MinioJob pods Either `stopOnFailure` or `continueOnFailure`, defaults to `continueOnFailure`.

|*`commands`* __xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-job-min-io-v1alpha1-commandspec[$$CommandSpec$$] array__
Expand Down
17 changes: 5 additions & 12 deletions docs/tenant_crd.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,16 +101,6 @@ CertificateStatus keeps track of all the certificates managed by the operator
|===


[id="{anchor_prefix}-github-com-minio-operator-pkg-apis-minio-min-io-v2-customcertificateconfig"]
==== CustomCertificateConfig

CustomCertificateConfig (`customCertificateConfig`) provides attributes associated of the TLS certificates manually added to the Operator as part of tenant creation. These fields contain no data if there are no custom TLS certificates.

.Appears In:
****
- xref:{anchor_prefix}-github-com-minio-operator-pkg-apis-minio-min-io-v2-customcertificates[$$CustomCertificates$$]
****



[id="{anchor_prefix}-github-com-minio-operator-pkg-apis-minio-min-io-v2-customcertificates"]
Expand Down Expand Up @@ -265,11 +255,11 @@ KESConfig (`kes`) defines the configuration of the https://github.com/minio/kes[

|*`gcpCredentialSecretName`* __string__
|*Optional* +
Specify the GCP default credentials to be used for KES to authenticate to GCP key store
Specify the GCP default credentials to be used for KES to authenticate to GCP key store

|*`gcpWorkloadIdentityPool`* __string__
|*Optional* +
Specify the name of the workload identity pool (This is required for generating service account token)
Specify the name of the workload identity pool (This is required for generating service account token)

|*`annotations`* __object (keys:string, values:string)__
|*Optional* +
Expand Down Expand Up @@ -313,6 +303,9 @@ KESConfig (`kes`) defines the configuration of the https://github.com/minio/kes[
* `runAsUser` +
* `seLinuxOptions` +

|*`containerSecurityContext`* __link:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#securitycontext-v1-core[$$SecurityContext$$]__
|Specify the https://kubernetes.io/docs/tasks/configure-pod-container/security-context/[Security Context] of MinIO KES pods.

|*`env`* __link:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#envvar-v1-core[$$EnvVar$$] array__
|*Optional* +
If provided, the MinIO Operator adds the specified environment variables when deploying the KES resource.
Expand Down
61 changes: 61 additions & 0 deletions helm/operator/templates/minio.min.io_tenants.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2048,6 +2048,67 @@ spec:
required:
- name
type: object
containerSecurityContext:
properties:
allowPrivilegeEscalation:
type: boolean
capabilities:
properties:
add:
items:
type: string
type: array
drop:
items:
type: string
type: array
type: object
privileged:
type: boolean
procMount:
type: string
readOnlyRootFilesystem:
type: boolean
runAsGroup:
format: int64
type: integer
runAsNonRoot:
type: boolean
runAsUser:
format: int64
type: integer
seLinuxOptions:
properties:
level:
type: string
role:
type: string
type:
type: string
user:
type: string
type: object
seccompProfile:
properties:
localhostProfile:
type: string
type:
type: string
required:
- type
type: object
windowsOptions:
properties:
gmsaCredentialSpec:
type: string
gmsaCredentialSpecName:
type: string
hostProcess:
type: boolean
runAsUserName:
type: string
type: object
type: object
env:
items:
properties:
Expand Down
6 changes: 6 additions & 0 deletions helm/tenant/templates/tenant.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,5 +187,11 @@ spec:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- end }}
{{- if hasKey .kes "containerSecurityContext" }}
containerSecurityContext: {{- if eq (len .kes.containerSecurityContext) 0 }} { } {{- end }}
{{- with (dig "kes" "containerSecurityContext" (dict) .) }}
{{- toYaml . | nindent 6 }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
5 changes: 5 additions & 0 deletions helm/tenant/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,11 @@ tenant:
# runAsGroup: 1000
# runAsNonRoot: true
# fsGroup: 1000
# containerSecurityContext:
# runAsUser: 1000
# runAsGroup: 1000
# runAsNonRoot: true
# allowPrivilegeEscalation: false
###
# Configures `Ingress <https://kubernetes.io/docs/concepts/services-networking/ingress/>`__ for the Tenant S3 API and Console.
#
Expand Down
2 changes: 2 additions & 0 deletions pkg/apis/minio.min.io/v2/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,8 @@ type KESConfig struct {
// * `seLinuxOptions` +
// +optional
SecurityContext *corev1.PodSecurityContext `json:"securityContext,omitempty"`
// Specify the https://kubernetes.io/docs/tasks/configure-pod-container/security-context/[Security Context] of MinIO KES pods.
ContainerSecurityContext *corev1.SecurityContext `json:"containerSecurityContext,omitempty"`
// *Optional* +
//
// If provided, the MinIO Operator adds the specified environment variables when deploying the KES resource.
Expand Down
5 changes: 5 additions & 0 deletions pkg/apis/minio.min.io/v2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions pkg/client/applyconfiguration/minio.min.io/v2/kesconfig.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions pkg/client/informers/externalversions/factory.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 44 additions & 0 deletions pkg/resources/statefulsets/kes-statefulset.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ func KESServerContainer(t *miniov2.Tenant) corev1.Container {
Args: args,
Env: KESEnvironmentVars(t),
Resources: t.Spec.KES.Resources,
SecurityContext: kesContainerSecurityContext(t),
}
}

Expand All @@ -142,6 +143,49 @@ func kesSecurityContext(t *miniov2.Tenant) *corev1.PodSecurityContext {
return &securityContext
}

// Builds the security context for kes containers
func kesContainerSecurityContext(t *miniov2.Tenant) *corev1.SecurityContext {
// Default values:
// By default, values should be totally empty if not provided
// This is specially needed in OpenShift where Security Context Constraints restrict them
// if let empty then OCP can pick the values from the constraints defined.
containerSecurityContext := corev1.SecurityContext{}
runAsNonRoot := true
var runAsUser int64 = 1000
var runAsGroup int64 = 1000
poolSCSet := false

// Values from pool.SecurityContext ONLY if provided
if t.Spec.KES != nil && t.Spec.KES.SecurityContext != nil {
if t.Spec.KES.SecurityContext.RunAsNonRoot != nil {
runAsNonRoot = *t.Spec.KES.SecurityContext.RunAsNonRoot
poolSCSet = true
}
if t.Spec.KES.SecurityContext.RunAsUser != nil {
runAsUser = *t.Spec.KES.SecurityContext.RunAsUser
poolSCSet = true
}
if t.Spec.KES.SecurityContext.RunAsGroup != nil {
runAsGroup = *t.Spec.KES.SecurityContext.RunAsGroup
poolSCSet = true
}
if poolSCSet {
// Only set values if one of above is set otherwise let it empty
containerSecurityContext = corev1.SecurityContext{
RunAsNonRoot: &runAsNonRoot,
RunAsUser: &runAsUser,
RunAsGroup: &runAsGroup,
}
}
}

// Values from kes.ContainerSecurityContext if provided
if t.Spec.KES != nil && t.Spec.KES.ContainerSecurityContext != nil {
containerSecurityContext = *t.Spec.KES.ContainerSecurityContext
}
return &containerSecurityContext
}

// NewForKES creates a new KES StatefulSet for the given Cluster.
func NewForKES(t *miniov2.Tenant, serviceName string) *appsv1.StatefulSet {
replicas := t.KESReplicas()
Expand Down
61 changes: 61 additions & 0 deletions resources/base/crds/minio.min.io_tenants.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2048,6 +2048,67 @@ spec:
required:
- name
type: object
containerSecurityContext:
properties:
allowPrivilegeEscalation:
type: boolean
capabilities:
properties:
add:
items:
type: string
type: array
drop:
items:
type: string
type: array
type: object
privileged:
type: boolean
procMount:
type: string
readOnlyRootFilesystem:
type: boolean
runAsGroup:
format: int64
type: integer
runAsNonRoot:
type: boolean
runAsUser:
format: int64
type: integer
seLinuxOptions:
properties:
level:
type: string
role:
type: string
type:
type: string
user:
type: string
type: object
seccompProfile:
properties:
localhostProfile:
type: string
type:
type: string
required:
- type
type: object
windowsOptions:
properties:
gmsaCredentialSpec:
type: string
gmsaCredentialSpecName:
type: string
hostProcess:
type: boolean
runAsUserName:
type: string
type: object
type: object
env:
items:
properties:
Expand Down
Loading