Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically adjust Elastic Agent hostPath permissions #6599

Closed
wants to merge 11 commits into from
6 changes: 0 additions & 6 deletions config/recipes/elastic-agent/fleet-apm-integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,6 @@ spec:
spec:
serviceAccountName: fleet-server
automountServiceAccountToken: true
securityContext:
runAsUser: 0
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
Expand All @@ -102,10 +100,6 @@ spec:
mode: fleet
deployment:
replicas: 1
podTemplate:
spec:
securityContext:
runAsUser: 0
---
apiVersion: v1
kind: Service
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,6 @@ spec:
spec:
serviceAccountName: fleet-server
automountServiceAccountToken: true
securityContext:
runAsUser: 0
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
Expand All @@ -114,8 +112,6 @@ spec:
spec:
serviceAccountName: elastic-agent
automountServiceAccountToken: true
securityContext:
runAsUser: 0
containers:
- name: agent
volumeMounts:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,6 @@ spec:
spec:
serviceAccountName: fleet-server
automountServiceAccountToken: true
securityContext:
runAsUser: 0
Copy link
Contributor

@barkbay barkbay Apr 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is still required for the container's CA bundle to be updated, see:

---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
Expand All @@ -97,11 +95,7 @@ spec:
podTemplate:
spec:
serviceAccountName: elastic-agent
hostNetwork: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing hostNetwork: true?

dnsPolicy: ClusterFirstWithHostNet
automountServiceAccountToken: true
securityContext:
runAsUser: 0
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
Expand Down
2 changes: 0 additions & 2 deletions config/recipes/elastic-agent/kubernetes-integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ spec:
serviceAccountName: elastic-agent
containers:
- name: agent
securityContext:
runAsUser: 0
env:
- name: NODE_NAME
valueFrom:
Expand Down
2 changes: 0 additions & 2 deletions config/recipes/elastic-agent/multi-output.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ spec:
spec:
containers:
- name: agent
securityContext:
runAsUser: 0
volumeMounts:
- mountPath: /var/log
name: varlog
Expand Down
2 changes: 0 additions & 2 deletions config/recipes/elastic-agent/system-integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ spec:
spec:
containers:
- name: agent
securityContext:
runAsUser: 0
config:
id: 488e0b80-3634-11eb-8208-57893829af4e
revision: 2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,7 @@ spec:
version: {version}
elasticsearchRefs:
- name: quickstart
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0 <1>
daemonSet: {}
config:
inputs:
- name: system-1
Expand All @@ -62,9 +58,6 @@ spec:
period: 10s
EOF
----
+
<1> The root user is required to persist state in a hostPath volume. See <<{p}_storing_local_state_in_host_path_volume>> on how to disable this feature.
+
Check <<{p}-elastic-agent-configuration-examples>> for more ready-to-use manifests.

. Monitor the status of Elastic Agent.
Expand Down Expand Up @@ -140,11 +133,7 @@ spec:
version: {version}
elasticsearchRefs:
- name: quickstart
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0 <1>
daemonSet: {}
config:
inputs:
- name: system-1
Expand All @@ -170,8 +159,6 @@ spec:
period: 10s
----

<1> The root user is required to persist state in a hostPath volume. See <<{p}_storing_local_state_in_host_path_volume>> on how to disable this feature.

Alternatively, it can be provided through a Secret specified in the `configRef` element. The Secret must have an `agent.yml` entry with this configuration:
[source,yaml,subs="attributes,+macros"]
----
Expand All @@ -183,11 +170,7 @@ spec:
version: {version}
elasticsearchRefs:
- name: quickstart
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0
daemonSet: {}
configRef:
secretName: system-cpu-config
---
Expand Down Expand Up @@ -239,11 +222,7 @@ metadata:
name: quickstart
spec:
version: {version}
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0
daemonSet: {}
elasticsearchRefs:
- name: quickstart
outputName: default
Expand Down Expand Up @@ -285,11 +264,7 @@ metadata:
name: quickstart
spec:
version: {version}
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0
daemonSet: {}
config:
outputs:
default:
Expand All @@ -316,11 +291,7 @@ metadata:
name: quickstart
spec:
version: {version}
daemonSet:
podTemplate:
spec:
securityContext:
runAsUser: 0
daemonSet: {}
strategy:
type: RollingUpdate
rollingUpdate:
Expand Down Expand Up @@ -350,8 +321,6 @@ spec:
spec:
automountServiceAccountToken: true
serviceAccountName: elastic-agent
securityContext:
runAsUser: 0
...
---
apiVersion: rbac.authorization.k8s.io/v1
Expand Down
1 change: 1 addition & 0 deletions pkg/controller/agent/pod.go
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ func buildPodTemplate(params Params, fleetCerts *certificates.CertificatesSecret
WithDockerImage(spec.Image, container.ImageRepository(container.AgentImage, spec.Version)).
WithAutomountServiceAccountToken().
WithVolumeLikes(vols...).
WithInitContainers(maybeAgentInitContainerForHostpathVolume(spec, v)...).
WithEnv(
corev1.EnvVar{Name: "NODE_NAME", ValueFrom: &corev1.EnvVarSource{
FieldRef: &corev1.ObjectFieldSelector{
Expand Down
153 changes: 153 additions & 0 deletions pkg/controller/agent/volume.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License 2.0;
// you may not use this file except in compliance with the Elastic License 2.0.

package agent

import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"

"github.com/blang/semver/v4"

agentv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/agent/v1alpha1"
container "github.com/elastic/cloud-on-k8s/v2/pkg/controller/common/container"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/common/version"
"github.com/elastic/cloud-on-k8s/v2/pkg/utils/pointer"
)

const (
hostPathVolumeInitContainerName = "permissions"
)

var (
hostPathVolumeInitContainerResources = corev1.ResourceRequirements{
Requests: map[corev1.ResourceName]resource.Quantity{
corev1.ResourceMemory: resource.MustParse("128Mi"),
corev1.ResourceCPU: resource.MustParse("100m"),
},
Limits: map[corev1.ResourceName]resource.Quantity{
corev1.ResourceMemory: resource.MustParse("128Mi"),
corev1.ResourceCPU: resource.MustParse("100m"),
},
}
)

// maybeAgentInitContainerForHostpathVolume will return an init container that ensures that the host
// volume's permissions are sufficient for the Agent to maintain state if the Elastic Agent
// has the following attributes:
//
// 1. Agent volume is not set to emptyDir.
// 2. Agent version is above 7.15.
// 3. Agent spec is not configured to run as root.
func maybeAgentInitContainerForHostpathVolume(spec *agentv1alpha1.AgentSpec, v semver.Version) (initContainers []corev1.Container) {
// Only add initContainer to chown hostpath data volume for Agent > 7.15
if !v.GTE(version.MinFor(7, 15, 0)) {
return nil
}

image := spec.Image
if image == "" {
image = container.ImageRepository(container.AgentImage, spec.Version)
}

if !dataVolumeEmptyDir(spec) && !runningAsRoot(spec) {
initContainers = append(initContainers, corev1.Container{
Image: image,
Command: hostPathVolumeInitContainerCommand(),
Name: hostPathVolumeInitContainerName,
SecurityContext: &corev1.SecurityContext{
RunAsUser: pointer.Int64(0),
},
Resources: hostPathVolumeInitContainerResources,
VolumeMounts: []corev1.VolumeMount{
{
Name: DataVolumeName,
MountPath: DataMountPath,
},
},
})
}

return initContainers
}

// hostPathVolumeInitContainerCommand returns the container command
// for maintaining permissions for Elastic Agent.
func hostPathVolumeInitContainerCommand() []string {
return []string{
"/usr/bin/env",
"bash",
"-c",
`#!/usr/bin/env bash
set -e
if [[ -d /usr/share/elastic-agent/state ]]; then
chmod g+rw /usr/share/elastic-agent/state
chgrp 1000 /usr/share/elastic-agent/state
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 1000?

if [ -n "$(ls -A /usr/share/elastic-agent/state 2>/dev/null)" ]; then
chgrp 1000 /usr/share/elastic-agent/state/*
chmod g+rw /usr/share/elastic-agent/state/*
fi
fi
`}
}

// runningAsRoot will return true if either the Daemonset or Deployment for
// Elastic Agent has a security context set where the container will run as root.
func runningAsRoot(spec *agentv1alpha1.AgentSpec) bool {
if spec.DaemonSet != nil {
templateSpec := spec.DaemonSet.PodTemplate.Spec
if templateSpec.SecurityContext != nil &&
templateSpec.SecurityContext.RunAsUser != nil && *templateSpec.SecurityContext.RunAsUser == 0 {
return true
}
return containerRunningAsUser0(templateSpec)
}
if spec.Deployment != nil {
templateSpec := spec.Deployment.PodTemplate.Spec
if templateSpec.SecurityContext != nil &&
templateSpec.SecurityContext.RunAsUser != nil && *templateSpec.SecurityContext.RunAsUser == 0 {
return true
}
return containerRunningAsUser0(templateSpec)
}
return false
}

// containerRunningAsUser0 will return true if the Agent container
// has its pod security context set to run as root.
func containerRunningAsUser0(spec corev1.PodSpec) bool {
for _, container := range spec.Containers {
if container.Name == "agent" {
if container.SecurityContext == nil {
return false
}
if container.SecurityContext.RunAsUser != nil && *container.SecurityContext.RunAsUser == 0 {
return true
}
return false
}
}
return false
}

// dataVolumeEmptyDir will return true if either the Daemonset or Deployment for
// Elastic Agent has it's Agent volume configured for EmptyDir.
func dataVolumeEmptyDir(spec *agentv1alpha1.AgentSpec) bool {
if spec.DaemonSet != nil {
return volumeIsEmptyDir(spec.DaemonSet.PodTemplate.Spec.Volumes)
}
if spec.Deployment != nil {
return volumeIsEmptyDir(spec.Deployment.PodTemplate.Spec.Volumes)
}
return false
}

func volumeIsEmptyDir(vols []corev1.Volume) bool {
for _, vol := range vols {
if vol.Name == DataVolumeName && vol.VolumeSource.EmptyDir != nil {
return true
}
}
return false
}
Loading