Elastic-Agent don't support kubernetes deployment

[pguan-phl]

App: Elastic-Agent
Version: 7.10.1
Environment: Kubernetes (azure aks)
( Not sure if this is the proper repo to report this issue, but I just can't find elastic/elastic-agent. If it's not proper repo please point me to the correct one)

I'm trying to deploy docker.elastic.co/beats/elastic-agent:7.10.1 to our k8s, I provided the elastic-agent.yml from configMap, only to see errors like
Error: fail to enroll: could not save enrollment information: could not backup /tmp/config/elastic-agent.yml: rename /tmp/config/elastic-agent.yml /tmp/config/elastic-agent.yml.2021-01-24T23-02-17.4921.bak: device or resource busy
or
Error: fail to enroll: could not save enrollment information: could not backup /tmp/config/elastic-agent.yml: rename /tmp/config/elastic-agent.yml /tmp/config/elastic-agent.yml.2021-01-25T00-06-50.5729.bak: read-only file system
despite I tried mount the config to different path, with different securityContext(fsGroup:0/fsGroup:1000 etc), into folder or as file.
I'm not sure why it need to backup the config-file, but it seems to me the it is trying to "rename" instead of "copy" the config file when doing back-up. I feel it most reasonable to only READ the config file instead of WRITE to it.

[andresrc]

Hi @pguan-phl , thanks for the issue. We are still defining the recommended configuration for deploying Agent in Kubernetes.

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[pguan-phl]

@andresrc Thanks for the quick response, just FYI that I managed to work around this issue by adding a relay-volume (cp config-file form configMap to another shared volume), as below. But still, if this is solved(i.e. no WRITE action to the mounted config file) I can get rid of this uncomfortable resource definition.

      volumes:
        - name: shared-config
          emptyDir: {}
        - name: config-volume
          configMap:
            name: agent-ingest-management-config-nodescope
            defaultMode: 0666
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
      initContainers:
        - name: init
          image: docker.elastic.co/beats/elastic-agent:7.10.1
          command: ["cp"]
          args: ["/tmp/elastic-agent.yml", "/usr/share/elastic-agent/conf/elastic-agent.yml"]
          workingDir: /usr/share/elastic-agent
          volumeMounts:
            - name: config-volume
              mountPath: /tmp/elastic-agent.yml
              subPath: elastic-agent.yml
              readOnly: false
            - name: shared-config
              mountPath: /usr/share/elastic-agent/conf/
        - name: init2
          image: docker.elastic.co/beats/elastic-agent:7.10.1
          command: ["elastic-agent"]
          args: ["enroll", "$(KIBANA_HOST)", "$(FLEET_ENROLLMENT_TOKEN)", "-i", "-f", "-c", "/usr/share/elastic-agent/conf/elastic-agent.yml", "-e"]
          workingDir: /usr/share/elastic-agent
          env:
            - name: FLEET_ENROLLMENT_TOKEN
              value: 
            - name: KIBANA_HOST
              value: "http://kibana:5601"
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: shared-config
              mountPath: /usr/share/elastic-agent/conf
      containers:
        - name: agent-ingest-management-clusterscope
          image: docker.elastic.co/beats/elastic-agent:7.10.1
          command: ["elastic-agent"]
          args: ["run", "-c", "/usr/share/elastic-agent/conf/elastic-agent.yml", "-e"]
          workingDir: /usr/share/elastic-agent
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: shared-config
              mountPath: /usr/share/elastic-agent/conf

[ph]

Thanks @pguan-phl for trying it out in k8s and making it working, we have a meta issue open in the Beats repository and it would be great to have your input. I am going to close this issue lets have the discussion on the other repository.

FYI @blakerouse