[prometheus-node-exporter] Promethus chart with EKS fargate

[ngoyal16]

Describe the bug a clear and concise description of what the bug is.

We have a EKS cluster with mix of fargate and EC2 nodes. node export runs as daemonset which is working for ec2 nodes but for farget pods it is creating demonset pod but not able to schdule on farget as farget do not support daemonset..

I guess we have to have some antiaffinitiy rule for prometheus node exporter dameonset not to run on AWS farget type node.

What's your helm version?

version.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", GitTreeState:"clean", GoVersion:"go1.16.10"}

What's your kubectl version?

Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.7", GitCommit:"e6f35974b08862a23e7f4aad8e5d7f7f2de26c15", GitTreeState:"clean", BuildDate:"2022-10-12T10:57:14Z", GoVersion:"go1.18.7", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v4.5.4 Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.7-eks-fb459a0", GitCommit:"c240013134c03a740781ffa1436ba2688b50b494", GitTreeState:"clean", BuildDate:"2022-10-24T20:36:26Z", GoVersion:"go1.18.7", Compiler:"gc", Platform:"linux/amd64"}

Which chart?

prometheus-node-exporter

What's the chart version?

19.2.2.

What happened?

prometheus node exporter pods are trying to scheule on AWS EKS fargate

What you expected to happen?

As AWS EKS Farget do not support dameonset so it should not get schdule for farget type nodes

How to reproduce it?

• EKS cluster with EC2 and Fargate type node
• run a workload on both types of nodes
• install promethus in the EKS cluster

Enter the changed values of values.yaml?

NONE

Enter the command that you execute and failing/misfunctioning.

helm upgrade -i prometheus prometheus-community/prometheus -n prometheus --set alertmanager.persistence.storageClass="efs" --set server.persistentVolume.storageClass="efs"

Anything else we need to know?

N/A

[zeritti]

The Node Exporter chart supports nodeSelector which you could use to schedule the daemonset's pods on to your EC2 nodes only. If the scheduling is to be based on a distinction EC2 vs non-EC2, that would seem the simplest way forward. E.g. suppose your EC2 nodes were labelled with label ec2: true:

nodeSelector:
  ec2: true

[ngoyal16]

We need node affinity rule like

prometheus-node-exporter:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
            - key: eks.amazonaws.com/compute-type
              operator: NotIn
              values:
                - fargate

[AlissonRS]

If you are deploying through Argo CD using dependencies like below (either in Chart.yaml or requirements.yaml):

dependencies:
  - name: kube-prometheus-stack
    version: 48.4.0
    repository: https://prometheus-community.github.io/helm-charts

Then you need to put the configuration under "kube-prometheus-stack":

kube-prometheus-stack:    
  prometheus-node-exporter:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: eks.amazonaws.com/compute-type
                operator: NotIn
                values:
                  - fargate