[BUG] Invalid value for resources results in misleading error message.

[AndrzejBaldys]

System info:

• OS: [e.g. Linux? MaxOS? Windows?]
• Linux
• KubeLinter v0.6.2-0-g191de10fd5

Describe the bug
We are using Terraform provider to deploy our charts. This allows us to provide some values at deployment time (usually infrastructure resources IDs etc.). Recently we've added alerts for pods resource consumption. To be DRY we've moved real values for resources to Terraform files from which they can be used for chart deployments and as alerts thresholds. To be able to template chart we still needed to have some values in resources so we've used "DEPLOY_TIME_VARIABLE" like we do for other deployment time values. After this change kube-linter is failing with error message: .../my-service/templates/service.yaml: (object: <no namespace>/my-service-api-svc /v1, Kind=Service) no pods found matching service labels (map[app:myservice]) (check: dangling-service, remediation: Confirm that your service's selector correctly matches the labels on one of your deployments.) which seems to be not valid as the only change that was done was to values of the resources.

From:

# ./values.yaml

#...
app:
  replicaCount: 3
  resources:
    requests:
      cpu: 15m
      memory: 160Mi
    limits:
      cpu: 60m
      memory: 640Mi

#...

to:

# ./values.yaml

#...
app:
  replicaCount: 3
  resources:
    requests:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
    limits:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
#...

Sample YAML input

# ./Chart.yaml

apiVersion: v0.1
name: app-configuration-proxy
description: App Configuration Proxy Service
type: application
version: __RELEASE_TAG__
appVersion: __RELEASE_TAG__

# ./values.yaml

image:
  registry: __ACR_NAME__
  repository: app-configuration-proxy
  tag: __RELEASE_TAG__

app:
  replicaCount: 3
  resources:
    requests:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
    limits:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"

identity:
  type: 0
  resourceId: "DEPLOY_TIME_VARIABLE"
  clientId: "DEPLOY_TIME_VARIABLE"
  serviceAccountName: "DEPLOY_TIME_VARIABLE"

azureAd:
  tenantId: "DEPLOY_TIME_VARIABLE"
  domain: "DEPLOY_TIME_VARIABLE"
  clientId: "DEPLOY_TIME_VARIABLE"
  swaggerclientId: "DEPLOY_TIME_VARIABLE"
  groups:
    deploymentSlot: "DEPLOY_TIME_VARIABLE"
  allowToBeAuthorizedByAcl: "true"

keyvaultName: "DEPLOY_TIME_VARIABLE"

ingress:
  pathPrefix: "/app-configuration-proxy-service"
  defaultHost:
    dnsname: "DEPLOY_TIME_VARIABLE"
    defaultSecretName: "DEPLOY_TIME_VARIABLE"
  sitespecificHosts:
    enabled: false
    dnsZoneName: "DEPLOY_TIME_VARIABLE"
    sitecodes: ["DEPLOY_TIME_VARIABLE"]
    siteSpecificSecretName: "DEPLOY_TIME_VARIABLE"
  wafPolicyId: "DEPLOY_TIME_VARIABLE"

appInfo:
  applicationName: "DEPLOY_TIME_VARIABLE"
  deploymentStampName: "DEPLOY_TIME_VARIABLE"
  slotName: "DEPLOY_TIME_VARIABLE"
  serviceName: "DEPLOY_TIME_VARIABLE"
  environment: "DEPLOY_TIME_VARIABLE"
  component: "DEPLOY_TIME_VARIABLE"

azureAppConfigEndpoint: "DEPLOY_TIME_VARIABLE"

exposeSwagger: "EXPOSE_SWAGGER"
swaggerDescription: __SWAGGER_DESCRIPTION__
swaggerContactEmail: __SWAGGER_CONTACT_EMAIL__

applicationInsights:
  logLevel:
    default: __ApplicationInsights_LogLevel_Default__

logging:
  logLevel:
    default: __Logging_LogLevel_Default__

secretstore:
  deploymentStamp:
    keyvaultName: kv-__APP_NAME__-app-__ENVIRONMENT__
    secrets:
      - kvName: "auto-acpx-sp-client-secret"
        envName: "AzureAd__Secret"

defaultDomain:
  dtlsKeyvaultCertName: my-cert-name

sitespecificDomain:
  enabled: false
  siteTlsKeyvaultCertName: my-cert-name

# ./templates/_helpers.tpl

{{- define "app-configuration-proxy-service.name" -}}
app-configuration-proxy
{{- end }}

{{- define "app-configuration-proxy-service.fullname" -}}
{{ include "app-configuration-proxy-service.name" . }}
{{- end }}

{{- define "app-configuration-proxy-service.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{- define "app-configuration-proxy-service.labels" -}}
helm.sh/chart: {{ include "app-configuration-proxy-service.chart" . }}
{{ include "app-configuration-proxy-service.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{- define "app-configuration-proxy-service.selectorLabels" -}}
app.kubernetes.io/name: {{ include "app-configuration-proxy-service.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{- define "app-configuration-proxy-service.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "app-configuration-proxy-service.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

# ./templates/deployment-stamp-kv-secrets.yaml

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secretproviderclass"
spec:
  provider: azure
  secretObjects:
  - data:
      {{- range .Values.secretstore.deploymentStamp.secrets }}
      - objectName: {{ .kvName }}
        key: {{ .kvName }}
      {{- end }}
    secretName: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secrets"
    type: Opaque

  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "false"
    clientID: "{{ .Values.identity.clientId }}"
    keyvaultName: "{{ .Values.secretstore.deploymentStamp.keyvaultName }}"
    cloudName: ""
    objects:  |
      array:
        {{- range .Values.secretstore.deploymentStamp.secrets }}
        - |
          objectName: {{ .kvName }}
          objectType: secret
          objectVersion: ""
        {{- end }}
    tenantId: "{{ .Values.azureAd.tenantId }}"

# ./templates/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-deployment
  labels:
    app: appconfigurationproxyservice
    {{- include "app-configuration-proxy-service.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.app.replicaCount }}
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: appconfigurationproxyservice
  template:
    metadata:
      labels:
        app: appconfigurationproxyservice
        azure.workload.identity/use: "true"
        {{- include "app-configuration-proxy-service.labels" . | nindent 8 }}
    spec:
      serviceAccountName: {{ .Values.identity.serviceAccountName }}
      securityContext:
        runAsUser: 1000
        runAsGroup: 2000
        fsGroup: 2000
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - appconfigurationproxyservice
              topologyKey: kubernetes.io/hostname
      containers:
      - name: app-configuration-proxy
        image: {{ .Values.image.registry }}.azurecr.io/{{ .Values.image.repository }}:{{ .Values.image.tag }}
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
        resources:
          requests:
            cpu: {{ .Values.app.resources.requests.cpu }}
            memory: {{ .Values.app.resources.requests.memory }}
          limits:
            cpu: {{ .Values.app.resources.limits.cpu }}
            memory: {{ .Values.app.resources.limits.memory }}
        ports:
        - containerPort: 8080
        envFrom:
        - secretRef:
            name: {{ include "app-configuration-proxy-service.fullname" . }}-azuread
        env:
          - name: PathPrefix
            value: {{ .Values.ingress.pathPrefix }}
          - name: ExposeSwagger
            value: {{ .Values.exposeSwagger | quote }}
          - name: SwaggerDescription
            value: {{ .Values.swaggerDescription | quote }}
          - name: SwaggerContactEmail
            value: {{ .Values.swaggerContactEmail | quote }}
          - name: APPINSIGHTS_INSTRUMENTATIONKEY
            value: {{ .Values.app_insights_instrumentation_key }}
          - name: AppInfo__ApplicationName
            value: {{ .Values.appInfo.applicationName }}
          - name: AppInfo__DeploymentStampName
            value: {{ .Values.appInfo.deploymentStampName }}
          - name: AppInfo__SlotName
            value: {{ .Values.appInfo.slotName }}
          - name: AppInfo__ServiceName
            value: {{ .Values.appInfo.serviceName }}
          - name: AppInfo__Environment
            value: {{ .Values.appInfo.environment }}
          - name: AppInfo__Component
            value: {{ .Values.appInfo.component }}
          - name: AzureAppConfigEndpoint
            value: {{ .Values.azureAppConfigEndpoint }}
          {{- range .Values.secretstore.deploymentStamp.secrets}}
          - name: {{ .envName}}
            valueFrom:
              secretKeyRef:
                name: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secrets"
                key: {{ .kvName}}
          {{- end }}
        volumeMounts:
          - name: secrets-store-stamp
            mountPath: /mnt/secrets-stamp
            readOnly: true
          - name: tmp
            mountPath: /tmp
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          periodSeconds: 3
          timeoutSeconds: 1
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          periodSeconds: 3
          timeoutSeconds: 1
      volumes:
      - name: secrets-store-stamp
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secretproviderclass"
      - name: tmp
        emptyDir: {}

# ./templates/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-api-ingress
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-path-prefix: "/"
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    appgw.ingress.kubernetes.io/waf-policy-for-path: "{{ .Values.ingress.wafPolicyId }}"
spec:
  rules:
  - host: "{{ .Values.ingress.defaultHost.dnsname }}"
    http:
      paths:
      - path: "{{ .Values.ingress.pathPrefix }}/*"
        pathType: ImplementationSpecific
        backend:
          service:
            name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
            port:
              number: 80
  {{- if .Values.ingress.sitespecificHosts.enabled }}
  {{- range .Values.ingress.sitespecificHosts.sitecodes }}
  - host: "{{ . }}.{{ $.Values.ingress.sitespecificHosts.dnsZoneName }}"
    http:
      paths:
        - path: "{{ $.Values.ingress.pathPrefix }}/*"
          pathType: ImplementationSpecific
          backend:
            service:
              name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
              port:
                number: 80
  {{- end }}
  {{- end }}
  tls:
  - hosts:
    - "{{ .Values.ingress.defaultHost.dnsname }}"
    secretName: {{ .Values.ingress.defaultHost.defaultSecretName }}
  {{- if .Values.ingress.sitespecificHosts.enabled }}
  - hosts:
    {{- range .Values.ingress.sitespecificHosts.sitecodes }}
    - "{{ . }}.{{ $.Values.ingress.sitespecificHosts.dnsZoneName }}"
    {{- end }}
    secretName: {{ .Values.ingress.sitespecificHosts.siteSpecificSecretName }}
  {{- end }}

# ./templates/secrets.yaml

apiVersion: v1
kind: Secret
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-azuread
type: Opaque
data:
  AzureAd__Instance: {{ printf "https://login.microsoftonline.com" | b64enc | quote }}
  AzureAd__TenantId: {{ .Values.azureAd.tenantId | b64enc | quote }}
  AzureAd__Domain: {{ .Values.azureAd.domain | b64enc | quote }}
  AzureAd__ClientId: {{ .Values.azureAd.clientId | b64enc | quote }}
  AzureAd__Groups__DeploymentSlot: {{ .Values.azureAd.groups.deploymentSlot | b64enc | quote }}
  AzureAd__AllowWebApiToBeAuthorizedByACL: {{ .Values.azureAd.allowToBeAuthorizedByAcl | b64enc | quote }}
  SwaggerUI__ClientId: {{ .Values.azureAd.swaggerclientId | b64enc | quote }}

# ./templates/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
spec:
  selector:
    app: appconfigurationproxyservice
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

# ./templates/serviceaccounts.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: {{ .Values.identity.clientId }}
  labels:
    azure.workload.identity/use: "true"
  name: {{ .Values.identity.serviceAccountName }}
automountServiceAccountToken: true

Expected behavior

• Error messages which would clearly indicate root-cause of the issue.
• It should be possible to configure linter to accept any value for resources including placeholders and only warn user if it cannot check if the value seems to be reasonable but it should not count it as error.

[SlevinWasAlreadyTaken]

I faced the same problem (using version 0.6.4 - currently latest). Here is a more minimalist way to reproduce it.

cat << EOF > test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: my_namespace_name
  name: my_deployment_name
  labels:
    name: my_label_value
spec:
  selector:
    matchLabels:
      name: my_label_value
  template:
    metadata:
      labels:
        name: my_label_value
    spec:
      containers:
        - name: my_container_name
          image: eclipse-temurin:17-jdk-jammy
          resources:
            limits:
              cpu: 10wrong_unit
              memory: 10Mi
            requests:
              cpu: 10m
              memory: 10Mi
          ports:
          - containerPort: 8080
            protocol: TCP
            name: http
          - containerPort: 9090
            protocol: TCP
            name: metrics
          securityContext:
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 1000
---
apiVersion: v1
kind: Service
metadata:
  namespace: my_namespace_name
  name: my_service_name
  labels:
    name: my_label_value
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 8080
    targetPort: http
    protocol: TCP
  - name: metrics
    port: 9090
    targetPort: metrics
    protocol: TCP
  selector:
    name: my_label_value

EOF
kube-linter lint test.yaml

Result:

test.yaml: (object: my_namespace_name/my_service_name /v1, Kind=Service) no pods found matching service labels (map[name:my_label_value]) (check: dangling-service, remediation: Confirm that your service's selector correctly matches the labels on one of your deployments.)

You can fix the warning by changing cpu: 10wrong_unit to cpu: 10m.