ci: enable helm-loki-ci with helm diff

[fcjack]

What this PR does / why we need it:

The previous helm-loki-ci was disabled and it is hard to check the changes and validate the code review for helm chart implementations.

Trying to make more clear what is happening when implementing a change inside the helm chart, this new workflow will post the changes in different scenarios comparing the current helm chart with the latest released version.

Which issue(s) this PR fixes:

Fixes https://github.com/grafana/loki-private/issues/1304

Special notes for your reviewer:

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
  • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[poyzannur]

Writing down notes from out pairing session

This is a much nicer solution than trying to find the diff in github-action logs! TY!
We are trying to get collapsable markdown to work to reduce the scrolling across different scenarios.
We will add a scenario for distributed chart, which can be done in a follow up PR.

[github-actions]

Helm Diff Output - Summary

Single Binary Scenario

Default Values Scenario

default, loki-backend, StatefulSet (apps) has changed:
  # Source: loki/templates/backend/statefulset-backend.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-backend
    namespace: default
    labels:
      helm.sh/chart: loki-6.24.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.3.2"
      app.kubernetes.io/component: backend
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-backend-headless
    revisionHistoryLimit: 10
    
    persistentVolumeClaimRetentionPolicy:
      whenDeleted: Delete
      whenScaled: Delete
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: backend
    template:
      metadata:
        annotations:
          checksum/config: 6074dc4b0d60af4991bb01fbda4550e5e2da5dd9c203362200c280b3e43407ea
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: backend
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki
        automountServiceAccountToken: true
        
        securityContext:
          fsGroup: 10001
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: loki-sc-rules
-           image: "kiwigrid/k8s-sidecar:1.28.0"
+           image: "kiwigrid/k8s-sidecar:1.29.0"
            imagePullPolicy: IfNotPresent
            env:
              - name: METHOD
                value: WATCH
              - name: LABEL
                value: "loki_rule"
              - name: FOLDER
                value: "/rules"
              - name: RESOURCE
                value: "both"
              - name: WATCH_SERVER_TIMEOUT
                value: "60"
              - name: WATCH_CLIENT_TIMEOUT
                value: "60"
              - name: LOG_LEVEL
                value: "INFO"
            volumeMounts:
              - name: sc-rules-volume
                mountPath: "/rules"
          - name: loki
            image: docker.io/grafana/loki:3.3.2
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=backend
              - -legacy-read-mode=false
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 30
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: tmp
                mountPath: /tmp
              - name: data
                mountPath: /var/loki
              - name: sc-rules-volume
                mountPath: "/rules"
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: backend
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: tmp
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: sc-rules-volume
            emptyDir: {}
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: "10Gi"

Ingress Values Scenario

default, loki-backend, StatefulSet (apps) has changed:
  # Source: loki/templates/backend/statefulset-backend.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-backend
    namespace: default
    labels:
      helm.sh/chart: loki-6.24.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.3.2"
      app.kubernetes.io/component: backend
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-backend-headless
    revisionHistoryLimit: 10
    
    persistentVolumeClaimRetentionPolicy:
      whenDeleted: Delete
      whenScaled: Delete
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: backend
    template:
      metadata:
        annotations:
          checksum/config: 6074dc4b0d60af4991bb01fbda4550e5e2da5dd9c203362200c280b3e43407ea
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: backend
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki
        automountServiceAccountToken: true
        
        securityContext:
          fsGroup: 10001
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: loki-sc-rules
-           image: "kiwigrid/k8s-sidecar:1.28.0"
+           image: "kiwigrid/k8s-sidecar:1.29.0"
            imagePullPolicy: IfNotPresent
            env:
              - name: METHOD
                value: WATCH
              - name: LABEL
                value: "loki_rule"
              - name: FOLDER
                value: "/rules"
              - name: RESOURCE
                value: "both"
              - name: WATCH_SERVER_TIMEOUT
                value: "60"
              - name: WATCH_CLIENT_TIMEOUT
                value: "60"
              - name: LOG_LEVEL
                value: "INFO"
            volumeMounts:
              - name: sc-rules-volume
                mountPath: "/rules"
          - name: loki
            image: docker.io/grafana/loki:3.3.2
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=backend
              - -legacy-read-mode=false
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 30
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: tmp
                mountPath: /tmp
              - name: data
                mountPath: /var/loki
              - name: sc-rules-volume
                mountPath: "/rules"
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: backend
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: tmp
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: sc-rules-volume
            emptyDir: {}
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: "10Gi"

Legacy Monitoring Scenario

default, loki-backend, StatefulSet (apps) has changed:
  # Source: loki/templates/backend/statefulset-backend.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-backend
    namespace: default
    labels:
      helm.sh/chart: loki-6.24.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.3.2"
      app.kubernetes.io/component: backend
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-backend-headless
    revisionHistoryLimit: 10
    
    persistentVolumeClaimRetentionPolicy:
      whenDeleted: Delete
      whenScaled: Delete
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: backend
    template:
      metadata:
        annotations:
          checksum/config: 6074dc4b0d60af4991bb01fbda4550e5e2da5dd9c203362200c280b3e43407ea
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: backend
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki
        automountServiceAccountToken: true
        
        securityContext:
          fsGroup: 10001
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: loki-sc-rules
-           image: "kiwigrid/k8s-sidecar:1.28.0"
+           image: "kiwigrid/k8s-sidecar:1.29.0"
            imagePullPolicy: IfNotPresent
            env:
              - name: METHOD
                value: WATCH
              - name: LABEL
                value: "loki_rule"
              - name: FOLDER
                value: "/rules"
              - name: RESOURCE
                value: "both"
              - name: WATCH_SERVER_TIMEOUT
                value: "60"
              - name: WATCH_CLIENT_TIMEOUT
                value: "60"
              - name: LOG_LEVEL
                value: "INFO"
            volumeMounts:
              - name: sc-rules-volume
                mountPath: "/rules"
          - name: loki
            image: docker.io/grafana/loki:3.3.2
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=backend
              - -legacy-read-mode=false
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 30
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: tmp
                mountPath: /tmp
              - name: data
                mountPath: /var/loki
              - name: sc-rules-volume
                mountPath: "/rules"
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: backend
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: tmp
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: sc-rules-volume
            emptyDir: {}
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: "10Gi"

Simple Scalable AWS Kube IRSA Scenario

1145c1145
<           image: "kiwigrid/k8s-sidecar:1.29.0"
---
>           image: "kiwigrid/k8s-sidecar:1.28.0"

[poyzannur]

LGTM!