Prometheus scrape config has the wrong format

[WolfspiritM]

When using the prometheus operator configuration then the scrape config gets created in the wrong format.

The content of the secret "minio-prom-additional-scrape-config" is:

- BearerToken:  ey...
  JobName: minio-minio-job
  MetricsPath: /minio/v2/metrics/cluster
  Scheme: http
  StaticConfigs:
  - Targets:
    - minio.namespace.svc.cluster.local:80
  TLSConfig:
    CAFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

Expected Behavior

The correct format should be used. (e.g. bearer_token instead of BearerToken)

Current Behavior

Prometheus fails to start cause the scrape config is wrong.

level=error ts=2022-01-28T11:47:19.304Z caller=main.go:356 msg="Error loading config (--config.file=/etc/prometheus/config_out/prometheus.env.yaml)" err="parsing YAML file /etc/prometheus/config_out/prometheus.env.yaml: yaml: unmarshal errors:\n line 1494: field BearerToken not found in type config.ScrapeConfig\n line 1495: field JobName not found in type config.ScrapeConfig\n line 1496: field MetricsPath not found in type config.ScrapeConfig\n line 1497: field Scheme not found in type config.ScrapeConfig\n line 1498: field StaticConfigs not found in type config.ScrapeConfig\n line 1501: field TLSConfig not found in type config.ScrapeConfig"

Possible Solution

I think I found the reason.

operator/pkg/controller/cluster/main-controller.go

Line 32 in 86ec304

"sigs.k8s.io/yaml"

yaml.Marshal is from "sigs.k8s.io/yaml" which doesn't respect the struct tags!
https://go.dev/play/p/3byqOcao1je

While the "gopkg.in/yaml.v2" version does:
https://go.dev/play/p/iJyWgTr8gZa

The PR that broke it is:
#986

Steps to Reproduce (for bugs)

1. Use the prometheus operator setting
2. Look into the "minio-prom-additional-scrape-config" secret

Context

Prometheus stopped working completly

Regression

No

Your Environment

• Version used (minio-operator): 4.4.4
• Environment name and version (e.g. kubernetes v1.17.2): kubernetes v1.21.8+rke2r1
• Server type and version:
• Operating System and version (uname -a): Ubuntu
• Link to your deployment file:

[WolfspiritM]

@dvaldivia @kanagarajkm Hi. Maybe you can take a look as this currently prevents prometheus from starting at all (Crashloop)