feat: Add a version of the mixin dashboards for meta monitoring

[MichelHollands]

What this PR does / why we need it:

This builds dashboards for the meta monitoring charts based on the mixin dashboards. These are stored in a new directory under production.

The job labels are changed so they can work with all 3 modes (single binary, SSD and micro services. So instead of

job=~\"$namespace/ingester.*\"

this is used:

job=~\"$namespace/(ingester.*|(loki|enterprise-logs)-write|loki-single-binary)\"

Something similar is done for pod selectors, which are used in some dashboards.

A fix was also made for the deletion dashboard in SSD mode. This still assumed the compactor was running in the read pods instead of the backend pods.

As this isn't easy to test further changes will be made in separate PRs after this has been installed in a few places.

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[trevorwhitney]

Question: why put these in the loki repo rather than have them live in the meta-monitoring repo?

[MichelHollands]

Question: why put these in the loki repo rather than have them live in the meta-monitoring repo?

Mainly because the code was here already and because it's nice to have the base set of files in 1 place. But it can be moved.

[trevorwhitney]

approving to unblock, but I'm against this strategy long term. this is now the 3rd compiled version of the mixin we'll have in loki which I think goes against the spirit of having these defined in dynamic, configurable jsonnet recipes.

I still like the idea of doing the actual jsonnet compilation in a k8s job deployed by the helm chart. that would allow lots of customization via the values file. wdyt?

[MichelHollands]

approving to unblock, but I'm against this strategy long term. this is now the 3rd compiled version of the mixin we'll have in loki which I think goes against the spirit of having these defined in dynamic, configurable jsonnet recipes.

I still like the idea of doing the actual jsonnet compilation in a k8s job deployed by the helm chart. that would allow lots of customization via the values file. wdyt?

Thanks. I've updated this PR so it doesn't generate the new version here. That will now be done in the meta-monitoring-chart repo in this PR. That is then a base to move it to a k8s job.