Expose operator metrics

[janhoy]

Users may want to monitor the operator health. See https://book.kubebuilder.io/reference/metrics.html for some docs about publishing Prometheus metrics.

No idea what metrics are most useful. Some counters for various operations and errors. Perhaps a gauge for number of SolrClouds deployed?

[HoustonPutman]

I think this would be a wonderful addition. Let's target it for the next version (v0.5.0), that should come shortly after v0.4.0.

A good start would just be making it easy to listen on the default operator metrics. We can then add other various metrics that we think would be useful.

[HoustonPutman]

There is very good default support for Operator metrics in Kubebuilder 3. Now that the operator uses Kubebuilder 3, we should investigate adding the metrics service & sidecar to the helm chart.

[HoustonPutman]

I think we can close this with the default operator metrics to start, since they are actually quite comprehensive. (including reconcile errors) We can split off another issue to look into custom metrics.