Monitoring improvements for CAPO for GA

[erkanerol]

User Story

• As a giant swarm customer on OpenStack I want to have a fully working monitoring system.

Details, Background

We implemented monitoring for CAPO with this epic #640 but there are some missing elements. We need to complete them for GA.

• Monitoring vault (Partially reverting Disable vault and cluster-service rules for CAPO and CAPV prometheus-rules#314)
• Bastion monitoring (not needed for now as we decided to toggle bastions on/off on demand)
• Do we need to monitor certificate expiration / automatic rolling?
  • api server and etcd certificate is only rolled during an update
    • following upstream issue might also very helpful: Automatically renew machine certificates kubernetes-sigs/cluster-api#6529 (comment)
• Replace docker metrics with containerd metrics for CAPI #1002)
• https://github.com/giantswarm/giantswarm/issues/21653
• https://github.com/giantswarm/giantswarm/issues/22875
• https://github.com/giantswarm/giantswarm/issues/22634
• service monitor for CAP(O|I) components #1209
• review existing alerting rules before beeing GA
  - based on existing metrics, are thresholds valid and do we have an opsrecipe for paging alerts
  - which metrics are we missing
  - review alerts needed to ask customer for higher ressource quotas

Blocked by / depends on