Application Level Prometheus Metrics and custom Grafana Dashboard

[J12934]

Add prometheus metrics for information application level information like:

• Number of registered teams
• Number of logins (successful, failed)
• Number of deleted instances
• Number of challenges solved (Requires Scoreboard / Trainer Dashboard #27)
• Http Response Codes (per path)[for Balancer and Proxied Requests]
• Http Request Latency from JuiceShop (per path)[for Balancer and Proxied Requests]
• Number of Requests requests proxied to the instances

The metrics endpoint should not be located at the usual /metrics but /balancer/metrics so that potential new JuiceShop Challenges(juice-shop/juice-shop#1275) are not hindered by this 😉.

A option should be added to the Helm Chart to enable / disable the endpoint.

The Helm Chart should provide a option to deploy a prometheus-operator ServiceMonitor for the /balancer/metrics endpoint.

The /balancer/metrics should also be secured using http basic auth. The credentials for it should be located in a kubernetes secrets to make it accessible to both the balancer and the ServiceMonitor.

To provide a baseline dashboard for the new metrics a grafana dashboard should be created and placed inside a specially labelled ConfigMap which can be picked up by the Grafana Sidecar: https://github.com/helm/charts/tree/master/stable/grafana#import-dashboards

[J12934]

Will be release with 3.0.0 later today 😀