Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change error handling when scraping metrics #551

Merged
merged 1 commit into from
Dec 6, 2021
Merged

Commits on Dec 6, 2021

  1. Change error handling when scraping metrics

    * If Envoy returns an error then also respond with a 500 in our merged
    metrics response so that Prometheus will know that we had an error, not
    that there are no metrics.
    * If the service metrics return with a non-2xx status code then don't
    include the response body in the merged metrics. This will stop issues
    where users accidentally turn on metrics merging but they don't have an
    exporter and so their metrics endpoint returns 404. I could have
    responded with a 500 in this case in order to indicate that there is an
    error, however I think it's more likely that users are accidentally
    turning on metrics merging and the error indication is accomplished via
    a new metric (see below).
    * Append a new metric that indicates the success of the service
    scraping. This can be used for alerting by users since the response code
    of the service metrics response is discarded:
      * success: consul_metrics_merging_service_metrics_success 1
      * fail: consul_metrics_merging_service_metrics_success 0
    * modify logging to use key/value pairs
    * Fixes #546
    lkysow committed Dec 6, 2021
    Configuration menu
    Copy the full SHA
    4b9963e View commit details
    Browse the repository at this point in the history