Extend canary analysis with external checks

[stefanprodan]

The canary analysis could be extended with webhooks. Flagger would call a URL and determine from the response status code if the canary is failing or not.

CRD spec:

kind: Canary
metadata:
  name: podinfo
  namespace: prod
spec:
  canaryAnalysis:
    webhooks:
    - name: check1
      url: http://validator.prod.svc.cluster.local:9898/check
      timeout: 10s
      # arbitrary key-value pairs 
      metadata:
        key1: "value1"
        key2: "value2"

Webhook payload (HTTP POST JSON):

{
    "name": "podinfo",
    "namespace": "prod", 
    "metadata": {
        "key1":  "value1",
        "key2":  "value2"
    }
}

Response status codes:

• 200-202 - advance canary by increasing the traffic weight
• timeout or non-2xx - halt advancement and increment failed checks

On a non-2xx response include the the response body (if any) in the failed checks log & event.