Prevent scaling immediately after API creation

[deliahu]

Description

Deploying a model and then updating it (e.g. setting compute.cpu to 400m) causes the horizontal pod autoscaler to scale the deployment to 2+ replicas. The same thing happens when running cortex delete and cortex deploy shortly after.

This may be due to the high CPU utilization during pod initialization, although the algorithm seems to try to account for this.

Possible flags:
--horizontal-pod-autoscaler-cpu-initialization-period
--horizontal-pod-autoscaler-initial-readiness-delay

It may also be due to a bug in the metrics server

Potentially related issues (although the problem here doesn't seem to be related to certs):

• Metrics Server problems kubernetes/kubernetes#67702
• no metrics known for pod kubernetes-sigs/metrics-server#237
• Unable to get CPU for container discarding data: missing cpu usage metric kubernetes-sigs/metrics-server#207
• Unable to fetch pod metrics & request failed - "401 Unauthorized" kubernetes-sigs/metrics-server#212