Increase scope of canary deployment revision check

[zmiccar]

Flagger currently only compares the PodSpec of a deployment when checking for updates. Implementation of the check can be found here: https://github.com/weaveworks/flagger/blob/3da86fe118b5ac2830e066eaf926b9a92f45fa3e/pkg/canary/deployer.go#L135 This is limiting as there are other mutable fields in the DeploymentSpec, such as .PodTemplateSpec.metadata, .replicas, etc.

A suggestion is to compare the whole DeploymentSpec instead so that we are in control of what triggers a new revision.

While discussing this on Slack, @stefanprodan linked to a previous issue that caused the label copying to be removed. I suggest making this configurable instead since the issue described is very specific to jx.

[stefanprodan]

A remark on the replica count: Flagger needs to manipulate the number of replicas running and I don't think it will ever be included in the diff check. To control the number of replicas you have to use HPA. You set the same value for min and max replicas in the HPA if you don't need auto scaling.

[stefanprodan]

Including the PodTemplateSpec.metadata in the diff is a backwards incompatible change.

I see two options:

• implement it the under a flag that can be enabled at install time and switched on by default on the next major release
• add a field in the Canary spec so this can be enabled on per deployment basis

Regarding the propagation of the pod annotations and labels from canary to primary, this is already happening. The deployment labels and annotations are being skipped not the pod ones.