automated and ignore annotations clash

[whitlaaa]

Having a resource annotated as both automated and ignored appears to be treated as both and seems to break automation of all cluster resources.

In our case, as long as we had some HelmRelease manifests annotated with both, none of the automated resources in our cluster were getting their images updated. Note that we were seeing the latest images available (via fluxctl list-images), they were just never used.

Upon removing one of the two, everything started flowing as normal.

One of our HelmReleases looked like this:

---
apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
  name: my-webapi
  namespace: default
  labels:
    chart: webapi
  annotations:
    flux.weave.works/automated: "true"
    flux.weave.works/tag.chart-image: glob:release-*
    flux.weave.works/ignore: true
...

With both of these annotations in place, we'd have log entries that seem to indicate it being both ignored and automated:

ts=2019-02-18T22:02:34.277718708Z caller=sync.go:66 component=sync-loop resource=default:helmrelease/my-webapi ignore=apply
ts=2019-02-18T22:07:27.170619954Z caller=images.go:33 component=sync-loop error="checking services for new images: helmreleases.flux.weave.works \"my-webapi\" not found"

After doing some digging, I think this occurs due to an issue in pollForNewImages.

It looks like GetUnlockedAutomatedResources doesn't take the policy.Ignore into account, so the image check fails when comparing with a non-existent flux controller on line 33. Since that error then causes a return, it short circuits image updates for everything else.

[hiddeco]

Thanks a lot for your high quality bug report and proposed solution. 🥇

I validated the bug (valid) and your proposed solution (correct) and turned it into #1749. I am aiming to get a patch release out as soon as possible after this has been merged into master.

[2opremio]

and turned it into #1479

#1749 , for the sake of correctness. Numbers like to move :)

[whitlaaa]

Awesome! To be honest, if this was still open by the weekend I was going to use it to cut my teeth on Go testing a bit more. 😁

Thanks for verifying so quickly!

[hiddeco]

To be honest, if this was still open by the weekend I was going to use it to cut my teeth on Go testing a bit more.

Our help-wanted label is always looking for contributors 😄

[hiddeco]

@whitlaaa until we have settled and prepped for release quay.io/weaveworks/flux:master-c9efad89 is your friend.