PodTemplate not found

[salaxander]

Issue seen by user of the AKS managed addon:

{
  "level": "error",
  "ts": 1694539328.40113,
  "msg": "Reconciler error",
  "controller": "imagejob-controller",
  "object": {
    "name": "imagejob-slchb",
    "namespace": "kube-system"
  },
  "namespace": "kube-system",
  "name": "imagejob-slchb",
  "reconcileID": "4caba556-ab6c-431a-81c8-36a3aa79cbf9",
  "error": "reconcile running: PodTemplate \"imagejob-slchb\" not found",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235"
}

Started immediately after startup of the Eraser controller.

Environment:

• Eraser version: v1.2.1
• Kubernetes version: (use kubectl version): v1.25.11

[pargall]

Saw the same issue on my AKS clusters starting around 9/12, running the same versions mentioned above. I found restarting the deployment(kubectl -n kube-system rollout restart deployment/eraser-controller-manager) mitigated the issue for me.

[mgacuzana]

Restarting the controller deployment enabled the new controller to kick off a new ImageJob.

I still see the same error in the logs, for the exact same ImageJob as the previous controller (imagejob-slchb), so it doesn't appear to completely mitigate the problem.