rotator.AddRotator doesn't exit when the process is terminated

[karlkfi]

If you use rotator.AddRotator to build a rotator.ReconcileWH and add it to the controller manager, it uses a context.Background() which is never cancelled. This means the Watch added to the controller manager is never terminated, causing the controller manager to wait its entire Options.GracefulShutdownTimeout (default: 30s) before exiting after SIGTERM/SIGINT.

https://github.com/open-policy-agent/cert-controller/blob/master/pkg/rotator/rotator.go#L110

Can rotator.AddRotator be made to accept a context and pass it through so the controller can exit quickly & gracefully?

Or am I using it wrong?

[maxsmythe]

I don't think the watch uses that context. Here is the code that sets up the controller:

cert-controller/pkg/rotator/rotator.go

Lines 568 to 595 in 54af894

	func addController(mgr manager.Manager, r *ReconcileWH) error {
	// Create a new controller
	c, err := controller.New("cert-rotator", mgr, controller.Options{Reconciler: r})
	if err != nil {
	return err
	}
	err = c.Watch(
	source.NewKindWithCache(&corev1.Secret{}, r.cache),
	&handler.EnqueueRequestForObject{},
	)
	if err != nil {
	return fmt.Errorf("watching Secrets: %w", err)
	}
	for _, webhook := range r.webhooks {
	wh := &unstructured.Unstructured{}
	wh.SetGroupVersionKind(webhook.gvk())
	err = c.Watch(
	source.NewKindWithCache(wh, r.cache),
	handler.EnqueueRequestsFromMapFunc(reconcileSecretAndWebhookMapFunc(webhook, r)),
	)
	if err != nil {
	return fmt.Errorf("watching webhook %s: %w", webhook.Name, err)
	}
	}
	return nil

IIRC controller-runtime registers the intent for a watch, but actually starts it via the manager, which passes on the context (though I could be misremembering).

The background context appears to be passed into the various client.Get() calls that occur during a reconcile event.