Fix slow leader reelection

[pjuarezd]

The leader lost was not being invoked properly, by moving leaderelection.RunOrDie out of a goroutine will ensure that a new leader is elected as soon as the existing leader pod exits, making any Operator restarts snappier.

You can confirm that the Leader lost was not being reported by Operator on SIGTERM because the log line leader lost: minio-operator-<pod id> was not printed, ergo OnStoppedLeading was not invoked.

https://github.com/minio/operator/blob/37194d9ad52edae98f2c3ec3295cf916c113ac73/pkg/controller/main-controller.go#L600C5-L603

Now, by waiting for the RunOrDie to exit we can guarantee the OnStoppedLeading is executed, the Operator logs show the log message to confirm it, and more important, a new leader is elected in matter of seconds instead of having to wait for the lease to expire.

Example output:

I0611 01:35:09.619847       1 main-controller.go:551] Received termination, signaling shutdown
I0611 01:35:09.625653       1 main-controller.go:610] leader lost: minio-operator-78b6fb57b5-n865f
I0611 01:35:09.625682       1 controller.go:185] Shutting down the MinIO Operator
I0611 01:35:09.625686       1 main-controller.go:655] Stopping the minio controller webservers
I0611 01:35:09.625817       1 main-controller.go:662] Stopping the minio controller
Stream closed EOF for minio-operator/minio-operator-78b6fb57b5-n865f (minio-operator)

[shtripat]

lgtm

[shtripat]

lgtm