Handle CA rotation for Catalogd web server trust

[trgeiger]

With the addition of TLS communication between operator-controller and catalogd, we now mount the CA certificate into operator-controller so it can trust connections to the catalogd web server.

It would be ideal if we could handle rotations of the mounted CA certificate without restarting the manager. As of now, the contents of the mounted CA certificate do change on disk when the certificate rotates, but the manager does not pick up any changes since Go does not have a mechanism for reloading rootCA.

There is the possibility to implement custom connection verification logic in tls.Config.VerifyConnection(), but for this to work you would also need to set InsecureSkipVerify to true which isn't ideal. The ideal solution would probably be to contribute something akin to GetCertificate but for the rootCA upstream into Go and then use that feature in operator-controller.

It might also be worth modifying the CertificateWatcher in controller-runtime so it can function without providing a key--currently it requires both the cert and key since it's built for managing rotation on servers. A GetRootCA-type functionality for re-loading the CA cert in tls.Config would utilize the modified CertificateWatcher in a similar manner to how the certificate rotation is handled in Catalogd.

[tmshort]

See also: #1062

[tmshort]

With #1062 merged, this is basically done.