implement trusting a CA bundle

[ReToCode]

If we are using cert-manager, we currently build our trust on reading the ca.crt in the generated certificate's Secret to configure trust in:

• activator
• net-* implementation

Digging into the docs and asking our cert-manager team, they strongly recommend to use trust-manager to distribute a trust bundle decoupled from the actual secret generation. With that in place, it enables to rotate the CA without breaking the trust:

1. The administrator, manually adds the old and new CA to the trust-bundle
2. trust-manager distributes it to all workload
3. The administrator updates the CA, all certificates are recreated when they are expired or when they are force-renewed
4. The administrator removes the old CA from the trust-bundle

Note the relevant bits from the docs

Say you have a cert-manager Issuer which has the root certificate you want to trust in ca.crt. It's tempting to use the Secret directly and point at ca.crt, but a best practice would be to copy that root into a separate ConfigMap (or Secret). The reason is - as with many TLS gotchas - certificate rotation. If you rotate your issuer such that it's issued from a new root certificate, trust-manager will see the Secret be updated and automatically update your trust bundle to include the new root - immediately distrusting the old root. That means that if any services were still using a certificate issued by the old root, they'll be distrusted and will break.

To not limit the use to just trust-manager, we can use the API of a ConfigMap containing one or more PEM formatted certificates as trust-bundle sources. This also works with other approaches like OpenShift PKI or OpenShift Service Signer which have the same interface, see https://github.com/ReToCode/knative-encryption/blob/main/8-trust-sources/README.md. In the future we can also add https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3257-cluster-trust-bundles as a possible extension or replacement.

Tasks

• net-kourier -> Add upstream TLS trust from CM bundles knative-extensions/net-kourier#1171
• net-istio --> we don't need that in istio, as it has its own trust mechanism already
• activator -> add upstream TLS trust from CM bundles #14717
• net-gateway-api --> not possible yet
• net-contour --> not possible yet
• networking (config only) -> Add TrustBundleLabelKey to register networking#900

[ReToCode]

/cc @skonto, @nak3, @KauzClay , @dprotaso , @pierDipi

[ReToCode]

Some initial investigation results here: https://github.com/ReToCode/knative-encryption/tree/main/8-trust-sources#trust-manager.

Having CA Bundles in a Configmap is pretty common. OpenShift also has solutions that work this way. All of them allow to add a custom label to filter on them. So it might be a good approach to not limit the implementation to trust-manager, but instead to just read Configmaps with a specific label having PEM based bundles in them.

[dprotaso]

Also there's something coming in Kubernetes

https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3257-cluster-trust-bundles