feat: [Zero-Downtime] Istio-Gateway Secret Management

[Tomasz-Smelcerz-SAP]

Description

Zero-Downtime: Implement the Istio Gateway Secret renewal logic.

The logic is based on the POC results, but it is simplified: The additional secret object is removed, instead the Istio Gateway secret plays the key role in the migration process.
This is based on the following observation:
We must adjust SKR watcher client TLS configuration if and ONLY IF the Istio Gateway TLS configuration changes.
This has an impact for the design of the zero-downtime certificate rotation solution.
The system is designed with two independent components, running asynchronously to each other:

• The first one observes rotation of the "Root" certificates in KCP and manages the Istio Gateway secret accordingly. It's not related to any particular Kyma or SKR
• The second one manages SKR watcher client TLS configuration. It generates/updates the relevant secrets in KCP and SKR. It is coupled to the reconciliation of the Kyma CR.

Note: This issue describes the first component

Scenario 1: Bootstrap

1. No Istio Gateway secret exists
2. Wait until Root Certificate secret is available
3. Create Istio Gateway secret as a copy of the Root secret

Scenario 2: Migration - phase 1

1. Change of the Root Certificate secret is detected
2. Update the ca.crt field with new Certificate at position 0

Scenario 3: Migration - phase 2

1. Expiration of the old certificate happens in less than configured number of hours (e.g: 24)
2. Switch the tls.crt, tls.key attributes to the new certificate

For details about the migration see the attached image.

Implementation notes:

• May be implemented as an additional controller in the KLM, watching for specific secret events
• additional attributes, like timestamps, certificate expiration dates etc. may be stored as annotations in the Gateway secret object.

Reasons

We need a robust, zero-downtime solution for the Watcher TLS certificate rotation

Acceptance Criteria

• Implement the solution along with necessary unit tests, integration tests
• Update the documentation
• Manually test the rotation logic
• Do not update the secret reference in the Istio Gateway setup, so it still uses the Secret created by the Cert-Manager directly
  • To make sure it does not break the system, but it can be verified that it works as expected
  • If we then want to switch, we jsut need to update the secret reference to the secret created by the new controller implemented in this task

Feature Testing

E2E test will be covered in a follow-up after EPOIC sub-issues have been implemented. Here it should be covered with unit and integration test

Testing approach

No response

Attachments

Related Issues

#1430