Istio HPA tuning

[strekm]

Description

Remove memory metric from Istio HPA for ingress gateway for regular profile. For light profile set max replicas for ingress gateway to 1 and disable autoscale. Execute performance tests to rule out OOM.

Reasons

DoD:

• [ ] Provide unit and integration tests.
• Provide documentation.
• Verify if the solution works for both open-source Kyma and SAP BTP, Kyma runtime.
• [ ] If you changed the resource limits, explain why it was needed.
• If the default configuration of Istio Operator has been changed, you performed a manual upgrade test to verify that the change can be rolled out correctly.
• [ ] Verify that your contributions don't decrease code coverage. If they do, explain why this is the case.
• Add release notes.~~

Attachments

istio/internal/istiooperator/istio-operator.yaml

Line 72 in 6001548

- resource:

istio/internal/istiooperator/istio-operator-light.yaml

Line 63 in 6001548

hpaSpec:

istio/internal/istiooperator/istio-operator-light.yaml

Line 199 in 6001548

autoscaleEnabled: true

PR

• Adapt HPA values and scaling base #819

[barchw]

Executed performance tests for scaling based on CPU + Memory vs CPU only. A moderate increase in RTT is seen with P(95) time getting 40-80 ms slower.

• CPU + MEMORY:
  • https://snapshots.raintank.io/dashboard/snapshot/XFZUcvWEnxKf96i1TCvuk8WpRx1O54NJ?orgId=0
  • https://snapshots.raintank.io/dashboard/snapshot/AjPH8eftuKJFowMRRrPm9d3KunZctQFj?orgId=0
• CPU only:
  • https://snapshots.raintank.io/dashboard/snapshot/4f7r7MYrY3b15qL3fWETgl9XBS1Bs5ws?orgId=0&from=1715601069000&to=1715601369000
  • https://snapshots.raintank.io/dashboard/snapshot/kQom3SDf29CCVXbHssRiFDZHHuN0wFXc?orgId=0&from=1715600577152&to=1715600698957

No significant raise in memory usage was seen, and the memory level stayed very far from the set limit, meaning that there should be no risk of OOMKills happening.

On light installation, when the replicaNumber is limited to 1, the pod get's CPU bound before it could get up to the memory limit, as CPU rises significantly faster.

I limited maxreplicas from 5 to 1 instead of disabling autoscaling to make sure that configuration from Istio CR is still possible