[ci:component:github.com/gardener/gardener:v1.54.1->v1.56.1]

[gardener-robot-ci-1]

Release Notes:

The `hvpa-controller` container image now uses a non root user by default.

The node-problem-detector image is updated from `eu.gcr.io/gardener-project/3rd/node-problem-detector:v0.8.10-gardener.1` to `registry.k8s.io/node-problem-detector/node-problem-detector:v0.8.12`.

Fixed an issue that caused `make gardener-down` to fail when deleting the `garden` `Project`.

The e2e tests do now also tear down the Gardener environment, effectively verifying whether the `Seed` deletion works as expected.

A Pod Topology Spread Constraints webhook has been added to the Gardener-Resource-Manager which mimics the [matchLabelKeys feature](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#spread-constraint-definition) on the `pod-template-hash` label. Gardener uses this webhook to circumvent imbalanced control plane deployments across nodes and zones.

The following dependency is updated:
- k8s.io/* : v0.24.4 -> v0.25.0
- sigs.k8s.io/controller-runtime: v0.12.3 -> v0.13.0

The following image is updated:
- ghcr.io/prometheus-operator/prometheus-config-reloader: v0.58.0 -> v0.59.1

The `kube-apiserver` deployment was changed from pod anti-affinity to [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/). Non-HA shoot clusters will still have the `kube-apiserver` pods being scheduled on different nodes on a best-effort basis. For HA clusters, the Topology Spread Constraints make sure that a distribution across nodes (single-zone) and zones (multi-zonal) is guaranteed, in order to tolerate failures in these domains.

Add a panel "Response Size Rate" to the API Server dashboard

gardener-apiserver now validates that the CloudProfile's `.spec.seedSelector` is matching Shoot's Seed when the `.spec.seedName` field of the Shoot is set or modified.

The number of e2e tests carried out in parallel is configurable now.

The `gardener-resource-manager` deployment was changed from pod anti-affinity to [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/). Non-HA shoot clusters will still have the `gardener-resource-manager` pods being scheduled on different nodes on a best-effort basis. For HA clusters, the Topology Spread Constraints make sure that a distribution across nodes (single-zone) and zones (multi-zonal) is guaranteed, in order to tolerate failures in these domains.

Enables etcd-druid to trigger restarts of the etcd pods when peer URL is TLS enabled.

Golang version used upgraded to `1.18.6`

etcd-custom-image upgraded to `v3.4.13-bootstrap-8`

`Etcd-custom-image` will now retry fetching etcd configuration in case of any error

To avoid potential race-condition between go-routines updated `probeEtcd func()` to use shorter timeout.

Always update member peer URL, changed the way scale-up of etcd cluster is identified.

Updated golang version used to build images to 1.18.6

Updated golang version to 1.18

Enables etcd-druid to trigger restarts of the etcd pods when peer URL is TLS enabled.

Golang version used upgraded to `1.18.6`

etcd-custom-image upgraded to `v3.4.13-bootstrap-8`

`Etcd-custom-image` will now retry fetching etcd configuration in case of any error

To avoid potential race-condition between go-routines updated `probeEtcd func()` to use shorter timeout.

Always update member peer URL, changed the way scale-up of etcd cluster is identified.

Updated golang version used to build images to 1.18.6

Updated golang version to 1.18

Updated base image of apiserver-proxy to alpine 3.16.2

Architecture diagram was updated discouraging the use of the Kubernetes dashboard (among other hints and cosmetic updates).

Add gardenlet feature gate to automatically rewrite some dns requests to reduce amount of requests being made due to dns search path and ndots=5.

Updated base image of vpn seed server and vpn shoot client to alpine 3.16.2

Decreases the likelihood of potential race condition between the go-routines while closing the snapshotter.

[bug-fix] backup-restore does not return error when it fails to update PeerURL of member.

Update envoy proxy to v1.23.1.

Introduce shoot spec field `spec.controlPlane` to allow enabling HA control planes with failure tolerance type of `node` or `zone`. Please consult `docs/usage/shoot_high_availability.md` for more information.

Add validations to disallow switching failure tolerance type for HA shoot control planes between `node` and `zone`.

The Gardener scheduler does now consider multi-zonal seeds as potential candidates for `non-HA` and `single-zonal` shoots.

Adding an alpha HA annotation to the shoot spec where none existed is now allowed.

If a config for `PodSecurity` admission plugin is provided in the Shoot spec, `kube-system` is added to the exempted namespace.

The `ExtensionsReady` condition for `Seed`s will first be set to `Progressing` instead of being directly set to `False` when a `ExtensionsReady` condition threshold is specified in the `controllers.seedExtensionsCheck.conditionThresholds` configuration for the gardener controller manager and that threshold has not expired yet.

A bug has been fixed which caused the `EveryNodeReady` condition on `Shoot`s to become `False` and complaining about outdated cloud configs on nodes during rolling updates.

The container_oom_events_total metric is allow listed and added to the Kubernetes Pods dashboard

It is now possible to override the kubeconfig validity as well as the auto-rotation jitter boundaries for the gardenlet via its component configuration. By default, the `--cluster-signing-duration` value of the `kube-controller-manager` in the garden cluster still applies, and the kubeconfig is renewed when 70%-90% of its validity expires.

The existing `ManagedSeed` e2e test has been enhanced with verifications for the three gardenlet kubeconfig rotation scenarios.

For Shoot clusters with kubernetes `v1.25`+ `.spec.kubernetes.allowPrivilegedContainers` should not be set. Please see [here](https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#speckubernetesallowprivilegedcontainers-in-the-shoot-spec).

Gardener-managed webhooks are no longer considered by the shoot care controller when it comes to finding problematic webhooks.

Specify the kubelet flag runtime-cgroups when using containerd
The node details dashboard shows the resource usage of the system services (kubelet and containerd) for containerd based clusters

Gardenlet now checks that the seed network configuration conforms to the reality in the seed cluster in case the seed is a shoot itself.

Gardener has been being prepared for more shoot HA use-cases and thus some assumption about currently running landscapes are required: If you use a `multi-zonal` labelled seed and scheduled non-HA shoots onto it, this release of Gardener will potentially cause scheduling conflicts to the control-plane pods as it will try to locate all pods into a single zone only. Pods that can't be re-scheduled (mainly because of volume dependencies) will remain in `Pending` state.

Gardener is prepared to run non-HA and single-zonal shoots on multi-zonal seeds. In such a setup, control-plane pods of the mentioned shoots are scheduled into a single availability zone only to avoid any extra cross zonal traffic that would usually involve higher latency and cost. **PLEASE NOTE**: The `StorageClass` in seeds used for control-plane components must have `volumeBindingMode: WaitForFirstConsumer` to let the zone-pinning work properly.

Operation of a seed using cilium as networking provider and node-local-dns is now working.

A bug in the monitoring configuration that was scraping the deprecated metric `etcd_object_counts` even for k8s >= 1.21 has been fixed.

VPA components do now have a liveness probe defined.

The `BackupBucketsReady` condition for `Seed`s will first be set to `Progressing` instead of being directly set to `False` when a `BackupBucketsReady` condition threshold is specified in the `controllers.seedBackupBucketsCheck.conditionThresholds` configuration for the gardener controller manager and that threshold has not expired yet.

Added condition with type `Progressing` to the `ControllerInstallation` resource, which is maintained based on the `ResourcesProgressing` condition of the `ManagedResource` created for the `ControllerInstallation`

When the `ExtensionsReady` condition is evaluated, the `ControllerInstallations` `Progressing` condition is now also taken into account. When the `Progressing` condition is not `False`, the `ExtensionsReady` condition will be evaluated to `False`

Kubernetes container images are now pulled from `registry.k8s.io` instead of `k8s.gcr.io`, see the [announcement](https://kubernetes.io/blog/2022/08/23/kubernetes-v1-25-release/#moved-container-registry-service-from-k8s-gcr-io-to-registry-k8s-io).

The local gardener setup includes pull-through cache registries now to speed up development and testing.

The `KubeletConfiguration.Registry{PullQPS,Burst}` fields are configurable via `Shoot.spec.{provider.workers[]}.kubernetes.kubelet.registry{PullQPS,Burst}` now.

The GA-ed `SecretBindingProviderValidation` feature gate is removed and can no longer be specified via the gardener-apiserver's `--feature-gates` flags .

Fix worker group dropdown in "Node/Worker Pool Overview" dashboard.

Improve the Node/Worker Pool Overview dashboard

gardenlet's `SeedKubeScheduler` feature gate is now deprecated in favor of the `bin-packing` scheduling profile that can be configured for a Shoot referred by a ManagedSeed.

Adds prometheus metrics required for multi-node etcd.

A bug in resourcemanager that not all truthy values were considered for the `resources.gardener.cloud/ignore` annotation value is fixed.

The gardener grafana dashboards are serialized with the "compact" JSON representation into the configmap to avoid reaching the configmap size limit.

Adapt blackbox exporter resource requests to VPA recommendations

An issue that could potentially cause Pod to fail to be scheduled when the `bin-packing` scheduling profile is used is now fixed. When the kube-apiserver fails to call the `pod-scheduler-name.resources.gardener.cloud` webhook the corresponding Pod will be scheduled according to the `default-scheduler`.

Liveness and startup probes for etcd were removed. After activating them, we noticed that they cause more harm than good since the startup time for etcd clusters varies and isn't predicable. Killing the `etcd` container in such a case doesn't solve the situation and will rather end in an endless loop of restarts. This change will cause a restart of etcd clusters.

This PR fixes an issue which caused the `sts.spec.podManagementPolicy` not to be updated to `Parallel` if an existing etcd cluster is scaled-up from `1 -> x`. This can cause an issue if the cluster is afterwards completely scaled-down (aka hibernation) and scaled-up again.

Kubernetes admission plugins that can be specified in `shoot.kubernetes.apiServer.admissionPlugins` are now validated aginst the kubernetes version of the shoot cluster.

Fix the network metrics for clusters with containerd.
The "Kubernetes Pods" dashboard's "Network I/O" panel showed no data for clusters with containerd. Now it correctly shows the network metrics (sent and received bytes/s) for pods that are not in the host network namespace, also for clusters with containerd. For pods in the host network namespace no network metrics are shown because by definition the host network namespace's network stats include all the pods and system services and hence are not meaningful in the context of a specific pod. This explanation is as also included on the dashboard to avoid confusion due to missing data.
The "Node Details" dashboard's "Network I/O Pressure" panel showed incorrect readings for clusters with docker and no data for clusters with containerd. Both aspects are fixed.

Liveness and startup probes for etcd were removed. After activating them in the last release, we noticed that they cause more harm than good since the startup time for etcd clusters varies and isn't predicable. Killing the `etcd` container in such a case doesn't solve the situation and will rather end in an endless loop of restarts. This change will cause a restart of etcd clusters.

A Helm chart for deploying Etcd-Druid is now available in `charts/druid`.

Developers can now run Druid e2e tests via `make test-e2e`. Please see `docs/development/local-e2e-tests.md` for detailed information.

Base alpine image upgraded from `3.15.4` to `3.15.6`.

Base alpine image upgraded from `3.15.4` to `3.15.6`

Handles the bolt database panic in case of database found to be corrupt.

Added new metrics for multi-node etcd: `etcdbr_defragmentation_duration_seconds`, `etcdbr_restoration_duration_seconds` , `etcdbr_cluster_size` , `etcdbr_is_learner `, `etcdbr_is_learner_count_total `, `etcdbr_add_learner_duration_seconds `, `etcdbr_member_remove_duration_seconds `, `etcdbr_member_promote_duration_seconds `.

Fix the `probeEtcd func()` to probe the corresponding Etcd by getting its Endpoint Status rather than just `Get` a key.

Adds an annotation to etcd lease which indicates if the peer url is TLS enabled.

Fix statefulset volumeClaimTemplate `StorageClassName` value population if etcd storageClass is an empty string.

An issue has been fixed that caused Etcd-Druid to update immutable fields `sts.spec.serviceName` and `sts.spec.podManagementPolicy` for older `etcd` resources that had different values configured. These updates must only happen when a etcd cluster is scaled up for the first time (`1 -> x`) because (a) then these values are mandatory and (b) a disruption is accepted.

The Golang version used to compile Etcd-Druid has been updated to `go 1.18.5`.

An issue has been fixed that caused Etcd-Druid to not consider the `hostPath` configuration in the referenced backup secret `etcd.spec.backup.store.secretRef`.

Adds a document mentioning the metrics for multi-node etcd.

Gardener can now support shoot clusters with Kubernetes version 1.25. In order to allow creation/update of 1.25 clusters you will have to update the version of your provider extension(s) to a version that supports 1.25 as well. Please consult the respective releases and notes in the provider extension's repository.

Gardener can now support shoot clusters with Kubernetes version 1.25. Extension developers have to prepare individual extensions as well to work with 1.25.

Update vpa-exporter:0.1.5->0.3.0
Add `targetName` and `targetKind` labels
Added unit-tests and added a check for no targetRef.
Updated alpine image.
Added a new metric to export new VPA recommendations provided via an annotation.
Published docker images for VPA-Exporter are now multi-arch ready. They support `linux/amd64` and `linux/arm64`.
The `vpa-exporter` container now uses `distroless` instead of `alpine` as a base image.

`gardenlet` is now using `gcr.io/distroless/static-debian11:nonroot` instead of versions of `alpine` as a base image.

Added handling for `v1alpha1` config of `PodSecurity` admission plugin for clusters v1.22.x.

Golang is updated to `1.19.1`.

gardener-admission-controller's log level and log format can be now configured.

The istio ingress gateway prefers backends within the same availability zone to reduce cross-zonal traffic.

[gardener-robot]

@gardener-robot-ci-1 Thank you for your contribution.