Skip to content

Clusterpedia v0.8.0-beta.0

Pre-release
Pre-release
Compare
Choose a tag to compare
@Iceber Iceber released this 26 Dec 06:37
· 106 commits to main since this release
v0.8.0-beta.0
b4117dd
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Notable Changes since v0.7.0

APIService

The kube apiserver no longer prints the following error log on a recurring basis:

E1225 07:23:43.660066       1 controller.go:116] loading OpenAPI spec for "v1beta1.clusterpedia.io" failed with: OpenAPI spec does not exist
I1225 07:23:43.660079       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.clusterpedia.io: Rate Limited Requeue.

Add the --tracing-config-file flag to allow configuration of Otel trace, example configuration

apiVersion: apiserver.config.k8s.io/v1beta1
kind: TracingConfiguration
# default
# endpoint: localhost:4317
samplingRatePerMillion: 1000000

Detailed configuration can be found in TracingConfiguration

ClusterSynchrManager

  • handle each of the pages in the resource list stage (@Iceber, #591)

This feature will reduce the maximum memory footprint for resource synchronisation.

It is controlled by two feature gates:

  • StreamHandlePaginatedListForResourceSync: stream handling of paged resources, resources within a pager will be processed as soon as possible instead of waiting until all resources are pulled before calling the ResourceHandler.
  • ForcePaginatedListForResourceSync: force paging, Reflector will sometimes use APIServer's cache, even if paging is specified APIServer will return all resources for performance, then it will skip Reflector's streaming memory optimization.

ForcePaginatedListForResourceSync may put extra pressure on the imported cluster APIServer, please be careful to enable it.

clustersynchro-manager add --page-size flags, you can control the page size of the List, the smaller the page size, the smaller the memory consumption, but may also generate more requests to APIServer.

In the future we may be able to configure these in PediaCluster.

  • add sharding feature for ClusterSynchroManager (@jxustc, #609)
    In order to avoid a single resource synchroniser taking up a lot of memory, it is now possible to synchronise the PediaCluster in shards.

In two steps:

  1. Start a new ClusterSynchroManager Deploymnet or Pod and set the sharding name with --sharding-name flag.
  2. Configure the spec.shardingName field of PediaCluster

PediaCluster will only be synchronised by ClusterSynchroManagers with the same sharding name.

  • add a feature gate IgnoreSyncLease to stop sync leases.coordination.k8s.io (@27149chen, #615, #622)

The leases resource usually needs to be skipped when using resource wildcards or synchronising full resources, and this feature gating ensures that the leases resource is globally skipped without modifying PediaCluster.

In the future, it is more recommended to use #617 to exclude certain resource types.

Multi-Cluster Resource State Metrics

  • Support resource state metrics like kube-state-metrics (@Iceber, #539)
  • Support for exposing single cluster metrics (@Iceber, #553)
  • add kube state metrics namespace command (@scydas, #566)

The cluster synchro has the same source data as kube-state-metrics available to it at sync time, so clusterpedia can easily support functionality similar to kube-state-metrics

# HELP kube_deployment_created [STABLE] Unix creation timestamp
# TYPE kube_deployment_created gauge
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="kube-system",deployment="coredns"} 1.673603507e+09
# HELP kube_deployment_status_replicas [STABLE] The number of replicas per deployment.
# TYPE kube_deployment_status_replicas gauge
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 0
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="kube-system",deployment="coredns"} 2

Added --enable-kube-state-metrics and a set of related commands, flags, to clustersynchro-manager to control multi-cluster kube-state-metrics capability.

$ ./bin/clustersynchro-manager -h
Metrics server flags:

      --metrics-disable-gzip-encoding
                Gzip responses when requested by clients via 'Accept-Encoding: gzip' header.
      --metrics-host string
                Host to expose clustersynchro manager metrics on. (default "::")
      --metrics-port int
                Port to expose clustersynchro manager metrics on. (default 8081)
      --metrics-tls-config string
                Path to the TLS configuration file of metrics

Kube state metrics flags:

      --enable-kube-state-metrics
                Enabled kube state metrics
      --kube-state-metrics-host string
                Host to expose kube state metrics on. (default "::")
      --kube-state-metrics-metric-allowlist string
                Comma-separated list of metrics to be exposed. This list comprises of exact metric names
                and/or regex patterns. The allowlist and denylist are mutually exclusive.
      --kube-state-metrics-metric-denylist string
                Comma-separated list of metrics not to be enabled. This list comprises of exact metric
                names and/or regex patterns. The allowlist and denylist are mutually exclusive.
      --kube-state-metrics-metric-opt-in-list string
                Comma-separated list of metrics which are opt-in and not enabled by default. This is in
                addition to the metric allow- and denylists
      --kube-state-metrics-namespaces string
                Comma-separated list of namespaces to be enabled. Defaults to ""
      --kube-state-metrics-namespaces-denylist string
                Comma-separated list of namespaces not to be enabled. If namespaces and
                namespaces-denylist are both set, only namespaces that are excluded in namespaces-denylist
                will be used.
      --kube-state-metrics-port int
                Port to expose kube state metrics on. (default 8080)
      --kube-state-metrics-resources string
                Comma-separated list of Resources to be enabled. Supported resources:
                "cronjobs,daemonsets,deployments,ingressclasses,ingresses,jobs,namespaces,nodes,pods,replicasets,secrets,services,statefulsets" (default "deployments,pods,services")

Storage

InternalStorage

  • make DB bad connection error message more specific (@KubeKyrie, #598)

Returns a more readable error message(storage error: database connection error: bad connection) in the case of driver.DBError

Existing url query - whereSQL needs to splice the value directly in sql, which can easily lead to sql injection problems.

We added three url queries to support parameterized query，give users the ability to prevent sql injection.

url query desc example
whereSQLStatement Splice the sql statement after WHERE,Use ? to represent the value. whereSQLStatement=(cluster=?) AND (namespace=(?))
whereSQLParam Multiple whereSQLParam can be set, each corresponding to a question mark. whereSQLParam=docker-desktop&whereSQLParam=default
whereSQLJSONParam whereSQLParam has the obvious drawback that it can't pass complex structures and special types such as arrays, etc., so you can use whereSQLJSONParam to do so. whereSQLStatement=(cluster=?) AND (namespace IN (?))&whereSQLJSONParams=WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K

whereSQLJSONParam is a base64 of an array:

$  echo WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K | base64 -d
["docker-desktop",["default","kube-system"]]

Parameterised SQL queries are enabled by the AllowParameterizedSQLQuery feature gate

BUG FIX

Others

$ kubectl kustomize ./kustomize | kubectl apply -f -

Contributers

Thanks everyone who contributed to this release!

The following users (sort alphabetically) are those who committed much in this release. Thank you!

Contributors

huiwq1990, wzshiming, and 13 other contributors
Assets 2
Loading