Skip to content

Releases: apecloud/kubeblocks

KubeBlocks v0.8.0

12 Jan 09:34
Compare
Choose a tag to compare

KubeBlocks 0.8.0 (2024-01-12)

Exciting news! KubeBlocks v0.8.0 is officially released! 🚀 🎉 🎈

KubeBlocks v0.8.0 takes a big step forward in user experience by introducing component-level APIs, which makes standardized building blocks even smaller and more convenient to be reused.

For example, popular meta-database components like ETCD and ZK, which are widely used in various database clusters, now can be directly referenced after component modularization.
We also made Vitess Proxy a standard component, so that developers do not need to repeat their work (heavy lifting) when defining the read-write separation topology of MySQL or PostgreSQL engines in various distributions.

Further, the Addon mechanism has also been significantly improved. The helm chart of the database engine has been split from the KubeBlocks repo. From then on, changes in the database engine or version have been unbound from the KubeBlocks release.

Highlights

Independent Component API

When integrating the new database engine, we found deficiencies in the abstract design of KubeBlocks. v0.8.0 splits Component from Cluster definition to better support database types with multiple components. It supports variable references between Components, including ConfigMap, Secret, Service, ServiceReference and other variable reference types, which can better connect the relationships between components and lay the foundation for building clusters with different topologies.

Remove the addon helm chart from KubeBlocks repo

In previous versions, Helm charts for database engines were located in the "deploy" directory, tightly coupled with the KubeBlocks Operator. However, this caused two problems: first, upgrading KubeBlocks would trigger the upgrade of the database engine; second, upgrading the database engine would overwrite existing CD/CV, restarting all the clusters.

To address these problems, KubeBlocks v0.8.0 has placed the database engines into a standalone repository called "kubeblocks-addon", with version numbers added for both the database engines and associated resources. This ensures that new installations do not overwrite existing resources, thereby eliminating cluster restarts. And KubeBlocks provides the "kbcli addon" command, which allows users to download, install, use, and uninstall specific engine versions.

Supports multiple versions of database engine definitions

Prior to v0.8.0, KubeBlocks upgrades might trigger a restart of the database cluster. KubeBlocks v0.8.0, with the new Component API and Addon helm chart storage mechanism, this problem has been solved to a certain extent. We will continue to optimize the design of multiple versions in the future, and ultimately achieve burden-free upgrades.

What's Changed

New features

Pika

Supports multiple Pulsar clusters that can share a zookeeper component. #5823

Clickhouse

Integrates monitoring, scale-out and high availability. #5784

Oceanbase

Adds new active and standby cluster modes to support the complete life cycle and integrate backup, restore, monitoring and switching

MySQL

  • The Community Edition of MySQL 5.7 and 8.0 supports a full lifecycle with integrated backup recovery, monitoring, and HA.

  • Adds log audit function on ApeCloud MySQL.

PostgreSQL

Postgresql supports wal-g full backup and PITR. #180

OpsRequest

Supports custom OpsRequest, enabling specific operational actions. For example, creating and deleting Kafka topics.

NodePort

Enables NodePort access for Redis.

Compatibility

Compatible with the pre-refactored ClusterDefinition and ClusterVersion APIs.

Usability

  • opsRequest supports task queuing, allowing users to submit multiple tasks at once. For tasks that cannot run concurrently, the system will automatically execute the next task after the previous one is completed #5887

  • During KubeBlocks installation, specifying the image repository address is now possible, speeding up image pull.

Observability

Unifies configuration and management for logs and metrics collection.

API

  • Adds interface definition in ComponentDefinition #5031
  • Adds OpsDefinition API #5940
  • Adds PreDelete Action for ActionSet . You can perform this action before deleting backup. #6361
    Stability
  • Stability testings and related improvements.[wip]Kubeblocks Fault Test

kbcli

  • Enhances addon sub-command, you can install the addon from the index repository.

Incompatible changes

  • In KubeBlocks 0.8.0, we have made improvements to Oceanbase (adding functions like creating master/standby clusters, support host network and dynamic ports, support backup/restore, monitoring, logging, etc.), the clusters created in version 0.7.0 are incompatible with those in version 0.8.0, so if you are using the 0.7.0 version to manage Oceanbase, you are recommended to upgrade to the 0.8.0 version of KubeBlocks. Upgrade KubeBlocks first, then upgrade Oceanbase Addon. It is recommended to use OceanBase official data import and export tools (OBLOADER and OBDUMPER) to migrate data.

  • KubeBlocks 0.8.0 streamlines the data engines installed by default when deploying KubeBlocks by removing greptime, influxdb, neon, oracle-mysql, oroledb, tdengine, mariadb, nebula, risingwave, starrocks, tidb, zookeeper. You can install them on-demand from the addon indexing repository with the kbcli addon subcommand or the kubectl apply command; if you are upgrading from a lower version, follow the upgrade manual to avoid deleting addons that are in use, which may affect running clusters.

  • The Helm Chart of KubeBlocks 0.8.0 no longer contains dependent CRDs. When installing or upgrading KubeBlocks with the helm command, you need to install the corresponding CRDs before installing or upgrading KubeBlocks, please refer to the upgrade manual for details.

KubeBlocks v0.7.2

22 Dec 00:20
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.7.1...v0.7.2

KubeBlocks v0.7.1

24 Nov 07:27
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.7.0...v0.7.1

KubeBlocks v0.6.5

17 Nov 01:39
Compare
Choose a tag to compare
KubeBlocks v0.6.5 Pre-release
Pre-release

Full Changelog: v0.6.4...v0.6.5

KubeBlocks v0.7.0

03 Nov 10:34
Compare
Choose a tag to compare

KubeBlocks 0.7.0 (2023-11-06)

We're thrilled to announce the official release of KubeBlocks v0.7.0! 🚀 🎉 🎈

This version supports the function of referencing external components, making it possible to assemble building blocks - add-ons more flexibly.

Backup is decoupled from the cluster with a separate API, and three new object storage services from public cloud providers are added as backup storage options.

Thanks to the support of our community, KubeBlocks has included 31 open-source database engines, including MariaDB, Elasticsearch, Pulsar, Pika and more, providing more choices for you.

We would like to extend our appreciation to all contributors who helped make this release happen.

Highlights

Support referencing external components

The new feature of referencing external components brings greater flexibility to KubeBlocks clusters, allowing them to be assembled like building blocks to create new clusters. Currently, this function can be applied to two scenarios. Firstly, you can reference external components, such as external Zookeeper, that are not created and managed by KubeBlocks. You just need to define the service description and Kubernetes deployment is not required. Another scenario is that clusters created by KubeBlocks can be referenced as components by other clusters. For example, a Redis Sentinel cluster can be referenced by other Redis Replication clusters for managing high-availability.

Decouple cluster and backup #4877, #4494

The backup policy supports customizing backup methods, allowing you to specify a backup method when creating a backup. The BackupTool has been replaced with the ActionSet CRD, which defines actions for backup and restore. Three new object storage services, GCS, OBS, and COS, have been added as backup storage options. Besides, BackupSchedule is added to decouple automatic scheduling configurations from BackupPolicy that can be used for both automatic and manual backup.

Support Pika v3.5

Pika, developed by Qihoo, is an open-source NoSQL database that is compatible with Redis interfaces such as string, hash, list, zset, and set operations. It offers a cost advantage in 100 GB level and larger data volumes. Switching from Redis to Pika is seamless, as it maintains the same usage and operation habits. Currently, KubeBlocks has already supported deploying the sharded cluster mode from Pika v3.5. After a cluster is created by KubeBlocks, Pika is automatically added to the Codis cluster with rebalancing applied. KubeBlocks can also automatically manage Pika Primary-Secondary clusters, which can automatically add one primary with one or multiple secondaries.

What's Changed

New features

Pulsar #4587

Supports multiple Pulsar clusters that can share a zookeeper component.

Backup and restore

Supports three object storage services, namely GCS(Google Cloud Storage), OBS (Huawei Cloud Object Storage), and COS (Tencent Cloud Object Storage), as backup storage options.

Compatibility

Compatible with Huawei Cloud. #4697

Support multiple open-source engines

The table below provides an overview of the integrated engines and their capabilities in KubeBlocks.

V0.7.0 Vscale Hscale Volumeexpand Stop/Start Restart Backup/Restore Logs Config Upgrade (DB engine version) Account Failover Switchover Monitor
apecloud-mysql ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ N/A ✔️ ✔️ ✔️ ✔️
postgresql ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
redis ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ N/A ✔️ ✔️ N/A ✔️
mongodb ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A ✔️ ✔️ ✔️
kafka ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A ✔️ N/A N/A N/A N/A ✔️
pulsar ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A ✔️ N/A N/A N/A N/A ✔️
weaviate ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A ✔️ N/A N/A N/A N/A ✔️
qdrant ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A ✔️
greptimedb ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
nebula ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
risingwave ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
starrocks ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
etcd ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
oceanbase ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
foxlake ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
orioledb ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
oracle-mysql ✔️ N/A ✔️ ✔️ ✔️ ✔️ N/A ✔️ N/A N/A N/A N/A N/A
official-postgresql ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
mysql (Primary-Secondary replication) ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A ✔️
openldap ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
neon ✔️ N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
opensearch ✔️ N/A ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
vllm N/A N/A N/A ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
ggml N/A N/A ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
milvus ✔️ N/A ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
elasticsearch ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
tdengine ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
clickhouse ✔️ ✔️ ✔️ ✔️ ✔️ N/A N/A N/A N/A N/A N/A N/A N/A
PolarDB-X ✔️ ✔️ N/A ✔️ N/A N/A N/A N/A N/A N/A N/A N/A ...
Read more

KubeBlocks v0.6.4

23 Oct 08:02
Compare
Choose a tag to compare

What's Changed

  • fix: merge dags lost the dependencies of sub-dags by @leon-inf in #5403

Full Changelog: v0.6.3...v0.6.4

KubeBlocks v0.6.3

09 Oct 10:58
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.6.2...v0.6.3

KubeBlocks v0.6.2

07 Sep 07:59
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.6.1...v0.6.2

KubeBlocks v0.6.1

29 Aug 08:25
Compare
Choose a tag to compare

KubeBlocks v0.6.0

18 Aug 07:56
Compare
Choose a tag to compare

KubeBlocks 0.6.0 (2023-08-18)

We are happy to announce the official release of KubeBlocks 0.6.0! 🚀 🎉 🎈

This version supports stream computing engines Kafka and Pulsar and supports MySQL read-write splitting, introduces an interactive parameter configuration method.

We would like to extend our appreciation to all contributors who helped make this release happen.

Highlights

  • KubeBlocks supports Kafka v3.3
    • Apache Kafka is an open-source distributed event storage and streaming computing system that provides high reliability, throughput, and low latency for data pipelines, streaming analysis, and data integration. It is widely used in log collection and metric monitoring scenarios. KubeBlocks supports Kafka v3.3, which announced that KRaft meets production requirements, providing better partition scalability and resilience, and saving the additional costs of ZooKeeper. In addition, KubeBlocks also supports pushing data changes from MySQL and PostgreSQL to Kafka, making it easier for users to further process and handle the data.
  • KubeBlocks supports Pulsar v2.11
    • Apache Pulsar is an open-source distributed messaging and stream processing platform. It aims to provide scalability, high performance, and reliability to meet the needs of modern data processing and real-time messaging. KubeBlocks supports Apache Pulsar v2.11, and compared to traditional deployment methods, KubeBlocks can automate fault tolerance, scaling, and other day2 operations.
  • KubeBlocks supports MySQL read-write splitting
    • Read-write splitting is designed to improve the read-only processing capability of a MySQL database cluster. All write queries are sent to the master node, while read-only queries that do not modify data are distributed to multiple slave nodes. Read-write splitting is used together with the MySQL Raft Group cluster, which automatically detects changes in the master node and uses the current master node of the cluster to achieve failover. Set read_write_splitting_policy as read-write splitting at the global or session level, and the default policy is LEAST_CURRENT_OPERATIONS, which routes read-only queries to the slave node with the least active operations. The MySQL Raft Group cluster supports up to 5 nodes.

Breaking changes

  • For KubeBlocks v0.6, newly created Redis and PG clusters need password to connect, while v0.5 used to allow password-free login
  • For KubeBlocks v0.6, the backup and restore function is updated greatly. You need to manually upgrade the 0.5 version cluster to 0.6 and update backupPolicy to use the new function.
  • For KubeBlocks v0.6, since password-free login function is not supported for Postgres cluster, when restoring cluster upgraded from v0.5, if the new restored cluster is in creating status for a long time, you can check the pod logs to see whether there is password authentication failed error, you can update password to fix this.

Acknowledgements

Thanks to everyone who made this release possible!

What's New

Pulsar

  • Cluster lifecycle management and operation management, supporting the creation of Pulsar clusters, cluster deletion, cluster restarting, horizontal scaling, vertical scaling, volume expanding, and parameter modifying.
  • For monitoring, support performance monitoring of CPU, memory, network read/write traffic, and more for ZooKeeper, BookKeeper, and Broker.

Kafka

  • Cluster lifecycle management and operation management, supporting cluster creation, deletion, horizontal scaling, vertical scaling, volume expanding, and modifying parameters.
    • Horizontal scaling: In combined mode, broker replicas support 1, 3, or 5 copies. In separated mode, brokers support 1 to 100 copies and Kafka controllers support 1, 3, or 5 copies.
  • For monitoring, supports performance monitoring of Broker's CPU, memory, network read/write traffic, etc.

MySQL

  • MySQL RaftGroup Cluster supports Switchover
  • MySQL RaftGroup Cluster supports MySQL read-write splitting.
  • Data Recovery, Non-Destructive Recovery by Point-in-time(PITR). It's a beta feature.
  • Support proxy enable when creating cluster. (Beta)
    • The default specification of Vitess proxy can meet the vast majority of user scenarios. Vitess proxy automatically triggers resource increase or decrease based on changes in database nodes and node specifications, without manual operation. The CPU of Vitess proxy is 1/6 of the total number of CPU cores in the cluster nodes (three nodes), rounded up to 0.5c granularity, with a minimum of 0.5c and a maximum of 64c. The default number of replicas is 1, and currently, modifying the number of replicas is not supported
    • Connection Address: The proxy has a default connection address that supports read-write splitting. The expose command can generate VPC addresses and public network addresses for the proxy connection address.
    • Supports setting Vitess proxy read-write splitting policies.

PostgreSQL

  • Support switchover for PG Replication Cluster
  • Built-in pgBouncer

MongoDB

  • MongoDB replicaset supports switchover
  • Data recovery, non-destructive PITR (Beta)

Data migration

  • Add kbcli migration command, including creating migration tasks, viewing migration task list, viewing migration task details, terminating migration tasks, viewing logs, viewing migration templates, and other functions. Supports full migration and incremental synchronization.
  • Support MySQL data migration from MySQL 8.0 to MySQL 8.0.
  • Support PostgreSQL data migration from PostgreSQL 14 to PostgreSQL 14.
  • Support MongoDB data migration from MongoDB5.X/6.0 to MongoDB 5.X/6.0.

Compatibility

  • Support remote write to Prometheus server through Prometheus v2.41 - 2.45 compatibility testing
  • Adapt kbcli to package managers of Ubuntu and Amazon Linux 2
  • Adapt kbcli to Windows PowerShell and package managers
  • Support local environment running on Ubuntu and Amazon Linux 2 for kbcli playground
  • Support local environment running on Windows for kbcli playground

Usability

  • kbcli supports users to modify KubeBlocks parameters with local editing tools on the operating system
  • kbcli supports the "fault-inject" extension for fault injection
  • kbcli supports the "report" command to package cluster context information into a compressed file for problem troubleshooting assistance
  • kbcli supports interactive editing of configuration information for DB clusters. For cluster create, support --edit to interactively edit the yaml, and show the cluster summary before create.
  • Support canceling Hscale/Vscale OpsRequest that are running (Beta)
  • Add kbcli playground Grafana overview page
  • Kbcli alert can set email server
  • Support initializing the creation of databases and users
  • Specify a configuration file for initialization settings when installing KubeBlocks
  • Disk full lock for MySQL, PostgreSQL, MongoDB: When the database storage space is about to be full (disk usage is more than 90%), set the database to a read-only state.

Backup and Restore

  • The backup storage repository (backupRepo) refers to the target path where backup files are stored. KubeBlocks supports public cloud object storage and S3-compatible object storage.
  • Support modifying cluster configuration when restoring a new cluster
  • Add "describe-backup" command to view backup details

Observability

  • Support integration with external monitoring systems include Prometheus, VictoriaMetrics, and AMP. Cluster performance monitoring metrics are output to the target monitoring system through remote write.
  • Real-time dumping of operation logs for K8s/KubeBlocks clusters to object storage, divided by time and size, and providing read-only addresses
  • Real-time dumping of error logs for K8s/KubeBlocks clusters to object storage, divided by time and size, and providing read-only addresses

Bug Fixes

Read more