v24.1.10

Features

• Adds configuration options to trigger cache trim before the cache reaches its maximum size. by @jcipar in #19850
  • cloud_storage_cache_trim_threshold_size
  • cloud_storage_cache_trim_threshold_objects
    These mirror the options for controlling maximum size: cloud_storage_cache_size and cloud_storage_cache_max_objects
• Schema Registry: Support subjectPrefix for GET /subjects by @BenPope in #20155
• The new default behavior, if these are not set, is to trigger a trim when the cache is 100% full. by @jcipar in #19850

Bug Fixes

• Fix an issue where transforms would miscalculate their initial start offset, leading to consuming the whole input topic. by @oleiman in #20139
• Fix timequery failing when requested timestamp is in the future and local log is empty. by @nvartolomei in #21320
• Schema Registry: Improve handling of deleted schema by @BenPope in #19973
• #20591 Avoid spurious step downs if the append_entries are blocked for longer than usual on the followers. by @bharathv in #20801
• #21252 Fixes incorrect updates to next offset to be applied when racing with applying raft snapshots. by @bharathv in #21254
• rpk: fixed a bug that prevented --any-port from working with Redpanda Console when using a cluster with more than 1 node. by @r-vasquez in #20138

Improvements

• Schema Registry: GET /subject/<subject>/versions/ now supports -1 as an alias for latest. by @BenPope in #20151
• Schema Registry: remove the schema from memory when the last subject version referencing it is deleted. by @pgellert in #21259
• Short description of how this PR improves existing behavior. by @jcipar in #19850
• #21297 HTTP Proxy: Avoid large allocations during JSON serde of requests and responses by @BenPope in #21450
• #21297 Schema Registry: Avoid large allocations during JSON serde of requests and responses by @BenPope in #21450
• #21466 Schema Registry: Avoid large contiguous allocations whilst handling schema. by @BenPope in #21468

Full Changelog: v24.1.9...v24.1.10

v23.3.18

Features

• Schema Registry: Support subjectPrefix for GET /subjects by @BenPope in #20156

Bug Fixes

• PR #20589 [CORE-4878] [v23.3.x] http: change stop() to shutdown() on TLS errors in client (manual backport) by @WillemKauf
• Fix timequery failing with exceptions when the queried partition is empty. by @nvartolomei in #19951
• Schema Registry: Improve handling of deleted schema by @BenPope in #19972

Improvements

• Run directory walking during cache trimming concurrently. On some deployments it was observed that it can take hours for 600K objects with busy reactor during which fetch operations that need to cache data are blocked. by @nvartolomei in #19816
• Schema Registry: Improve handling of Accept and Content-Type by @BenPope in #19879
• Schema Registry: GET /subject/<subject>/versions/ now supports -1 as an alias for latest. by @BenPope in #20150
• #19897 rpk: group describe supports --regex flag by @daisukebe in #19898
• new metric providing more insight into recovery process by @mmaslankaprv in #19874
• refined vectorized_raft_leadership_changes_total metric by @mmaslankaprv in #19892

Full Changelog: v23.3.17...v23.3.18

v24.1.9

Bug Fixes

• Fix timequery failing with exceptions when the queried partition is empty. by @nvartolomei in #19950

Improvements

• Schema Registry: Improve handling of Accept and Content-Type by @BenPope in #19880
• #19877 rpk debug bundle now fallback to loaded profile's admin API URLs if we fail to discover the cluster in the collection steps. by @r-vasquez in #19878
• #19881 #19882 #19883 #19884 rpk debug bundle mark the generated bundle file with the node's advertised_rpc address. by @r-vasquez in #19885
• #19881 #19882 #19883 #19884 rpk debug bundle now collects the log since yesterday by default, you can still change them with --logs-since. by @r-vasquez in #19885
• #19895 rpk: group describe supports --regex flag by @daisukebe in #19896
• refined vectorized_raft_leadership_changes_total metric by @mmaslankaprv in #19891

Full Changelog: v24.1.8...v24.1.9

v24.1.8

Bug Fixes

• fixed overflow that may lead to unnecessary moves by @mmaslankaprv in #19803
• rpk cluster config get: does not round float numbers anymore. by @r-vasquez in #18850
• PR #18835 [v24.1.x] Fixed possible log discrepancy when using forced reconfiguration by @mmaslankaprv
• PR #19313 [v24.1.x] cpu_profiler: prealloc result buffers by @StephanDollberg
• PR #19801 [v24.1.x] s/disk_log_impl: don't prefix-truncate empty segments by @ztlpn
• PR #19808 [v24.1.x] t/fetch_test: wait longer for leadership after stepping down by @mmaslankaprv

Improvements

• Run directory walking during cache trimming concurrently. On some deployments it was observed that it can take hours for 600K objects with busy reactor during which fetch operations that need to cache data are blocked. by @nvartolomei in #19815
• #19831 Don't try to transfer leadership to just restarted nodes when balancing leaders. by @ztlpn in #19832
• new metric providing more insight into recovery process by @mmaslankaprv in #18840
• reduced the amount of data required to transfer over the network by @mmaslankaprv in #19834
• PR #18854 [v24.1.x] cst: manual backport of chunk download changes PR 18278 by @abhijat

Full Changelog: v24.1.7...v24.1.8

v23.3.17

Features

• Adds configuration options to trigger cache trim before the cache reaches its maximum size. by @jcipar in #19624
  • cloud_storage_cache_trim_threshold_size
  • cloud_storage_cache_trim_threshold_objects
    These mirror the options for controlling maximum size: cloud_storage_cache_size and cloud_storage_cache_max_objects
• The new default behavior, if these are not set, is to trigger a trim when the cache is 100% full. by @jcipar in #19624
• #18739 Schema Registry: Support /mode endpoints for READONLY by @BenPope in #18742

Bug Fixes

• Fixes a bug where crashes within the redpanda http client could occur when encountering tls exceptions by @graphcareful in #18696
• #18633 rpk: fixes an error in rpk topic consume that prevented the usage of the --regex flag. by @r-vasquez in #18634
• #18734 Fixes incorrect ordering of arguments in the cloud cache trim admin endpoint. by @andrwng in #18764
• #18770 Fixes a bug that would allow requests to complete that created acls for topics with invalid kafka topic names by @graphcareful in #19791
• fixed overflow that may lead to unnecessary moves by @mmaslankaprv in #19805
• rpk cluster config get: does not round float numbers anymore. by @r-vasquez in #18849
• PR #18784 [v23.3.x] raft: fix node_id mismatch log message by @ztlpn
• PR #18855 [v23.3.x] Fixed possible log discrepancy when using forced reconfiguration by @mmaslankaprv
• PR #18573 rm_stm: couple of stability fixes noticed when down scaling max_concurrent_producer_ids by @bharathv

Improvements

• Short description of how this PR improves existing behavior. by @jcipar in #19624
• #18645 rpk: topic describe supports --regex flag by @daisukebe in #18646
• made fast partition movements easier to debug. by @mmaslankaprv in #18689
• reduced the amount of data required to transfer over the network by @mmaslankaprv in #19835
• PR #18741 [v23.3.x] cloud_storage_clients: check for BlobNotFound in abs_client::do_delete_path() by @WillemKauf
• PR #19838 [v23.3.x] s/disk_log_impl: don't prefix-truncate empty segments by @ztlpn

Full Changelog: v23.3.16...v23.3.17

v24.1.7

Features

• Split cache into buckets using cloud_storage_cache_num_buckets configuration parameter. by @Lazin in #18780

Bug Fixes

• Fixes a bug that would allow requests to complete that created acls for topics with invalid kafka topic names by @graphcareful in #18769
• #18735 Fixes incorrect ordering of arguments in the cloud cache trim admin endpoint. by @andrwng in #18763

Full Changelog: v24.1.6...v24.1.7

v24.1.6

Full Changelog: v24.1.5...v24.1.6

v24.1.5

Bug fix

• Fix connect so it work's with rpk parent autocomplete

Full Changelog: v24.1.4...v24.1.5

v24.1.4

Bug Fixes

• concurrent requests of set_log_level + expiration now work as expected by @andijcr in #18437

Improvements

• #18643 rpk: topic describe supports --regex flag by @daisukebe in #18644
• #18675 rpk now will exit (1) when running rpk with unknown commands by @r-vasquez in #18676
• made fast partition movements easier to debug. by @mmaslankaprv in #18690

Full Changelog: v24.1.3...v24.1.4

v24.1.3

Features

• Schema Registry: Support /mode endpoints for READONLY by @BenPope in #18623
• Schema Registry: Support for deleted=true query parameter on POST /subjects/<subject>. by @BenPope in #18433
• #18458 rpk: ability to transfer partition leadership by @daisukebe in #18459

Bug Fixes

• Don't mark partition rebalance complete if some partitions are not moveable (e.g. due to partial recovery mode) by @ztlpn in #18518
• Enforce client quota throttling in a Kafka-compatible way, meaning we enforce the throttle delay on the next request if the client did not enforce it on its side. by @pgellert in #18568
• Fixes a bug in the http client where a crash may occur in the event certain tls verification errors are observed by @graphcareful in #18428
• #18439 Fixed an assertion triggering in a full-disk scenario by @andijcr in #18440
• #18565 Fix an edge case where a timequery returns no results if it races with tiered storage retention and garbage collection. This is important at least for consumers that fall behind retention. They interpret such response as the partition is empty and jump to the HWM instead of resuming consuming from the first available message. by @nvartolomei in #18597
• #18631 rpk: fixes an error in rpk topic consume that prevented the usage of the --regex flag. by @r-vasquez in #18632
• fixes possible stall in raft::state_machine_manger by @mmaslankaprv in #18638
• PR #18392 [v24.1.x] archival: clamp uploads to committed offset by @ nvartolomei

Improvements

• Made electing a leader faster by @mmaslankaprv in #18493
• PR #18448 [v24.1.x] cloud_storage: correct list_object() request headers and parameters (manual backport) by @WillemKauf
• PR #18476 [v24.1.x] rptest: be more permissive with errors in stress fibers test by @andrwng
• PR #18488 [v24.1.x] tests: wait for messages before adding a node to cluster by @mmaslankaprv
• PR #18503 [v24.1.x] storage: change map type for _db in kvstore by @ WillemKauf
• PR #18520 [v24.1.x] Made client id parsing vcluster aware by @ mmaslankaprv
• PR #18560 [v24.1.x] cst/ducktape: Accept errors due to gap in manifest by @abhijat
• PR #18588 [v24.1.x] archival: Disable housekeeping jobs on startup by @ Lazin
• PR #18620 [v24.1.x] tests: fix replaced segments accounting in TopicRecoveryTest by @ztlpn
• PR #18639 [v24.1.x] schema_registry: Make mode_mutability: true by default by @BenPope

Full Changelog: v24.1.2...v24.1.3