v24.3.6

Bug Fixes

• Fix the endianness of snappy_java_compressor headers to match that of snappy-java. by @WillemKauf in #25110
• Fixed a bug over-restricting bundled json schemas to treat both "id" and "$id" as restricted keywords, regardless of the active draft. by @IoannisRP in #25060
• Updated libxml2 to v2.10.12 to remediate known security vulnerabilities
• #25007 Schema Registry/Protobuf: Fix a regression with maps. by @BenPope in #25009
• #25026 rpk: fixes a bug where rpk incorrectly handles IPv6 by adding extra brackets. by @r-vasquez in #25027

Improvements

• Redpanda will now schedule local segment merges of compacted topics, even when windowed compaction has occurred in a given housekeeping round. This ensures progress in reducing segment count in compacted topics with high produce traffic. by @andrwng in #24956
• #25082 Use streaming parsing of transaction range manifests in tiered storage to avoid large allocations which lead to OOMs with heavy use of transactions. by @nvartolomei in #25084

Full Changelog: v24.3.5...v24.3.6

v24.2.19

Bug Fixes

• Fix the endianness of snappy_java_compressor headers to match that of snappy-java. by @WillemKauf in #25112

Full Changelog: v24.2.18...v24.2.19

v24.1.19

Bug Fixes

• Avoid large allocations for the kafka response sequencing map. by @pgellert in #24741
• Fix the endianness of snappy_java_compressor headers to match that of snappy-java. by @WillemKauf in #25137
• Fixes a bug where rpk wasn't parsing --help when used alongside --redpanda-id in rpk cloud <provider> byoc apply by @r-vasquez in #24398
• Fixes a bug where failing to audit an authentication event could lead to a broker crash. by @pgellert in #24740
• If a discrete disk is used for cloud storage cache Redpanda previously rejected writes if that disk (cache disk) was full (in degraded state). This is incorrect since the cache disk isn't in the way of writes. From now on, reject writes only if the data disk is full (in degraded state). by @nvartolomei in #24485
• Remove partial kvstore snapshots at startup. by @ztlpn in #24844
• #24427 Schema Registry: fixes a bug in the Avro compatibility check reader_field_missing_default_value where it was too lenient for missing default values of null-able types. by @pgellert in #24429

Improvements

• Added metrics for pandaproxy resource usage. by @IoannisRP in #24605

Full Changelog: v24.1.18...v24.1.19

v24.2.18

Bug Fixes

• #23363 #25028 rpk: fixes a bug where rpk incorrectly handles IPv6 by adding extra brackets. by @r-vasquez in #25067
• #25008 Schema Registry/Protobuf: Fix a regression with maps. by @BenPope in #25010

Improvements

• #25081 Use streaming parsing of transaction range manifests in tiered storage to avoid large allocations which lead to OOMs with heavy use of transactions. by @nvartolomei in #25083
• PR #25013 [v24.2.x] k/fetch: fix selecting replica with highest high watermark by @mmaslankaprv
• PR #25086 [v24.2.x] pandaproxy/sr: Improve schema lookup by @IoannisRP

Full Changelog: v24.2.17...v24.2.18

v24.3.5

Features

• rpk: rpk group describe now supports printing instance IDs by @daisukebe in #24908

Bug Fixes

• Schema Registry: Fix a regression with protobuf map fields by @BenPope in #24975

Improvements

• rpk security user is now available for users with Cloud profiles. by @r-vasquez in #24951

Full Changelog: v24.3.4...v24.3.5

v24.2.17

Bug Fixes

• Fixes a crash during partition shutdown. This can happen during partition moves (cross core/broker) or at broker shutdown. by @bharathv in #24938
• Schema Registry: Fix a regression with protobuf map fields by @BenPope in #24974
• #24915 Fixes integer overflow issues when given a schema via the POST /subject/{subject}/version where version was > INT_MAX or a negative value was provided. by @michael-redpanda in #24917
• #25008 Schema Registry/Protobuf: Fix a regression with maps. by @BenPope in #25010

Full Changelog: v24.2.16...v24.2.17

v24.3.4

Features

• Introduces the node config crash_loop_sleep_sec, which sets the time the broker sleeps before terminating the process when the limit on the number of consecutive times a broker can crash has been reached. This is most useful in Kubernetes environments where setting this value allows customers to have ssh access into a crash looping pod for a short window of time. by @pgellert in #24825

Bug Fixes

• Avoid large allocations for the kafka response sequencing map. by @pgellert in #24743
• Fixes Iceberg metadata serialization to avoid writing an extraneous empty Avro block. This would previously prevent some query engines (e.g. BigQuery) from reading tables created by Iceberg Topics. by @andrwng in #24920
• Fixes a bug where failing to audit an authentication event could lead to a broker crash. by @pgellert in #24738
• Fixes a crash during partition shutdown. This can happen during partition moves (cross core/broker) or at broker shutdown. by @bharathv in #24939
• Fixes an issue where transactions incorrectly timeout due incorrect cleanup of evicted producers. by @bharathv in #24879
• Remove partial kvstore snapshots at startup. by @ztlpn in #24845
• #24914 Fixes integer overflow issues when given a schema via the POST /subject/{subject}/version where version was > INT_MAX or a negative value was provided. by @michael-redpanda in #24916

Improvements

• stable leadership under load by @mmaslankaprv in #24773
• PR #24672 [v24.3.x] raft: Make load_snapshot exception safe by @Lazin
• PR #24747 [v24.3.x] [CORE-8485] Reset translation state on snapshot by @mmaslankaprv
• PR #24764 [v24.3.x] [CORE-8787] Schema Registry: Support normalize=true for protobuf by @BenPope
• PR #24765 [v24.3.x] [CORE-8754] Handle new TLS error code by @michael-redpanda
• PR #24783 [CORE-8787] [v24.3.x] schema_registry: Normalization improvements by @BenPope
• PR #24785 [v24.3.x] bazel: Update protobuf to v29.0 by @BenPope
• PR #24786 [v24.3.x] bazel: add rp_util and compat by @rockwotj
• PR #24790 [v24.3.x] c/topic_table: notify ntp delta waiters in batches by @ztlpn
• PR #24799 [v24.3.x] [CORE-8450] schema_registry/protobuf: Optimize construction of iobuf by @BenPope
• PR #24805 [v24.3.x] rm_stm: improved logging related to producer eviction by @bharathv
• PR #24809 Manual backport 24772 v24.3.x 935 by @michael-redpanda
• PR #24836 [v24.3.x] ducktape: bump spark version by @ztlpn
• PR #24841 [v24.3.x] r/stm_manager: stop state machines before waiting for gate by @mmaslankaprv
• PR #24872 [v24.3.x] pandaproxy/sr: Fix normalized rendering for custom options by @IoannisRP
• PR #24903 [v24.3.x] CORE-8804 Make brokers field no longer required by @michael-redpanda
• PR #24906 [v24.3.x] Fix partitions local summary by @bashtanov
• PR #24910 [v24.3.x] pandaproxy/sr: Add rendering support for extension range options by @IoannisRP

Full Changelog: v24.3.3...v24.3.4

v24.2.16

Features

• #24826 Introduces the node config crash_loop_sleep_sec, which sets the time the broker sleeps before terminating the process when the limit on the number of consecutive times a broker can crash has been reached. This is most useful in Kubernetes environments where setting this value allows customers to have ssh access into a crash looping pod for a short window of time. by @pgellert in #24846

Bug Fixes

• Avoid large allocations for the kafka response sequencing map. by @pgellert in #24742
• Fixes a bug where failing to audit an authentication event could lead to a broker crash. by @pgellert in #24739
• Fixes an issue where transactions incorrectly timeout due incorrect cleanup of evicted producers. by @bharathv in #24878
• Remove partial kvstore snapshots at startup. by @ztlpn in #24843
• #24915 Fixes integer overflow issues when given a schema via the POST /subject/{subject}/version where version was > INT_MAX or a negative value was provided. by @michael-redpanda in #24917

Improvements

• #24668 stable leadership under load by @mmaslankaprv in #24708

Full Changelog: v24.2.15...v24.2.16

v24.3.3

Bug Fixes

• Fixes a bug in Redpanda's Iceberg manifest list Avro definition that previously resulted in an end-of-file (EOF) error when reading manifest list Avro files written by other engines. This could previously crash Redpanda or block Redpanda from appending Iceberg data, and could also prevent certain query engines from successfully reading Iceberg data written by Redpanda. by @andrwng in #24650
• Fixes a bug which may lead to archival_metadata_stm inconsistencies when reconfiguring clusters with recovered compacted topics. by @mmaslankaprv in #24678
• #24684 Fixes an issue that blocked the compaction of consumer offsets with group transactions. by @bharathv in #24688
• fixes rare bug leading to offset translation inconsistency in recovered topics by @mmaslankaprv in #24628

Improvements

• Added metrics for pandaproxy resource usage. by @IoannisRP in #24603
• Adds logging to mention data removed by compaction. by @andrwng in #24736
• Move failed authorization log statements from the kafka logger to a new kafka/authz logger, allowing for fine grained control over log statements for failed authorization. by @rockwotj in #24718
• rpk now supports well-known protobuf types when encoding/decoding records using Schema Registry. by @r-vasquez in #24699
• PR #24591 [v24.3.x] pandaproxy: add missing internal metrics by @IoannisRP
• PR #24608 [v24.3.x] storage: add tombstones_removed metric to probe by @WillemKauf
• PR #24619 [v24.3.x] Offset translator consistency validation by @mmaslankaprv
• PR #24627 [v24.3.x] rpk remote debug bundle: job-id help text change by @r-vasquez
• PR #24705 [v24.3.x] kafka/client: replace std::vector with chunked vector by @IoannisRP
• PR #24729 [v24.3.x] rpk bundle: Fix race condition in SASL credential redaction by @r-vasquez

Full Changelog: v24.3.2...v24.3.3

v24.2.15

Bug Fixes

• Fixes a bug where failing to audit an authentication event could lead to a broker crash. by @pgellert in #24739
• Fixes a bug which may lead to archival_metadata_stm inconsistencies when reconfiguring clusters with recovered compacted topics. by @mmaslankaprv in #24680
• #24685 Fixes an issue that blocked the compaction of consumer offsets with group transactions. by @bharathv in #24689
• fixes rare bug leading to offset translation inconsistency in recovered topics by @mmaslankaprv in #24629

Full Changelog: v24.2.14...v24.2.15