Release v2.10.23

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.4 (#6228)

Dependencies

• golang.org/x/crypto v0.30.0 (#6230)
• golang.org/x/sys v0.27.0 (#6229)
• golang.org/x/time v0.8.0 (#6105)
• github.com/nats-io/nkeys v0.4.8 (#6192)

Added

JetStream

• Support for responding to forwarded proposals (for future use, #6157)

Windows

• New ca_certs_match option has been added in the tls block for searching the certificate store for only certificates matching the specified CAs (#5115)
• New cert_match_skip_invalid option has been added in the tls block for ignoring certificates that have expired or are not valid yet (#6042)
• The cert_match_by option can now be set to thumbprint, allowing an SHA1 thumbprint to be specified in cert_match (#6042, #6047)

Improved

JetStream

• Reduced the number of allocations in consumers from get-next requests and when returning some error codes (#6039)
• Metalayer recovery at startup will now more reliably group assets for creation/update/deletion and handle pending consumers more reliably, reducing the chance of ghost consumers and misconfigured streams happening after restarts (#6066, #6069, #6088, #6092)
• Creation of filtered consumers is now considerably faster with the addition of a new multi-subject num-pending calculation (#6089, #6112)
• Consumer backoff should now be correctly respected with multiple in-flight deliveries to clients (#6104)
• Add node10 node size to stree, providing better memory utilisation for some subject spaces, particularly those that are primarily numeric or with numeric tokens (#6106)
• Some JetStream log lines have been made more consistent (#6065)
• File-backed Raft groups will now use the same sync intervals as the filestore, including when sync always is in use (#6041)
• Metalayer snapshots will now always be attempted on shutdown (#6067)
• Consumers will now detect if an ack is received past the stream last sequence and will no longer register pre-acks from a snapshot if this happens, reducing memory usage (#6109)
• Reduced copies and number of allocations when generating headers for republished messages (#6127)
• Adjusted the spread of filestore sync timers (#6128)
• Reduced the number of allocations in Raft group send queues, improving performance (#6132)
• Improvements to Raft append entry handling and log consistency (#5661, #5689, #5714, #5957, #6027, #6073)
• Improvements to Raft stepdown behaviour (#5666, #5344, #5717)
• Improvements to Raft elections and vote handling (#5671, #6056)
• Improvements to Raft term handling (#5684, #5792, #5975, #5848, #6060)
• Improvements to Raft catchups (#5987, #6038, #6072)
• Improvements to Raft snapshot handling (#6053, #6055)
• Reduced the overall metalayer snapshot frequency by increasing the minimum interval and no longer pre-empting consumer deletes (#6165)
• Consumer info requests for non-existent consumers will no longer be relayed, reducing overall load on the metaleader (#6176)
• The metaleader will now log if it takes a long time to perform a metalayer snapshot (#6178)
• Unnecessary client and subject information will no longer be included in the meta snapshots, reducing the size and encoding time (#6185)
• Sourcing consumers for R1 streams will now be set up inline when the stream is recovered (#6219)
• Introduced additional jitter to the timer for writing stream state, to smooth out sudden spikes in I/O (#6220)

Fixed

General

• Load balancing queue groups from leaf nodes in a cluster (#6043)

JetStream

• Invalidate the stream state when a drift between the tracking states has been detected (#6034)
• Fixed a panic in the subject tree when checking for full wildcards (#6049)
• Snapshot processing should no longer spin when there is no leader (#6050)
• Replicated stream message framing can no longer overflow with extremely long subjects, headers or reply subjects (#6052)
• Don’t replace the leader’s snapshot when shutting down, potentially causing a desync (#6053)
• Consumer start sequence when specifying an optional start time has been fixed (#6082)
• Raft snapshots will no longer be incorrectly removed when truncating the log back to applied (#6055)
• Raft state will no longer be deleted if creating a stream/consumer failed because the server was shutting down (#6061)
• Fixed a panic when shutting down whilst trying to set up the metagroup (#6075)
• Raft entries that we cannot be sure were applied during a shutdown will no longer be reported as applied (#6087)
• Corrected an off-by-one error in the run-length encoding of interior deletes, which could incorrectly remove an extra message (#6111)
• Don’t process duplicate stream assignment responses when the stream is being reassigned due to placement issues (#6121)
• Corrected use of the stream mutex when checking interest (#6122)
• Raft entries for consumers that we cannot be sure were applied during a shutdown will no longer be reported as applied (#6133)
• Consistent state update behavior between file store and memory store, including a fixed integer underflow (#6147)
• No longer send a state snapshot when becoming a consumer leader as it may not include all applied state (#6151)
• Do not install snapshots on shutdown from outside the monitor goroutines as it may race with upper layer state (#6153)
• The consumer Backoff configuration option now correctly checks the MaxDelivery constraint (#6154)
• Consumer check floor will no longer surpass the store ack floor (#6146)
• Replicated consumers will no longer update their delivered state until quorum is reached, fixing some drifts that can occur on a leader change (#6139)
• Resolved a deadlock when removing the leader from the peer set (#5912)
• Don’t delete disk state if a stream or consumer creation fails during shutdown (#6061)
• The metalayer will no longer generate and send snapshots when switching leaders, reducing the chance that snapshots can be sent with stale assignments (#5700)
• When restarting a Raft group, wait for previous goroutines to shut down, fixing a race condition (#5832)
• Correctly empty the Raft snapshots directory for in-memory assets (#6169)
• A race condition when accessing the stream assignments has been fixed (#6173)
• Subject state will now be correctly cleared when compacting in-memory streams, fixing some potential replica drift issues (#6187)
• Stream-level catchups no longer return more than they should (#6213)

Leafnodes

• Fixed queue distribution where a leafnode expressed interest on behalf of a gateway in complex setups (#6126)
• A number of leafnode interest propagation issues have been fixed, making it possible to distinguish leaf subscriptions from local routed subscriptions (#6161)
• Credential files containing CRLF line endings will no longer error (#6175)

WebSockets

• Ensure full writes are made when compression is in use (#6091)

Windows

• Using the LocalMachine certificate store is now possible from a non-administrator user (#6019)

Tests

• A number of unit tests have been improved (#6086, #6096, #6098, #6097, #5691, #5707, #5991, #6107, #6183, #6218)

Complete Changes

v2.10.22...v2.10.23

Release v2.10.23-RC.12

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.4

Dependencies

• golang.org/x/crypto v0.30.0 (#6230)
• golang.org/x/sys v0.27.0 (#6229)

Fixed

JetStream

• Fixed datarace in filestore calling tombs() (#6233)

Complete Changes

v2.10.23-RC.11...v2.10.23-RC.12

Release v2.10.23-RC.11

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.4 (#6228)

Improved

JetStream

• Sourcing consumers for R1 streams will now be set up inline when the stream is recovered (#6219)
• Introduced additional jitter to the timer for writing stream state, to smooth out sudden spikes in I/O (#6220)

Fixed

JetStream

• Stream-level catchups no longer return more than they should (#6213)
• Revert a JSON encoder switch made in #6165, although still maintaining the snapshot frequency (#6214)

Complete Changes

v2.10.23-RC.10...v2.10.23-RC.11

Release v2.10.23-RC.10

Important

This version fixes a regression from RC.8 that could prevent safe downgrades to earlier versions. If you have installed RC.8, you must upgrade to RC.10 before downgrading to versions before RC.8. Earlier release candidates are not affected.

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Fixed

JetStream

• Reverted #6174, introduced in RC.8, which caused a performance regression when checking duplicate message IDs (#6199)
• Reintroduce a field to the metalayer consumer assignments to fix a backward compatibility issue when downgrading to earlier versions (#6202)
• A panic that could happen when performing a stream update request during server shutdown has been fixed (#6200)

Complete Changes

v2.10.23-RC.8...v2.10.23-RC.10

Release v2.10.23-RC.8

Warning

It is not safe to downgrade from RC.8 to earlier versions due to an unexpected backward compatibility issue. A fix for this issue is in RC.10, therefore if you have already installed RC.8, you must upgrade to RC.10 first before downgrading to earlier versions. Stream data is not affected by this regression and is not at risk.

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Dependencies

• github.com/nats-io/nkeys v0.4.8 (#6192)

Improved

JetStream

• Tracking of duplicate messages using the Nats-Msg-Id header has been optimised and should now use less memory (#6174)
• Meta snapshots have been further reduced in size, reducing the encoding time (#6189)

Fixed

JetStream

• Subject state will now be correctly cleared when compacting in-memory streams, fixing some potential replica drift issues (#6187)

Complete Changes

v2.10.23-RC.7...v2.10.23-RC.8

Release v2.10.23-RC.7

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Improved

JetStream

• Unnecessary client information will no longer be included in the meta snapshots, reducing the size and encoding time (#6185)

Complete Changes

v2.10.23-RC.6...v2.10.23-RC.7

Release v2.10.23-RC.6

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Improved

JetStream

• Optimised metalayer snapshots by using a faster JSON encoder and compressor, as well as reducing the overall metalayer snapshot frequency by increasing the minimum interval and no longer pre-empting consumer deletes (#6165)
• Consumer info requests for non-existent consumers will no longer be relayed, reducing overall load on the metaleader (#6176)
• The metaleader will now log if it takes a long time to perform a metalayer snapshot (#6178)

Fixed

JetStream

• Correctly empty the Raft snapshots directory for in-memory assets (#6169)
• A race condition when accessing the stream assignments has been fixed (#6173)

Leafnodes

• Credential files containing CRLF line endings will no longer error (#6175)

Complete Changes

v2.10.23-RC.5...v2.10.23-RC.6

Release v2.10.23-RC.5

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Improved

JetStream

• Improvements to Raft append entry handling and log consistency (#5661, #5689, #5714, #5957, #6027, #6073)
• Improvements to Raft stepdown behaviour (#5666, #5344, #5717)
• Improvements to Raft elections and vote handling (#5671, #6056)
• Improvements to Raft term handling (#5684, #5792, #5975, #5848, #6060)
• Improvements to Raft catchups (#5987, #6038, #6072)
• Improvements to Raft snapshot handling (#6053, #6055)

Fixed

JetStream

• Resolved a deadlock when removing the leader from the peer set (#5912)
• Don’t delete disk state if a stream or consumer creation fails during shutdown (#6061)
• The metalayer will no longer generate and send snapshots when switching leaders, reducing the chance that snapshots can be sent with stale assignments (#5700)
• When restarting a Raft group, wait for previous goroutines to shut down, fixing a race condition (#5832)

Tests

• A number of unit tests have been improved (#5691, #5707, #5991, #6107)

Complete Changes

v2.10.23-RC.4...v2.10.23-RC.5

Release v2.10.23-RC.4

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3

Added

JetStream

• Support for responding to forwarded proposals (for future use, #6157)

Fixed

JetStream

• Consistent state update behavior between file store and memory store, including a fixed integer underflow (#6147)
• No longer send a state snapshot when becoming a consumer leader as it may not include all applied state (#6151)
• Do not install snapshots on shutdown from outside the monitor goroutines as it may race with upper layer state (#6153)
• The consumer Backoff configuration option now correctly checks the MaxDelivery constraint (#6154)
• Consumer check floor will no longer surpass the store ack floor (#6146)
• Replicated consumers will no longer update their delivered state until quorum is reached, fixing some drifts that can occur on a leader change (#6139)

Leafnodes

• A number of leafnode interest propagation issues have been fixed, making it possible to distinguish leaf subscriptions from local routed subscriptions (#6161)

Complete Changes

v2.10.23-RC.3...v2.10.23-RC.4

Release v2.10.23-RC.3

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.23.3 (#6135)

Improved

JetStream

• Reduced copies and number of allocations when generating headers for republished messages (#6127)
• Adjusted the spread of filestore sync timers (#6128)
• Reduced the number of allocations in Raft group send queues, improving performance (#6132)

Fixed

JetStream

• Don’t process duplicate stream assignment responses when the stream is being reassigned due to placement issues (#6121)
• Corrected use of the stream mutex when checking interest (#6122)
• Raft entries for consumers that we cannot be sure were applied during a shutdown will no longer be reported as applied (#6133)

Leafnodes

• Fixed queue distribution where a leafnode expressed interest on behalf of a gateway in complex setups (#6126)

WebSockets

• Ensure full writes are made when compression is in use (#6091)

Complete Changes

v2.10.23-RC.2...v2.10.23-RC.3