-
Notifications
You must be signed in to change notification settings - Fork 892
Change Log
Dmytro Vyazelenko edited this page Nov 11, 2024
·
243 revisions
- [Java] Update RecoverPlan after standby snapshot replication completes with the replicated snapshot entries.
- [Java] Prevent an untethered subscription re-joining the stream at an old position.
- [Java] Prevent an untethered subscription re-joining the stream at an old position.
- [Java] Prevent an untethered subscription re-joining the stream at an old position.
- [Java] Fix a performance regression in
ControlTransportPoller/DataTransportPoller
when the number of transports is larger than five, i.e. thebytesReceived
count was zero and as a resultIdleStrategy.idle()
was invoked on each duty cycle. - [Java] Assert that synchronous ControlSession calls are only allowed from a conductor thread.
- [C] Check subscribed sessions before removing a stream interest before removing by stream_id.
- [Java/C] Do not fail fast on I/O exceptions in the send/receive path, i.e. process remaining transports/publications/destinations in the same duty cycle.
- [Java] Make it clear that appointed leader config is a testing feature only.
- [Java] Keep publishing position updates from canvass and into nominate state so other cluster node members can take action in extended election timeout duration.
- [Java] Track total snapshot duration on the follower nodes.
- [Java] Upgrade to Agrona 1.23.1.
- [C] Check subscribed sessions before removing a stream interest before removing by stream_id.
- [Java] Add end-to-end test for session interest fix.
- [Java] Delete segment files without renaming so that the Archive conductor thread will not be blocked for a long period of time.
- [Java] Remove expensive file existence checks when scheduling segment file deletion.
- [C] Align loss and SMs handling between Java and C media drivers.
- [C] Align loss and SMs handling between Java and C media drivers.
- [Java] Fix ClusterBackup issue whereby upon restart it could end up reading data from the previously started replay thus failing to recover the log.
- [Java] Fail build on JavaDoc errors.
- [Java] Remove Nashorn dependency.
- [Java] Upgrade to Checkstyle 10.18.0.
- [Java] Upgrade to Mockito 5.13.0.
- [Java] Upgrade to ByteBuddy 1.15.1.
- JDK 17 is required for compiling and running!
- [Java] Use new Selector API and fix samples.
- [Java] Fix config validation on JDK 17 by using an external Nashorn ScriptEngine.
- [CI] Execute tests using
ubuntu-24.04
andmacos-latest
. - [Java] Upgrade to Agrona 1.23.0.
- [Java] Upgrade to SBE 1.33.0.
- [Java] Upgrade to Gradle 8.10.
- [Java] Upgrade to ByteBuddy 1.15.0.
- [Java] Upgrade to Shadow 8.3.0.
- [Java] Upgrade to bnd 7.0.0.
- [Java] Upgrade to Mockito 5.12.0.
- [Java] Upgrade to AsciiDoctor 2.5.13.
- [Java] Upgrade to JGit 6.10.0.202406032230-r.
- [Java] Force ASM 9.7.
- Per-Stream Session Limits
- Allows users to limit the number of different sessions (i.e. distinct publication images) that can be created connecting to the same subscription (channel & stream). This can be used to avoid resource exhaustion the driver hosting the subscription, when clients on other drivers are over zealously creating publications.
- Infer Group Subscriptions for Recovery Behaviour
- When using Multi-Destination Cast, if the user wants to make use of group semantics for nakking and retransmission, then previously group=true needed to be added to the channel configuration on the subscription. Failing to do so would potentially cause excessive nakking. This feature will set a flag on the SETUP message that the subscription can use to automate the setting of this configuration option. Setting group=true/false will override the derived value.
- Static Counters
- Add new API to create static counters whose lifetime is decoupled from an Aeron client instance that created them. Unlike the normal counters which are deleted when an Aeron client is closed or times out, the static counters remain valid until the MediaDriver is closed.
- Static counters are similar to the system counters which are created by the MediaDriver upon startup. Such counters cannot be delete and only one instance of each counter can exist per MediaDriver instance. A static counter is identified by a tuple <typeId, registrationId> which are specified at the creation time. Calling addStaticCounter with the same <typeId, registrationId> multiple times will only create a counter once with subsequent calls returning the id of an existing counter. An exception will be thrown upon an attempt to specify <typeId, registrationId> of an existing non-static counter.
- Archive Integration of Response Channels
- Users of the AeronArchive client can make use of response channels for control, replay and replication communication paths.
- Archive Logging Improvements
- To increase visiblity of replay and recording session state changes.
- Enable either REPLAY_SESSION_STATE_CHANGE or RECORDING_SESSION_STATE_CHANGE when configuring the aeron agent.
- ReplayMerge Backoff
- To slow down internal state transitions that involve calls to ‘get max recorded position’.
- Reduces network traffic.
- Greatly reduces log output when debugging is enabled.
- RetransmitHandler Improvements
- On unicast channels, a received NAK may immediately replace a previous received NAK so long as the offset has increased.
- Retransmitted Bytes Counter
- To understand network conditions, it can be useful to know how many bytes have been retransmitted due to reception of NAKs.
- We have exposed a new counter “Retransmitted bytes”, which serves as an approximate indicator of this information.
- The counter only approximates the true value, as MDC retransmits are only recorded once, rather than N times (once for each destination). It is also worth noting that the retransmitted bytes are not included in the total bytes sent counter value. We may improve these aspects in the future.
- C++ API Deprecation
- Aeron currently has two C++ APIs, a pure C++ version and a wrapper over the C API. This release we are no longer adding new features to the C++ API. This include being able to offer raw
uint8_t *
values and the new static counters feature.
- Aeron currently has two C++ APIs, a pure C++ version and a wrapper over the C API. This release we are no longer adding new features to the C++ API. This include being able to offer raw
- [Java Cluster] IngressAdapter to delegate to ConsensusModuleAgent when schema id is unknown
- [C++ Wrapper] Correctly implement tracking of context on the header.
- [C/C++] Do not compile with
-Ofast
compilation level. - [C/C++] Pass custom optimization level to the MSVC compiler.
- [Build] Download and install CMake as part of the build process to ensure latest version of CMake.
- [C] Tidy up aeron_reallocf to work correctly on allocation failure.
- [C Driver] Expose try_connect_stream method for use from within ATS.
- [C Driver] Remove unconnected stream message.
- [C Driver] Fix string format for AERON_SET_ERROR arguments.
- [C/C++/Java] Add per channel untethered window limit and resting timeouts (#1588)
- [C++ Wrapper] Add findByTypeIdAndRegistrationId to C++ Wrapper.
- [Java Driver] Pre-start async task executor threads to avoid inheriting affinity from the conductor thread.
- [Java Cluster] Print
serviceId
when throwing "ack out of sequence" exception. - [C++] Fixed set thread name. (#1594)
- [Java/C++ Archive] Use response channels in the Archive Client (#1560)
- [PS] Add Powershell script for Windows build.
- [C/Java Driver] Do not resolve self hostname when starting MediaDriver to avoid a DNS-induced stall. Make
resolverName
required when driver name resolution is used. - [Java/C/C++ Client] Propagate context though assembled header when using the buffer builder.
- [Java] Fix FixedLossGenerator to not drop frames before the specified "drop region".
- [C/Java Driver] Expose counter for number of bytes retransmitted.
- [Java Cluster] Remove redundant param.
- [C] Move validation earlier in the flow to prevents leaks in ATS on validation failure.
- [Java Samples] Use context instead of image by session id to resolve Image on request message.
- [C/Java Driver] Add session limits for a stream to prevent resource exhaustion of resources on drivers hosting subscriptions (#1598)
- [Java Cluster] Add support for installing schema extensions into the ConsensusModule.
- [Java Samples] Change response channels sample to use controlled poll to prevent response channel test failures.
- [Java] Add some additional error test to ReplayMerge and test for concurrent ReplayMerges along with additional error reporting.
- [C++ Archive] Rethrow original exception in ReplayMerge so information is not lost. #1587
- [C] add and use aeron_mkdir_recursive (#1602)
- [Java] Version interface to allow testing usage of it (#1605)
- [Java] Include file name when mark file version validation fails.
- [Java] Correct offsets for response setup flyweight in ReceiveChannelEndpointThreadLocals.
- [Java] Default cluster log flow control to driver default which will commonly be max now that overruns can go up to half a term.
- [Java Archive] Assign client name to Archive's Aeron client.
- [Java Cluster] Assign client name to Cluster Aeron clients.
- [C++/Java Client] Pre-touch mapped log buffers on the client if
Aeron.Context.preTouchMappedMemory()
returnstrue
, i.e. client configuration takes precedence over channel parameters and media driver configuration. - [Java Cluster] Delegate decision on unknown schema to consensus agent
- [Java Cluster] Ensure service count is zero when consensus module extension is installed.
- [C/C++/Java] Annotations to support config/counters documentation/validation (#1593)
- [C/Java Driver] Add unicast semantics to RetransmitHandler (#1603)
- [Java Cluster] Add archive and egress response endpoints for cluster and make them optional.
- [Java Cluster] Remove default for cluster services directory configuration and leave resolution of that value until conclude.
- [Java Archive] Use publication.availableWindow instead of looking up the pubLmt counter.
- [Java Cluster] First cut at consensus module extension integration.
- [C Driver] Fixes for fixed loss interceptor used for testing.
- [Java Cluster] Remove unused params.
- [Java Archive] Separate lines for debugging.
- [Java Cluster] Provide access to archive and aeron for consensus module extension.
- [Java Cluster] Delegate cluster session creation to extension if available.
- [Java Cluster] delegate callbacks to extension on state changes.
- [C/Java Driver] Track EOS for each receiver connected to the NetworkPublication (#1606)
- [C Client] Declare a volatile pointer to
aeron_image_list_stct
instead of a pointer to a volatile struct. (#1607) - [Java Cluster] A more decoupled approach to ConsensusModuleExtension.
- [Java Cluster] Provide access to log publication after an election for consensus module extension.
- [Java] Improve error messages when publication errors occur.
- [Java Cluster] Consensus module extension API refinements.
- [Java Cluster] Add replay of session open and close delegation to consensus module extension.
- [Java Cluster] Add replay of extension message delegation to consensus module extension.
- [Java Cluster] ConsensusControlState to be passed from Agent to Extension and allows control.
- [C/C++ Client] Use
aeron-client-conductor
as a client conductor agent name across all implementations. - [Java Cluster] pass actingVersion to onMessage of ConsensusModuleExtension
- [Java] Version generated class to have toString
- [Bash] Add option to disable C++ Archive API.
- [Java Cluster] Add ability to access response publication for a session and only update keepalive status when open.
- [Java Cluster] Handle cluster extension messages on egress listener side.
- [Java Cluster] Add extension message support to AeronCluster client.
- [C++ Wrapper] Allow pass through of
const uint8_t *
buffers. - [C++] Replace array with pointer to address code-ql issue on Publication in C++ API and Wrapper.
- [Build] Fix aeronmd rpath in the generated package.
- [Java Cluster] Use separate archive control session for consensus module extension.
- [Java Cluster] Add work cycle callback for ConsensusModuleExtension.
- [C/Java Driver] Replace exception with increment of counter for overflowing the retransmit pool.
- [C] add -w to C AeronStat app (#1614)
- [Java Cluster] Provide access to encoded principle for authorising consensus module extension session.
- [Java Archive] Add support for response channels startReplay and startBoundedReplay.
- [Java Samples] Fix StreamStat mapping of
sub-pos
which had a join position included in the channel uri and prevented it to be merged with the rest of stream positions. Also ensure that longest channel uri is printed. - [Java Cluster] Extract ReplicationParams for RecordingReplication to allow for more flexible configuration.
- [Java Cluster] Use response channels for log replication during the election.
- [C Driver] Handle a case when
getifaddrs
call fails. - [Java Cluster] Provide access to cluster client session from extension via an interface to restrict usage.
- [Java Cluster] Add source response channel to MultipleRecordingReplication.
- [Java Cluster] Simplify hash code for RecordingLog.
- [Java Cluster] Use response channel in ClusterBackup when connecting to the cluster archive.
- [Java Archive] add helper method to determine if Archive Context is concluded (#1617)
- [Java Cluster] Validate standby snapshot (#1616)
- [Java] Add tests for HeaderWrite class + always combine values in the same order (from MSB to LSB).
- Intercept replay and recording session state changes (#1615)
- [Java Archive] Use
getMaxRecordedPosition
instead ofgetRecordingPosition
to avoid two different calls while resolving position. Also callgetMaxRecordedPosition
only once in theattemptLiveJoin
. - [Java Cluster] Add ability to close a cluster session from the consensus module extension.
- [C/Java Archive/Driver] add backoff to calls to getMaxRecordedPosition in ReplayMerge (#1618)
- [Java Archive] Prevent replay token clashes.
- [C Samples] Fix CnC version printed in AeronStat header.
- [Build] Modify CMake build to have exports for Aeron targets.
- [Java Cluster] separate ingress and log message callback
- [Java Cluster] Method renaming for log extension dispatch.
- [Java Cluster] Add authorization service to ConsensusModuleControl
- [Java Cluster] cluster session - do not delete principal when transitioning to open
- [Java Cluster] Clean up session logging.
- [Java Cluster] Add event log for getMaxRecordedPosition call.
- [Java Cluster] callback on prepare for new leadership in ConsensusModuleExtension
- [Java Cluster] Consistent checking of leadership id in admin request to cluster.
- [Java Cluster] Add reason for ELECTION_STATE_CHANGE + log state transition after the logic was executed.
- [Java Cluster] Process standby snapshot control signals like normal snapshot signals. (#1634)
- [C Client] Fix #1633, only get channel status if pointer is non-null.
- [C++ Wrapper] Fix Image wrapping in a couple of cases improvements. Delete implicit *FragmentAssembler copy/move ctors/operators (#1639)
- [Java Archive] Reduce archive listRecordingsForUri idling (#1638)
- [C Client] Return an error if the client name is too long (100 chars) instead of truncating.
- [C++ Wrapper] Use a copy constructor for wrapper Context to better handle resource management.
- [C/Java] Send a flag on SETUP to indicate that the publication is expecting a group of subscription and use that to select the multicast feedback delay generator when inferring behaviour.
- [Java Driver] Set default on stream session limit property.
- [Java Cluster] Consolidate logic for closing a cluster session in the consensus module.
- [Java/C/C++ wrapper] Add new API to create static counters. (#1625)
- [C++ Wrapper Client] Implement offer with raw pointer and length for exclusive publications.
- [Java Cluster] Do not reinstate cluster sessions that have timed out but not got committed before a change of leadership.
- [C/Java Samples] Align implementation of AeronStat.
- [Java Cluster] Do slow tick work in consensus module agent until no work is done in a cycle.
- [Java Client] Improve error message when failing to open CnC file.
- [C/Java] Retry on
ERROR_SHARING_VIOLATION (32)
on Windows when awaiting CnC file mapping. - [C++/Java Archive] Use Publication.publicationLimit instead of looking up counter.
- [Java Driver] Fix re-resolution to trigger when no SMs received by Publications and new Publications are constantly created.
- [Java Cluster] Fix for #1619. Handle the case where the last portion of data a cluster log is a padding frame.
- [Java Cluster] Make
describe recordingId
work with recordings in any state. - [Java Archive] Only index valida recordings and implement ArchiveTool methods via
forEach
. - [Java] Print offending value in exception with ChannelUriStringBuilder.
- [Java Archive] Do not mark recording as
VALID
if there are no segment files. - [Java Archive] Always report state OK even when not changing the state.
- [Java Archive] Mark purged recording as
DELETED
. - [C] Fix heap-buffer-overflow issue when removing entries from the
aeron_int64_counter_map
. - [C] Align
aeron_int64_counter_map
with Java, i.e. do not allow initial_value to be put into the map. - [Java Cluster] Add API for taking a snapshot in consensus module extension.
- [C/Java Driver] add config option and uri param for setting max retransmits (#1640)
- Upgrade to CMake 3.30.0.
- Upgrade to HdrHistogram 2.2.2.
- Upgrade to ByteBuddy 1.14.19.
- Upgrade to Agrona 1.22.0.
- Upgrade to Hamcrest 3.0.
- Upgrade to SBE 1.32.1.
- Upgrade to JUnit 5.11.0.
- [Java] Fix ClusterBackup getting stuck in the
LIVE_LOG_REPLAY
state. - [Java] Validate that the egressChannel is set.
- [CI] Add Clang 18 to the build matrix.
- Upgrade to SBE 1.31.1.
-
Response Channels
- A new experimental feature that provides server/client behaviour to Aeron clients.
- Asynchronous DNS Name Resolution
- Improve Publication/Subscription creation performance and avoid stalls by performing DNS resolution asynchronously with Aeron.
- Tethering Behaviour
- Prevent premature disconnections by utilizing the maximum window size for tethering, even in congested networks. OR Improve tethering behaviour when using congestion control.
- Adjust Overrun Behaviour
- Make better use of buffers and reduce dropped consumers.
- Enhance the performance of receivers in a max flow control use-case.
- Congestion Control Recovery
- Significantly reduce recovery time in congested networks, without the need for any configuration changes.
- Loss Recovery Improvements
- Reduce negative acknowledgements and message redelivery to improve the efficiency of message recovery.
- ClusterBackup is stuck in the
LIVE_LOG_REPLAY
state as it trying to find recording using wrongarchiveId
. Fixed in 1.44.1.
- [Java/C Driver] Support configuration flag for experimental features (#1577)
- [C Driver] Fix buffer overflow when updating counter labels.
- [Java ConsensusModule] Log cluster session failures.
- [Java/C/C++ Client/Driver] Add ability to set name on the Aeron client. (#1581)
- [Java Driver] Async DNS resolution. (#1564)
- [C Driver] async DNS resolution for the C media driver (#1566)
- [Java CluserBackup] Have cluster backup start from a specific position (#1579)
- [Java ConsensusModule] Introduce clustered service specific SnapshotDurationTracker (#1575)
- [Java Cluster] Allow setting ClusterClock via
aeron.cluster.clock
system property. - [Java Driver] Call force on CnC file when doing clean shutdown and signalling ready.
- [C Driver] Add
aeron_msync
and use to flush CnC file contents upon start and shutdown. - [Java Cluster/Archive] Add *MarkFile.force() method to flush any pending updates to disc and use it for signalling readiness and when closing the mark file.
- [C/Java Driver] Dissect frames and commands using key/value format.
- [C/Java Driver] Use nanosecond resolution for logging messages.
- [C Driver] Install pre-/post-interceptors logging interceptors if the incoming/outgoing chains are not NULL. Change logging order for outgoing events to log before executing command to match Java and log plain messages before encryption.
- [Java Driver] Fix
FRAME_IN/FRAME_OUT
event logging, i.e. allow redefinition of previously loaded classes. - [C Driver] Scope logging functions with a "log" struct. (#1556)
- [C Driver] Add C driver logging for send_nak_message and resend (#1553)
- [C Driver] Track pending setups for cases when notified of unconnected messages. Only allow pending setups for unknown interest or existing images when in ACTIVE state. Handle unconnected streams by sending setup eliciting SMs.
- [C Driver] Fix potential leak in when adding stream interest to subscription by session.
- [Java Samples] Improve response channels samples.
- [C Driver] Align min flow control implementation with Java + allocate memory only if a receiver is added.
- [Java ConsensusModule] Use fixed 64 byte alignment that does not depend on the CACHE_LINE_LENGTH and is backwards compatible.
- [Java ConsensusModule] Fix IndexOutOfBoundsException when recording entry straddles page size.
- [Java ConsensusModule] Add leadership term id counter to ease debugging.
- [Java ConsensusModule] Add election counter.
- [Java ClusterTool] Ensure publication is connected before making cluster members query to avoid exception on checking result from ClusterTool.
- [Java] Add missing properties to the
toString
+ print Archive/ConsensusModule/ClusteredServiceContainer configuration ifaeron.print.configuration=true
. - [C/C++/Java Client] Re-assemble header for the fragmented message before calling user code.
- [Java Archive] Add Archive threading mode to the conductor duty cycle counter labels.
- [Java ClusteredServiceContainer] Trap and log snapshot exceptions while allowing cluster to resume (#1568)
- [Java ClusteredServiceContainer] Track ClusteredService lifecycle via a single field.
- [Java ConsensusModule] Remove retry logic from the ConsensusModuleProxy.
- [Java ConsensusModule] Retry sending cluster member query.
- [Java Cluster] Clarify Javadoc around
clusterSessionId
foroffer/tryClaim
methods. - [C Samples] add locks around channel info map in response server sample app
- [Java Cluster Client] Close any partially created ingress publications and egress subscription if connect was not completed.
- [Java Client] Add
Aeron#asyncRemoveSubscription
. - [Java ArchiveTool] Update Help section of Archive Tool to describe the different 'describe' commands. (#1573)
- [C++ Archive Client] Add
archiveId
API to the AeronArchive. - [Java Archive Client] Use
archiveId
for looking up the RecordingPos counters. (#1571) - [C++/Java Archive Client] Add getMaxRecordedPosition to AeronArchive to get the length of a recording if it is active or stopped.
- [Java/C++ Archive Client] Update Aeron Archive control protocol minor version.
- [C Driver] Align implementation of retransmit handler with Java driver.
- [Java Cluster] Set isLeader appropriately on cluster member when listing. Issue #1569
- [Java Driver] Remove unused sourceAddress from PublicationImage.
- [Java Driver] Allow the RetransmitHandler to prevent retransmissions where the term offsets don't match but NAK'd region is wholly subsumed by an existing retransmission.
- [C Driver] Per stream nak delay (#1570)
- [Java Driver] Add support for per channel configuration of nak-delay.
- [Java Driver] Add fixed loss interceptor for testing (#1551)
- [Java Driver] Suppress retransmits of the offset of the NAK is in the range of any active retransmits.
- [Java Driver] Clamp all retransmissions at the point where we create the RetransmitAction to ensure clamped length is accurately applied.
- [C/Java Driver] Change multicast NAK backoff and unicast delay linger to 10ms from 60ms.
- [C++ Wrapper Client] Prevent sanitiser errors when shutting down subscription and client in quick succession (#1549)
- [Java Archive] Validate replay completely before creating a replay Publication.
- [C Samples] add response_client/server sample apps (#1559)
- [C Driver] check for response parameters when adding destinations (#1565)
- [Java Driver] add validation when control-mode=response (#1563)
- [C Utility] Add support for PUT requests using aeron_http_util. (#1561)
- [Build] Set aeronmd rpath in the generated package. (#1557)
- Upgrade to Agrona 1.21.1.
- Upgrade to SBE 1.31.0.
- Upgrade to ByteBuddy 1.14.13.
- Upgrade to ASM 9.7.
- [Java/C] Allow publication image flow control under runs if they are heartbeats so a stream can be kept connected and to get a more accurate EOS when using MDS.
- [Java/C] Restrict the allowing of heartbeat under runs to only one term behind.
- [Java/C] Count under runs for heartbeats.
- [Java/C] Track EOS position per image connection for MDS.
- [Driver] Support specifying endpoints and tags in the same URI
- [C Driver] Add reference id to counters.
- [Driver] Set reference id of sub-pos to registration id of the image/publication.
- [C Client] Fix aeron_print_counters.
- [Java Driver] Touch ups for RetransmitHandler.
- [C Driver] Put enum events in order.
- [C Driver] Remove reliance of max event num in enum for driver agent.
- [C Driver] Fix signed comparisons.
- [C Driver] Declare void param.
- [C Driver] Keep loop types consistent.
- [Java Driver] Add a clamp on retransmit length.
- [Java Client] Fix Aeron.getSubscription(long) JavaDoc.
- [Java Driver] Encapsulate retransmit multiples constants.
- [Samples] Add sample code to support stress testing of MDC and Unicast connections.
- [Cluster] Mark services as active only after they have successfully started.
- [Driver] Improve error messages reported by clients if the Media Driver has been shutdown.
- [C Client] Populate the client image subscriber_position_id.
- [Cluster] Improve error message when cluster fails to connect.
- Add detailed version information to Aeron error counters for all components. Include release version and Git SHA.
- [C Driver] Fix capture of spy subscription channels so that image availability callbacks include the correct subscription URI.
- [C] Use -fno-omit-frame-pointer on Linux native builds so provide easy and accurate profiling with tools like perf.
- Add
+guilty
to version information if using a build that has changes that have not been committed to git. - [C Client] Correctly handle the case where an image is used across multiple subscriptions. Including ensuring the unavailability callbacks a delivered correctly.
- [Archive] Exclude invalid recordings from ArchiveTool describe and add a
describe-all
command which shows valid and invalid recordings. - [C Driver] Introduce a parameters structure that can be used to carry transport configuration information when initialising a udp_channel_transport. Also pass MTU information through the bindings to allow for improved validation of within the DPDK bindings.
- [Driver] Only include timestamps on packets that carry the DATA_BEGIN_FLAG to prevent accidental corruption of packets when the timestamp is being carried within the payload.
- [Cluster] Remove the need for MDS subscription on cluster ingress by having a separate IPC ingress subscription when required. Plus await sockets closing on the log subscription in parallel with the other activities when preparing for an election.
- [Cluster] A leader should assert its leadership when a follower requests a vote and the leader has a more up-to-date log rather than re-initialising an election which could result in log truncation.
- [C Driver] Ensure loop iteration index is used when setting timestamps for publication.
- [Cluster] Log event for unknown session close in service container rather than throw and exception.
- [Cluster] Throw exception if a member id is not found in the cluster backup's configuration.
- [Cluster] Avoid list creation when throwing an exception.
- [Archive] Add ArchiveTool mark-valid, mark-invalid commands. These provide a way to patch the catalog in an emergency.
- [Cluster] Ensure that the ClusterMarkFile link is created using the parent directory from the MarkFile object.
- [Driver] Add stricter validation on control addresses, including: Require control address with
control-mode=dynamic
. Allow control addresses on non-MDC unicast publications. Add clearer validation on control-mode when using tags. - [Driver] Introduce an enum to track the control mode specified on Channel URIs.
- [Driver] Update pub-pos when updating pub-lmt.
- [Java Driver] Check if CnC file has error log buffer allocated.
- [Driver] Print usable space when throwing StorageSpaceException or returning an error.
- [C Samples] Make AeronStat respect update interval option. AeronStat was effectively ignoring
-u
and incorrectly using-t
instead. - [Java] Fix NullPointerException in LogBuffers when construction fails with an exception.
- [Driver] Add counter to represent the number of bytes mapped by the Driver.
- [Driver] Pass in setup triggering SMs into flow control so that tags can be matched for controlling setup sender limiting.
- [C] Fix client image reference counting by use the appropriate literals to ensure that 64-bit parameters are used.
- [Cluster] Call unexpectedTermination on incompatible app versions in the ConsensusModuleAgent.
- [C Driver] Implement low storage warnings for the C driver.
- [Java Driver] Backoff from sending when a short send it detected by polling for status messages as a means to provide a little breathing room for underlying stack.
- [Cluster] Detect when log subscription in an election has an error, such as bind exception, throw an exception giving context and restart election.
- [Cluster] Close log subscription after prepareForNewLeadership.
- [Cluster] Introduce SnapshotDurationTracker to track the time taken for snapshots in the Cluster.
- [Java] Rename aeron-version module to aeron-annotations.
- [C Driver] add max_retransmission_length.
- [Java] Remove suffix from version.
- [Cluster] Stop log recording even if an image has not been picked up and no need to try stop the recording by identity if stopped by log subscription id.
- [Cluster] Increase visibility from private to package-protected for a number of ClusterTool utility methods.
- [Cluster] Add STANDBY as a component type to cluster mark file codec.
- [Cluster] Add cluster services directory to the cluster mark file and allow it to be set via the ConsensusModule.Context. This means that ClusterTool can resolve the location of the service container mark files even if they are not stored in the same directory.
- [C++] Use static constexpr for declaring constants.
- [C/C++] Use functions to return version information to enable runtime version checks against the media driver.
- [C++ Wrapper Client] Allow to access the underlying client of the C++ wrapper.
- [C Driver] Fix error reporting for aeron_port_manager.
- [C Driver] Implement timestamp capturing for the recvmsg path.
- [C Driver] Switch sender poller to poll for a single SM, i.e. do the
recvmsg
syscall. - [C Driver] Add additional validation for frame_length on status messages.
- [C Driver] Fix error format strings when configuring sockets for the udp transport.
- [C/C++] Remove date/time from the version files for a reproducible build.
- [C] Fix line ordering bug and add missing return statements and error appending.
- [Driver] Allow publication image flow control under runs if they are heartbeats so a stream can be kept connected and to get a more accurate EOS when using MDS.
- [C Driver] Use correct type for array length calculation, fix #1539.
- [Driver] Track EOS position per image connection for MDS.
- [C++ Wrapper] Fix maxMessageLength.
- [C++ Wrapper] Fix FragmentAssembler.
- Upgrade to AsciiDoctorJ 2.5.10
- Upgrade to JGit 5.13.2.202306221912-r
- Upgrade to Versions Plugin 0.50.0
- Upgrade to ByteBuddy 1.14.10
- Upgrade to ASM 9.6.
- Upgrade to Agrona 1.20.0
- Upgrade to JUnit 5.10.1
- Upgrade to SBE 1.30.0
- [Java] A leader should assert its leadership when a follower requests a vote and the leader has a more up-to-date log rather than re-initialising an election which could result in log truncation.
- [Java] Change role to leader after winning election so leadership can be asserted during replay or replication.
- [Java] Log event for unknown session close in service container rather than throw and exception.
- [Java] Ensure that the
LoggingErrorHandler
instance is closed when wrapping a user-definedErrorHandler
. - [Java] Invoke background work when idle and service is active so external connections from a cluster can be maintained.
- [Doc] JavaDoc clarifications on MediaDriver dirDeleteOnStart behavior.
- Upgrade to Agrona 1.19.2.
- Upgrade to ByteBuddy 1.14.7.
- Add port manager to C and Java drivers to allow for a limited range of ports that can be used when subscription endpoint addresses and publication control addresses specify a port of
0
. This can make network access configuration simpler (Driver). - Wait for end of stream event on the log instead of the counter becoming unavailable before entering into a new Election (Cluster).
- Close log publications before trying to stop recording to speed up graceful leader step down (Cluster).
- Ignore resting subscription positions when calculating join position or draining driver entities when tethering is being used (C Driver).
- Extensions points to support Standby Snapshots (Cluster).
- Ensure that the
clusterDirectoryName
field is synced with theclusterDir
configuration parameter during conclude inConsensusModule
andClusterBacker
(Cluster). - Cluster tutorial updates (Documentation).
- Extract invocation of background work in the service container.
- Do not fail if Aeron directory exists, but the CnC file does not (C Driver).
- Use
aeron_errcode
to get the latest error code when reporting problems creating the Aeron directory (C Driver). - Don't let a bounded replay go beyond counter value to stop position so commit position can be used for replay (Archive).
- Add addition example Authentication and Authorisation services (Archive/Cluster).
- Add additional documentation to specify details when Authorisation is used (Archive/Cluster).
- Clarify logic about extending a replay while being bounded (Archive).
- Changes to host name resolution on start up (Driver).
- Perform host name resolution once and track its execution time via the duty cycle tracker and the event log.
- Use
<unresolved>
as a host name if it cannot be resolved instead of null. - Fix GCC 9 warning
- Await initial counters being updated by the conductor thread before reading the values.
- Trigger graceful leader close election based on recording stop signal (Archive).
- Add end of stream position to Subscriptions (Client).
- Allow for relocatable mark files for the
ConsensusModule
,ClusteredServiceContainer
, andClusterBackup
(Cluster). - Move close operations on MDS transports on the conductor not the receiver thread (C Driver).
- Allow preparing for new leadership to be less serial to reduce election time (Cluster).
- Async removal of destinations as log adaptor closes when preparing for election (Cluster).
- Fix CMD_OUT_ERROR log format (C Driver).
- Fix
aeron_err_clear()
(C). - C driver handle untethered subscriptions correctly, ensure that tethering behaviour matches Java driver (C Driver).
- Fix name in toString (Java Client).
- Improve error message (C++ Archive Client).
- Clean up Apple M1 compilation. Move loss reporter function into compilation unit. Add pragmas to account for unused but set variables (C).
- Add logging to track information when a
ReplicationSession
ends (Archive). - Add deduplication step for aeron-agent jar.
- Add deduplication step for aeron-all jar.
- Add an annotation processor to generate a version class for a number of the Aeron packages.
- Include the git sha in the version C binaries.
- Extract
untetheredWindowLimit
method. - Upgrade to SBE 1.29.0.
- Upgrade to JUnit 5.10.0.
- Upgrade to Agrona 1.19.1.
- Store
Aeron.NULL_VALUE
for lastActivityTimestamp in the snapshot for sessions as the value should be transient (Cluster). - On snapshot of PendingServiceMessageTracker, check to see if the buffer is empty and the nextServiceSessionId is lower than what it should be. Correct this and log an error that follower may not have executed the service IPC logic deterministically (Cluster).
- Include the recording position in the debug log when replicate session state changes (Archive).
- Don't cool down an Image that is EOS (Driver)
- Reset eos_flagged when adding a receiver (C Driver)
- Filter images by session id when adding new network subscriptions (C Driver)
- Allow the session-id of the publication used during replication to be specified to avoid session-id clashes on replication retries (Archive).
- Allow the authorisation credentials to be specified on a replication request to support simple authentication between the destination and source archives (Archive).
- Use custom session-ids when replicating snapshots and logs (Cluster).
- SBE 1.28.3
- Agrona 1.18.2
- ByteBuddy 1.14.5
- Version 0.47.0.
- [Java] Add debug method for a replication completing withing Cluster.
- [Java] Let Archive take care of handling Aeron exceptions when client is embedded.
- [C] Prevent double free of the
aeron_udp_channel_t
when creating asend_channel_endpoint_t
. - [C] Upgrade to HDR Histogram 0.11.8
- [Java] Upgrade to JUnit 5.9.3 (platform console 1.9.3)
- [Java/C] Name resolution execution time tracking
- [Java] Simplify Client Close
- [Java] Perform Runtime.exit(int) on another thread in default error handler to avoid deadlocks if Aeron instances are used on JVM shutdown hooks.
- [C] Fix potential memory leak. Use correct action with publication notification.
- [Java] Ensure segment file write of final byte for extension is successful
- [Java] Fixed ClusterBackup backwards compatibility issue caused by the missing
memberId
field. - [Java] Set isLeader flag on the list of cluster members passed to
ClusterBackupEventsListener
. - [C] Prevent negative shifts when receiving some messages from ATS.
- Upgrade to ByteBuddy 1.14.4.
- Upgrade to Gradle 8.1.
- Upgrade to Shadow 8.1.1.
- Upgrade to Agrona 1.18.1.
- Upgrade to SBE 1.28.2.
- Allow
NameResolver
to be configured for theConsensusModule
in order to support custom name resolution when configuring the ingress channel. - Delay election state transitions if there is an active leader to avoid unnecessary reset and new election.
- Make
AeronCluster.asyncConnect
work completely asynchronously. Don't report exceptions to the error handler that are used for async resources. - Add a system property and API to allow changing a directory where an Archive mark file (
archive-mark.dat
) is stored. - Check the state of the interface when trying to resolve the multicast interface. Only use interfaces that are up. Issue #1387
- CnC file length validation. Issue #1410
- Fix issue of not capturing return code when recording signal arrives after an error to the archive client.
- Support migrating segments to the beginning or end of an existing archive recording.
- [C] Fix issue of using transport after it had been removed.
- [Java] Fix concurrent close of receive destination counters on multi-destination subscriptions.
- [C] Fix
remove_if
methods on pointer value maps which previously could miss an item. - Add debug logging for clustered service acking.
- Add a specific error for archive replication failing to create a remote connection.
- Fix leak with Archive replay session if the async publication has a session clash.
- Shorten duration of cluster election after a leader has closed gracefully.
- [C] Fix image rejoin by swapping correcting cooldown map insertion and removal. PR #1338
- Candidate ballot for 5+ node cluster cannot be cut short on quorum otherwise most up to date member may not be elected.
- [C] Allow for attempted recreation of an Image if initial attempt fails. PR #1435
- Perform most replay validations before sending OK to the client so errors are synchronous when starting a replay.
- Delete all recording segment files when a recording is truncated to its start position.
- Close
ArchiveMarkFile
last when shutting down Archive to capture all errors. - [C++] Apply
std::forward
to fragment handler to avoid unnecessary copy. PR #1405 - Fix handling of padding greater than max message length in Archive replay.
- Add debug logging for Archive recording signals.
- Close log subscription first when clustered service is cleanly closed to drop follower out of flow control as soon as possible.
- Drop cluster follower as soon as possible out of flow control to allow cluster to progress when follower is cleanly closed.
- [C] Report timeout accurately when driver keepalive beyond timeout. PR #1429
- Add ability to run Archive with only IPC control channels for clients.
- Add
ClusterTool.isLeader
method. - Add
Image
toSubscription
before calling available handler rather than after. - Set URI in receiver counters to match subscription channel.
- Add cluster member node state file and migrate out state that needs to be persistent, such as
candidateTermId
and member list, so the mark file can be in /dev/shm. - [C] Fix issue with removing naming resolver neighbor that deleted adjacent memory.
- [C] Improve socket error handling on Windows.
- [Java] Add
toString()
to many Aeron classes to help debugging. - [C] Improve parsing of unsigned 32-bit integers.
- [C] Set max of resource free queue length and resource free limit to
INT32_MAX
. This stops them being incorrectly set to 0 by aeron_config_parse_uint32 when comparing against int32 0. PR #1421 - Deprecate cluster dynamic join feature. This is to be replaced with a more robust and user friendly premium offering.
- [C] Fix counter leak when subscription fails.
- [C] Fix spy channel memory leak when destination is removed for multi-destination subscription.
- [C] Fix channel memory leak on error when creating publications or subscriptions.
- Fix NPE on timeout exception for cluster client in some connect states.
- [Java] Improve efficiency of URI parsing.
- [C] Fix error messages with incorrect varargs.
- Warnings clean up in codebase to have less noisy CodeQL analysis.
- Support having mark files for
Archive
,ConsensusModule
, andClusteredServiceContainer
to be in alternative directory such a /dev/shm so timeouts can be avoided when recording writes queue up on a network filesystem. - Add timestamp params to stripped channel for pass through to Archive operations.
- Queue resource freeing operations in driver to avoid timeouts when unmapping operations are slow.
- [C++] Work around compiler concurrency bug for
AtomicArrayUpdater
that can impact client Subscriptions causing image list to become corrupted. - Improve javadoc for recording signal usage.
- Be strict on handling cluster leader liveness to the current leadership term.
- Only try unblocking a client command after liveness timeout to avoid "lost" commands. PR #1369
- Make archive counters unique so multiple archives can run on the same media driver.
- Truncate files after
ArchiveTool.compact
is invoked to free disk space. - Fix basic auction cluster tutorial configuration.
- Improve
ClusterConfig
sample to allow for ingress configuration. - Add counters for the number of active recordings or replays in an Archive.
- Add counters for reporting on read and write operations in an Archive.
- Support allowing a
ClusteredService
being started before theConsensusModule
. - Improve false sharing protections for more consistent latency.
- Simplify
ReplayMerge
samples to not require entity tags. - Add batch script for launching low-latency media driver on Windows.
- Support message lengths greater than MTU in ping pong samples.
- Fix options handling in
cping
sample. - Improve handling of timeouts in cluster elections for more robust state transitions when network is unstable. Effects are more pronounced in 5+ member clusters.
- [Java] Add
Aeron.addAsyncSubscripiton
for non-block setup. - Compute source identity of images more precisely based on channel configuration.
- Improved handling of out of disk space errors.
- Support taking a cluster consensus module snapshot when member names are greater than MTU in length.
- Allow a follower to veto a member being elected cluster leader if they believe the leader is not valid. This is important in 5+ node clusters.
- Extend debugging for voting in cluster elections.
- Increment error counter when invalid version exceptions occur.
- Handle backpressure from commands between dedicated threads in driver with controlled polls to avoid live locks.
- [C] Add support for controlled poll operations on SPSC and MPSC ring buffers.
- Increase command queues to allow for more concurrent active changes in publications and images.
- Serve cluster backup queries from followers to take load from the leader.
- [C] Fix build when dot is used as thousands separator. PR #1372
- Upgrade to JUnit 5.9.2.
- Upgrade to BND 6.4.0.
- Upgrade to ByteBuddy 1.14.3.
- Upgrade to Mockito 4.11.0.
- Upgrade to Version 0.46.0.
- Upgrade to Gradle 7.6.
- Upgrade to SBE 1.28.1.
- Upgrade to Agrona 1.18.0.
- Memory align allocated buffers in
PublicationTest
so it works on Apple M1 processors. - Check that
NoOpLock
is only allowed to be used when using Aeron client in invoker mode. - Handle case of a delayed concurrent offer to a publication in which other threads have raced terms ahead without throwing an exception.
- Collapse term appenders into publications to reduce memory footprint and avoid data dependent loads.
- Short circuit Image polling operation when bound limit is less than current position to prevent term overrun.
- Add different aliases for consensus module/service container subscriptions. PR #1366.
- Stop an active cluster log replay when
ClusterBackup
is closed rather than waiting for timeout. - Send unavailable counter events to Aeron clients when a client closes or times out.
- Allow Consensus Module Agent to be run via an Invoker in addition to having its own thread.
- Apply liveness checks to Archive and Cluster mark files so that multiple instances cannot be run in the same directory and corrupt files.
- [Java] Use fixed format for timestamps in agent debug logs.
- Allow Archive replicate to overwrite all metadata for an empty recording.
- [C] Handle log buffer files with
term_length == AERON_LOGBUFFER_TERM_MAX_LENGTH
on Windows. PR #1360. - [C] Fix inclusion of symbols for debug builds on Windows.
- Remove
localhost
defaults for Archive and Cluster to help avoid mis-configuration in production. PR #1356. - Await 'REPLICATE_END' when catching up as a follower across multiple leadership terms to avoid clashing session-id.
- Allow setting of receive socket buffer and window on cluster log channel subscribers. PR #1345.
- Fix application of send socket buffer lengths as configured when using MDC.
- Fix
ArchiveTool.dump
when fragment length is set <= 0. - Capture closing sessions into snapshot so session close event is lost on cluster shutdown.
- Remove brackets from counters labels to make it easier for extract to Prometheus.
- Send cluster client session open acknowledgement before appending to the log to avoid race with service sending egress on open event. Issue #1351.
- [C] Fix off by one error local socket address into channel indicator counter.
- Add protocol version support to cluster consensus protocol.
- Add more context to error messages on Archive
ReplaySession
. PR #1349. - Apply strict validation of consensus module snapshot state when messages are offered from clustered services. A number of customers have not been strict with all cluster nodes being deterministic and doing exactly the same thing which can result in corrupted and diverged snapshots.
- Consensus module state snapshot can be inspected with the
describe-latest-cm-snapshot
option toClusterTool
. - If a consensus module snapshot is shown to be corrupt it may be fixed by running
ConsensusModuleSnapshotPendingServiceMessagesPatch
and if non-support customers wish to have help then they can contact sales@aeron.io. The patch can fix the leader and the fixed snapshot then needs to be replicated to the followers which can be done withAeronArchive.replicate
using the correct recording ids. - Add a tool to replicate a specific recording between archives. PR #1363.
- [C++] use
getAsString
calls for pollers for record descriptors for channel fields. Add test from PR #1348. - Add
ClusteredService.doBackgroundWork
which can be used for maintaining external connections beyond ingress and egress. - Increase default message timeout from 5 to 10 seconds for Archive clients.
- Add EOS flag to status messages (SMs) once a stream is totally received so the sender can take clean up action.
- When EOS status message is received by a sender then allow the publication linger on unicast to be cut short so resources are received sooner.
- When EOS status message is received by a sender then remove the receiver from flow control for multicast and MDC with tagged and min FC.
- Fix the closing of session specific subscriptions to prevent resource leak.
- Add scripts for testing raw network performance on Windows.
- Close egress from cluster on change of leader so clients can detect it before a new leader is elected.
- Don't timeout and close cluster client session if quorum cannot be temporarily reached.
- Add logging support for
ClusterBackup
state changes. - Close cluster clients when complete cluster is restarted.
- Support automatic reconnect from cluster client when the same leader is re-elected after a net split or temporarily loosing quorum.
- Add authentication for
ClusterBackup
to a cluster. - Validate Archive mark file length before reading when mapped read-only to avoid access violations.
- Preserve iteration order for cluster client session based on session id so snapshots can have binary compatibility.
- Capture leadership term id for cluster backup queries.
- Account for padding when sweeping pending services messages to avoid out of bounds exception.
- Prevent
-1
leadership term ids appearing in theRecordingLog
. - Allow Archive replication and replay request to specify session level file IO max buffer length for throttling a stream.
- Add support for custom app version validation to clustered services with
AppVersionValidator
. - Add false sharing protection to
DutyCycleTracker
. - Update doc on
ReplayMerge
to indicate theAeronArchive
client should not be shared. Issue #1340. - Upgrade to Versions 0.43.0.
- Upgrade to Mockito 4.8.1.
- Upgrade to Google Test 1.12.1.
- Upgrade to JUnit 5.9.1.
- Upgrade to ByteBuddy 1.12.18.
- Upgrade to Gradle 7.5.1.
- Upgrade to SBE 1.27.0.
- Upgrade to Agrona 1.17.1.
- [Java] Fix
IllegalStateException
that could exist for an MDS subscription on the rapid recycling ofReplayMerge
operations. - [C] Align ring buffer implementations and feature set with Java.
- [Java] Make sure that C and Java are aligned on resend window. Re-instate the max message length being accounted in the bottom of the resend window for Java.
- Add duty cycle duration tracking to all agents across all modules.
- [C++] Improve efficiency by reducing the number of copy operations for fragment assembly when a stream has many fragmented messages.
- [C] Default to
CLOCK_REALTIME
for send/receive timestamps. - [Java] Add setters for send/receive timestamp clocks to the
MediaDriver.Context
. - Fix handling of fragment assembly when
reliable=false
is set for a channel and loss occurs. - Improve handling of short sends on MDC publication to backoff from overloading a socket.
- Add round-robin facility to MDC publication for increased fairness.
- [Java] Publish
aeron-test-support
package as a JAR. - [Java] Downgrade "unknown replay" errors to warnings for cluster catchup.
- [Java] Add
appVersion
to event logging for consensus module and check for correct app version when replaying log. - [Java] Prevent timeout warnings with cluster dynamic nodes and log replication.
- [Java] Add cluster dynamic join state change logging events.
- Add counters for the number of receivers in min and tagged flow control strategies.
- [Java] Avoid race unmapping buffers on concurrent close of media drivers.
- Modify flow control strategies to have new method for when elicited setups are sent and add counters manager to
init
methods. Modify Min and Tagged flow control to use setupsnd-lmt
as min position until timeout or receiver added on SM. - [Java] Account for possible padding in log buffer when checking for bottom resend window for retransmits.
- [C] Flush output when printing configuration.
- [C] Raise warning on failure to setup media timestamping.
- [Java] Update
recordingId
on any signal with a valid recording id when handling signals for snapshot replication. - [Java] When attempting
ClientSession.tryClaim
, ensure that there is enough buffer space when returning a mocked offer for a follower. - [C] Ensure publication image is released before it it freed.
- [C] Fix
scanf
that could result in buffer overflow when parsing HTTP for configuration. - [Java] Change default cluster session timeout from 5 to 10 seconds.
- Prevent receiver joining min/tagged flow control if they are more than a window behind.
- [C] Add sample for working with large messages.
- [Java] Add logging event for appending a cluster session close.
- Upgrade to BND 6.3.1.
- Upgrade to Mockito 4.6.1.
- Upgrade to ByteBuddy 1.12.10.
- Upgrade to SBE 1.26.0.
- Upgrade to Agrona 1.16.0.
- [C] Driver - Ensure the correct control address is used when adding multicast destinations with MDS.
- [C] Driver - Allow thread affinity on CPU 0.
- [C] API - Check handler parameter before polls. Check images for NULL before polling images.
- Upgrade to SBE 1.25.3.
- Upgrade to Agrona 1.15.1.
- [Java/C/C++] Ensure driver is in ready state when requesting termination from client.
- [Java] Reduce allocation when listing archive directories to find segment files.
- [Java] Add flag to
ClusterTerminationException
to indicate if the termination was expected. - [Java] Expand agent logging for consensus module operations, be careful if using
all
for cluster events as volume may now be greatly expanded. - [C] Use connect and send to improve latency in C driver when sending data at lower volumes.
- [Java] Improve reliability of transferring snapshots to
ClusterBackup
via archive replication with improved re-try semantics. - [Java] Support adding an IPC ingress destination to cluster leader for ingress optimisation.
- [Java] Create replay publication asynchronously to reduce latency pauses in Archive.
- [Java/C++] Add new
RecordingSignal.REPLICATE_END
recording signal to indicate end of a replication operation. - [Java/C++] Make delivery of
RecordingSignal
s to archive client sessions reliable and ordered. - [Java] Support specifying interface with endpoints in cluster config for multi-home members. PR #1290.
- [C] Add thread affinity support to C media driver. PR #1298.
- [C/C++] Update CMake build to use
FetchContent
instead ofExternalProject
. - [C/C++] Fix build on ARM with clang. PR #1291.
- [Java] Improve progress tracking and retry semantics for cluster members catching up in elections.
- [C/C++] Enable support for parallel build on Windows.
- [Java] Add ability to async remove/close a publication by registration id.
- [Java] Fix publication leak in
ClusterBackup
when backup response timesout. - [C] Improve agent logging in C media driver to be more consistent with Java driver.
- [C] Allow for configurable IO vector for
sendmmsg
andrecmmsg
in the C media driver. PR #1285. - [C] Support static linking of the C media driver. PR #1261.
- [Java/C] Support ability to extend concurrent publications by setting initial values to be equivalent to exclusive publications.
- [Java] Fixed bug in
PriorityHeapTimerService.cancelTimerByCorrelationId
. PR #1281. - [C++] Improve error reporting in Archive client when a response is not received.
- [Java/C++] Additional user specified delegating Invoker for Archive client to be used for progressing actions when awaiting responses.
- [Java] Rename Archive segment files before delete to avoid races with streams being extended.
- [C++] Fixes for
ChannelUriStringBuilder
. PR #1268. - [Java] Add admin command so that cluster snapshot can be triggered remotely via an authorised session.
- [Java] Support authorisation of service actions with a new API
AuthorisationService
. The hooks for this have been added to Archive requests and Cluster Snapshot requests. - [Java/C] Support adding spy and IPC destinations to MDS subscriptions so destinations can be all channel types.
- [Java] Ensure Cluster will start on a consistent initial term id when racing to create first term.
- [Java] Prevent unnecessary creation of
RecordingLog
files when usingClusterTool
. - [Java] Add cluster session timeout to set adjusted when debugging.
- [C] Fixes to prevent message duplication and unnecessary sending of messages in MDS.
- Minimum CMake version was raised to 3.14.
- Upgrade to HdrHistogram_c 1.11.4.
- Upgrade to BND 6.2.0.
- Upgrade to Versions 0.42.0.
- Upgrade to Mockito 4.4.0.
- Upgrade to ByteBuddy 1.12.9.
- Upgrade to Shadow 7.1.2.
- Upgrade to Gradle 7.4.2.
- Upgrade to JUnit 5.8.2.
- Upgrade to Checkstyle 9.3.
- Upgrade to SBE 1.25.2.
- Upgrade to Agrona 1.15.0.
- [Java] Improve error messages on channel conflicts.
- [C] Remove replicated command prefix in debug agent logging.
- [Java] Use async publication add for async connect to an Archive to minimise the impact of name resolution pauses.
- [Java] Make
ClusterConfig.calculatePort
public. - [C] Correct channel length on metadata for stream counters.
- [Java] Extract channel value from counter label when longer than what will fit in metadata for
StreamStat
. - [Java] Relocate HdrHistogram and ByteBuddy in
aeron-all
JAR. - Upgrade to BND 6.1.0.
- Upgrade to ByteBuddy 1.12.2.
- Upgrade to Mockito 4.1.0.
- Upgrade to SBE 1.25.1.
- Upgrade to Agrona 1.14.0.
- [C/C++] Handle SIGINT in code samples.
- [Java] Retry adding cluster member publication in election canvass to address late name registration in containers such as Kubernetes.
- [Java] Log resolution failures in Cluster as warning event rather than exception.
- [Java] Fix timestamp when publishing new leadership terms. PR #1254.
- [C] Use separate transport bindings for the conductor doing name resolution. PR #1253.
- [Java/C++] Allow the setting of a
RecordingSignalConsumer
in the archive client context which is delegated to when processing control channel responses. - [C] Improve error handling and logging on Windows when dealing with network system calls.
- [Java] Verify cluster log is always contiguous when joining a new image in a service.
- [Java] Fix race condition when sending
RecordingSignal.SYNC
during archive replication. PR #1252. - [Java/C] Improve choice of subscription for choosing channel URI when labelling receiver counters.
- [Java] Sort counters displayed with
StreamStat
so they are logically grouped. - [Java] Improve error messages so they are more contextual.
- [Java] Extend debugging logging for archive and cluster operations.
- [Java] Check for errors when cluster snapshots are replayed.
- [Java] Improve tracking of cluster commit position when replicating during an election.
- [Java] Allow replication to skip over empty leadership terms due to failed elections when initially starting cluster.
- [C] Better handling of finding user for default
aeron.dir
whenUSER
is not set in environment. - [Java/C++] Reduce cache invalidations when using pollers for archive and cluster response streams.
- [Java] Add support for changing cluster log params by truncated to the latest snapshot and resetting configuration. PR #1233.
- [Java] Don't catch subclasses of
Throwable
and instead catchException
so that the JVM can handle subclasses ofError
. - [Java/C] Improve validation of ports used in channel URIs.
- [C] Support building on Apple ARM.
- [Java] Add priority heap backing implementation for cluster timers as an alternative to the default timer wheel implementation
- Upgrade to Mockito 4.0.0.
- Upgrade to Shadow 7.1.0.
- Upgrade to BND 6.0.0.
- Upgrade to Gradle 7.2.
- Upgrade to ByteBuddy 1.12.1.
- Upgrade to Checkstyle 9.1.
- Upgrade to SBE 1.25.0.
- Upgrade to Agrona 1.13.0.
- [Java] Fix selection of channel based on add publication registration id rather than original registration id. Issue #1218.
- [Java] Fix truncation of linger timeout in
ChannelUriStringBuilder
which lead to a short linger of Archive replays. - [Java] Remove incorrect publication linger validation.
- [C] Add sanitize build for MSVC and fix issues found.
- [C] Add missing free of counters associated with Cubic congestion control.
- [C++] Fix missing use of
FragmentAssembler
in Archive response and clean up type warnings. - [Java] Fix packaging declaration in POM file.
- [Java] Javadoc improvements.
- [Java] Separate thread factories for replay and recording agents in Archive for when setting thread affinity is required.
- [C] Agent logging fixes. PR #1198.
- [Java/C] Support a list of bootstrap neighbours for fault tolerance in gossip protocol for driver naming.
- [C] Handle connection reset without error when polling a socket on Windows.
- [C++] Don't progress with archive connect until response subscription is available. PR #1196.
- [Java] Use async publication adding for response channels from the Archive and response channels for egress and backup queries from the Cluster to reduce latency pauses for existing operations.
- [Java] Ability to add publications asynchronously to Aeron client.
- [C/Java] Support timestamping of packets for channel send and receive plus media/hardware receive timestamping if supported. PR #1195.
- [Java] Ensure termination hook is run on unexpected interrupt during cluster election.
- [Java] Reset cluster election state if in election and an exception happens outside the election work cycle.
- [Java] Finish deleting pending archive recording for deletion on shutdown.
- [Java] Ensure cluster log recording has stopped before restarting the election process to avoid spurious election failure from past recording stopping.
- Upgrade to Google Test 1.11.0.
- Upgrade to Mockito 3.11.2.
- Upgrade to ByteBuddy 1.11.9.
- Upgrade to Gradle 7.1.1.
- Upgrade to SBE 1.24.0.
- Upgrade to Agrona 1.12.0.
- [Java]: added nanoClock to
AeronArchive.Context
to control time more directly. PR #1188. - [Java]: added
ClusterBackup.Context.toString
. - Various changes for Cubic congestion control, Status Message generation, and overrun determination to handle high loss scenarios with congestion control better.
- [Java]: use separate archive contexts for local and remote archive clients in cluster backup. Local archive must be configured to use IPC.
- [Java]: relocated ByteBuddy in agent jar.
- [Java]: support constructing a
ChannelUriStringBuilder
from an existing URI. PR #1186. - Several improvements to handling initial name resolution failures for cluster and cluster clients when using name resolution from containers.
- [Java]: improve tag usage for
IndexedReplicatedRecording
example. - [Java]: more information included in
extendRecoding
failures. - Added name resolution logging to agents.
- Append cycle time threshold to counter label.
- [Java]: support connecting to a cluster when a minority of the members are not active in a name service.
- [C]: retain entropy in large collections for hashing and include full range of possible masks for UINT32.
- [Java]: timeout Archive replication if recording subscription endpoint fails to resolve.
- [Java]: added
AeronEvent
exception type that does not generate a stack trace. - MDC manual destinations can now be initially unresolved.
- [Java]: Fix NPE on cluster client after multiple redirects. PR #1179.
- [C]: improve common hash functions. PR #1178.
- Various fixes for re-resolution of endpoints and adding more tests to re-resolution scenarios.
- [C]: add interface URI param to MDC publication channels.
- [Java]: MDS will now use the base subscription URI for congestion control, receive window, and socket buffer URI params.
- Upgrade to SBE 1.23.0
- Upgrade to Agrona 1.11.0
- Upgrade to Versions 0.39.0
- Upgrade to Unit 5.7.2
- Upgrade to Gradle 7.0.2
- Upgrade to Shadow 7.0.0
- Upgrade to Mockito 3.10.0, then to 3.11.1
- Upgrade to ByteBuddy 1.11.0, then to 1.11.2
- [C] Fix clean up in CSV name resolver on error.
- Improve error messages for channel URI configuration and clash errors.
- [Java/C++] Add missing arguments for full replicate and tagged replication API to Archive.
- [Java] Avoid channel leak on error configuring send and receive channel.
- [Java] Avoid double suffix of exception category to message for
RegistationException
. - [Java] Allow setting of socket and receiver buffer lengths in
ChannelUriStringBuilder
fromChannelUri
with short form human friendly names.
- The focus for this release has been a significant rework of cluster to make consensus more robust especially in recovery scenarios. We consider this the penultimate release to cluster being GA. As of this release we plan to stabilise the API and only make breaking changes if a significant issue is raised by a customer on commercial support.
- Expand the range of channel URI params supported by the archive on a per stream basis.
- Add support for dynamically switching debug logging on and off. PR #1155.
- Add debug logging support for flow control.
- [C] Fix memory leak and reassembly of fragmented message greater than 8K in client.
- Fix short send of recording start event when tracking recording progress. PR #1155.
- Improve clean up of subscriptions and control sessions in the Archive when failures occur.
- [Java] Fix bug with flow control
gtag
being carried over erroneously which can cause issues withReplayMerge
and other features dependent on group flow control semantics. - Reduce the number of memory fences used with
min
andtagged
flow control. - Set initial window segments to 10 for Cubic congestion control and fix issue measuring RTT in the presence of loss.
- Add the ability to configure archive replication channel on a per operation basis. This enables the setting of congestion control and socket buffer lengths which are important for cluster backup.
- Use Archive replication for cluster replication, dynamic join, and cluster backup. This requires the cluster and archive config to be correct as configuration errors will not be evident until used - be careful of using localhost for endpoints.
- Check tag for match when reusing send channel endpoint. PR #1147.
- Add the ability to configure socket buffer and receive window on a per channel basis. PR #1143.
- Rework Cluster backup and dynamic join to use archive replication.
- Add support for using a 0 port for cluster catchup endpoints.
- [Java] Better clean up of allocated resources in the driver when failures occur so it can continue without leaks.
- [Java] Reduce linger on explicitly closed resources in the client.
- [C/C++] Improve the performance of pre-faulting memory mapped file on Linux and Windows. PR #1127.
- [C/C++] Clean up warnings in Windows build.
- Improve Javadoc.
- Provide sender and receiver with their own cached clocks to be more responsive and isolated from conductor stalls.
- Add new counters to detect work cycle stalls which track max work cycle latency and count of threshold exceeded observations.
- Continue to send status messages and heartbeats when running in
DEDICATED
orSHARED_NETWORK
thread modes to keep connections alive if the driver stalls due to DNS lookups or file IO. - Reduce the number of commands from client from 10 to 2 per work cycle to help prevent timeouts and reduce latency pauses.
- Improve validation of adding destinations to publications.
- Better handling of race conditions when clients and driver are started/restarted at the same time.
- Extend debug logging events.
- Improve diagnostics collection on failed cluster tests.
- Add
disable
event codes for debug logging so all can be enabled and merged with a disabled set. - Add error stacks to C driver to aid debugging of issues.
- Add storage space warnings and specific exception codes on errors returned to archive client. Archive has new config for low storage thresholds.
- Detect archive failures in cluster so appropriate action can be taken.
- Propagate recording errors from the archive back to the archive client that initiated the failed operation.
- Add specialised
ClusterTerminationException
for expected cluster termination. - Reduce network syscalls with Java 11+ for higher numbers of active streams.
- Respond to cluster client with session open event only after the open session is successfully appended to the cluster log.
- Upgrade to Version 0.38.0.
- Upgrade to BND 5.3.0.
- Upgrade to Mockito 3.9.0.
- Upgrade to ByteBuddy 1.10.22.
- Upgrade to JUnit 5.7.1.
- Upgrade to Gradle 6.8.3.
- Upgrade to SBE 1.22.0.
- Upgrade to Agrona 1.10.0.
- Respond to cluster client with session open event only after the open session is successfully appended to the cluster log.
- [C/Java] Fix unexpected image unavailable when a rush of connections comes in for MDC or multicast publication. Issue #1115.
- Increase default flow control receiver timeout from 2 to 5 seconds.
- [Java] Cluster performance improvements.
- [Java] Improve liveness tracking for followers catching up with a cluster leader when service logic is running slow.
- [Java] Configuration option for cluster log consumption fragment limit.
- [Java] Improve coordination of Cluster services during an election for log catchup and state changes.
- [Java] Rework Cluster elections to better handle edge conditions in resource limited environments.
- [Java] Add multicast support for cluster log channel.
- [C++] Add missing methods to
ExclusivePublication
so it is compatible withPublication
. - [C] Support compatible command line options for the C driver when running on Windows.
- [C] Fix the deletion of directories on driver shutdown when running on Windows.
- [C] Fix the transposed observation times in the loss report.
- [C/C++] Migrate the C++ client tools to wrap the C tools for
AeronStat
,DriverTool
,LossStat
, andErrorStat
. - [C] Reduce memory footprint and copying in client when sending driver commands.
- [Java] Delete Archive segments asynchronously when purge, truncate, or delete operations are carried out so that deleting a large number of segments does not block the Archive conductor so that the Archive stays responsive. A new
RecordingSignal
has been added for tracking the completetion of the delete. - [C/Java] Run Cluster system tests against both the Java and C Media Drivers.
- [C/Java] Complete logging and align of feature set with the same configuration that can applied to the Java or C media drivers. PR #1091.
- [C] Support URIs larger than the label length on publications and subscriptions in the C media driver to be compatible with the Java media driver.
- [Java] Add Java 16-ea to the test matrix.
- [Java] Improve tracking of connection activity to more accurately detect the need for address re-resolution.
- [C++] Improve samples for better usage illustration and error reporting.
- [C] Complete the feature set for the C client so the C++ wrapper client is a pure wrapper, e.g. provide access to a late bound port for a Subscription.
- [Java] Allow the setting of different error handler when polling a
Subscription
, e.g. use aRethrowingErrorHandler
to propagate the error out the caller and stop progress. - [C] Fix throughput issue with C Media Driver debugging logging.
- [Java] Support variable length entries in Archive Catalog and allow for complete purging of old entries. Requires migration. PR #1069.
- [Java] Reduce memory footprint and copying in client for sending driver commands.
- [Java] Improved Javadoc.
- Upgrade to Checkstyle 8.38.
- Upgrade to ByteBuddy 1.10.19.
- Upgrade to Mockito 3.7.7.
- Upgrade to Versions 0.36.0.
- Upgrade to Gradle 6.7.1.
- Upgrade to SBE 1.21.0.
- Upgrade to Agrona 1.9.0.
- Fix bug in C++ client managing images under a subscriptions due to bug with GCC 7.3.1 failing to emit an acquire fence.
- Fix bug with cleaning up log buffers which could result in segfault in native driver.
- Fix bug in C++ client with
putValueVolatile
. - Add
AeronException.Category
name to the beginning of error message to indicate the severity in theDistinctErrorLog
. - Improved Javadoc.
- Schedule Status Messages with more relaxed memory ordering for a ~3% throughput improvement in the Java driver.
- Memory order fix for scheduling NAKs and Status Messages in native C driver.
- Enable higher-resolution timers on Windows for native driver so sleep periods less than 16ms.
- Upgrade to Mockito 3.5.15.
- Handle failed log buffer delete in C media driver on Windows. This can happen when a client holds a mapped file open and the driver tries to delete it. PR #1073.
- Increase default client liveness timeout from 5->10s and publication unblock timeout from 10-15s to be softer on clients that experience bad GC pauses or run in resource starved environments.
- Add C++
ChannelUriStringBuilder#initialPosition
method to set the initial position of a publication. - Add
ownerId
to publication limit counters for being able to track which client created a publication. - Improve javadoc and reduce the scope of some methods that should not have been public.
- Fix C++
AtomicCounter::getAndSet
. - Fix timer cancellation when scheduling in cluster. Issue #1071.
-
ReplayMerge
now substitutes the endpoint from thereplayDestination
into thereplayChannel
to simplify configuration. - Support using a port of
0
on the replay destination forReplayMerge
so that it is assigned by the OS from the ephemeral range. - Support using a port of
0
on the replication channel between archives so that it is assigned by the OS from the ephemeral range. - Fix the ability to add and remove a destination with port
0
to an MDS Subscription. - New subscriptions now late join a stream at the min of existing subscriptions rather than max.
- Fix implementation of
ExclusivePublication::tryClaim
in C++ wrapper client. - Add Cubic congestion control support to the C media driver. PR #1065.
- Default to building the C++ archive client as part of the native build.
- Improve the native Windows build for CLion.
- Remove the need for having 7-Zip installed for native build on Windows.
- Improve error handling for archive errors in the consensus module so warnings can be issued and retried.
- Set media driver heartbeat to
-1
on clean shutdown so it can be immediately restarted without waiting for driver timeout. - Add Clang 11 to build mix.
- Add Java 15 to build mix.
- Change stop replay failures in the cluster from errors to warnings.
- Improve
ExtendRecordingTest
to be a better example. - Fix cluster tutorial scripts.
- Improve samples code.
- Upgrade to Checkstyle 8.36.2.
- Upgrade to Shadow 6.1.0.
- Upgrade to ByteBuddy 1.10.17.
- Upgrade to HdrHistogram_c 0.11.2.
- Upgrade to SBE 1.20.3.
- Upgrade to Agrona 1.8.0.
- Add hooks so ATS (Aeron Transport Security) can be loaded as a premium feature on the C media driver. Issue #203.
- Numerous improvements for the native driver on Windows.
- Further refinement and additions to the C client which is currently at preview status.
- Remove a number of data dependent loads caused by indirection to reduce latency outliers.
- Improve logic for expansion of
BufferBuilder
for fragmented messages to be correct at extremes and to be more efficient. - Set ANY ADDR to correct protocol family for endpoint based on control when IPv6.
- Scope Cluster Backup counters by cluster id.
- Improve Archive client connect error messages.
- Add deadline checking to C++ Archive client connect.
- Improve the efficiency of counter searching.
- Add extra validation for the relationships between timeouts.
- Change tracking of untethered subscriptions so the bottom 1/4 rather then 1/8 of the window is used to make for easier eviction.
- Add registration and owner id to counters to help avoid ABA issues and to aid monitoring.
- Avoid updating the commit position counter when the consensus module is closed.
- Improve active transport tracking to be more timely and accurate.
- Make use of cached clocks when referencing counters to reduce system call overhead.
- Improve
ReplayMerge
tests to show a better example of usage. - Add driver and hostname to re-resolution counter for Java and C media drivers.
- Fix memory corruption with driver naming resolution events in C media driver.
- Fix dynamic agent dissector logging for C media driver.
- Improve liveness tracking for channels to reduce overhead and false sharing in the Java Driver.
- Add a
ChannelUriStringBuilder.toString()
method. - Provide a registration id on the add and remove handler methods in clients so they can be removed by the registration id and not rely on the pointer or reference to the callback.
- Allow the setting of port 0 on Archive and Cluster control response channels for clients so they are automatically allocated from the ephemeral range.
- Improve native code use of atomics across all platforms and especially on Windows.
- Improve error messages in the native driver to help indicate which is the offending command and URI.
- Auto resize the Archive Catalog when full so the Archive does not need shutdown and manually extended.
- Improve startup code for all clients finding a running media driver which is racing to start at the same time.
- Support the C++ Archive client on Windows.
- Set CMake 3.6.1 as the min required version.
- Upgrade to JUnit 5.7.0.
- Upgrade to HdrHistogram_c 0.11.1.
- Upgrade to Version 0.33.0.
- Upgrade to Checkstyle 8.36.
- Upgrade to Gradle 6.6.1.
- Upgrade to Mockito 3.5.10.
- Upgrade to ByteBuddy 1.10.14.
- Upgrade to BND 5.1.2.
- Upgrade to SBE 1.20.2.
- Upgrade to Agrona 1.7.2.
- Further refinement and additions to the C client which is currently at experimental status.
- Improve error messages when parsing URI params.
- Fix application of sparse terms in Java Media driver when not used on a per channel basis.
- Add support for session based subscriptions on IPC and spies to the C media driver.
- Use
ssc
(Spies Simulate Connection) only in cluster when membership size is 1. This avoids the leader racing ahead of followers which are catching up and a number of cases where the start of a recording can be missed. - Add the ability to have spies simulation connection (ssc) configured on a per stream basis for both Java and C media drivers.
- Fix some false sharing issued introduced for channel re-resolution checking to give a tighter latency distribution.
- Add state checks to Cluster operations so services do no use features at inappropriate times.
- Rework build script to help IDEA recognise generated classes and not give false compilation errors.
- Significantly improve throughput of C media driver when used with the Solarflare ef_vi premium extension to provide the best latency and throughput possible.
- Fix short send counting in C media driver.
- Change Archive session workers to behave more like normal Agents so that stack traces are more informative when debugging.
- Improve error handling and cluster elections when dynamic membership is being used and increase test coverage.
- Improve session checks when re-adding a publication with the same session id.
- Refinements to Cluster Backup.
- Change defaults for throughput tests to use 8k rather than 16k MTUs to better fit with jumbograms.
- Close recording Archive recording subscriptions with
autoStop = true
that have an error on first image. - Detect Archive errors in Cluster so waiting operations can abort and be retried.
- Fix
aeron_ftruncate
on Windows for native driver so it behaves more like Linux. This addresses races with client and driver starting at the same time which can result in a corrupt CnC file. - Avoid int overflow with Cluster snapshots greater than 2GB in length. PR #959.
- Fix C++ client compile for CentOS 7 with GCC 4.8.5.
- Add flow control (fc) and group tag (gtag) URI params to Archive stripped channels.
- Configurable buffer length for Archive record and replay file operations to control batch size via
aeron.archive.file.io.max.length
. New default shows a marked increase in throughput and reduced latency in all our tests. - Capture logs from failed Cluster tests to aid debugging.
- Agent logging for untethered subscription state changes in Java and C media driver.
- Expanded agent logging for archive activities to aid debugging.
- Fix segfault in C media driver if transport cannot bind.
- Add Java 14 to CI.
- Add native sanitize builds to CI.
- Upgrade to Versions 0.29.0.
- Upgrade to Checkstyle 8.34.
- Upgrade to Mockito 3.4.4.
- Upgrade to BND 5.1.1.
- Upgrade to ByteBuddy 1.10.13.
- Upgrade to HdrHistogram 0.11.0 for C.
- Upgrade to Gradle 6.5.1.
- Upgrade to SBE 1.19.0.
- Upgrade to Agrona 1.6.0.
- Fix issue with replaying cluster log when a snapshot is invalidated after a clean termination.
- Correct arguments to
onReplayNewLeadershipTerm
which got transposed in 1.27.0 release. - Validate lower bound of MTU in config so payload must have some contents.
- Fix race condition when calling size on C queues.
- Remove clashing non
const
ExclusivePublication::channelStatus()
method. Issue #946. - Upgrade to SBE 1.18.2.
- Upgrade to Agrona 1.5.1.
- An experimental C API client is now available. We are happy to take feedback but be aware the API is subject to change as it gets refined.
- Cluster has changed status from experimental to being a preview feature. Many refinements and bug fixes have been made to cluster in the last few months as a result of significant destructive testing. The API is now stable as of this release and will only change before going GA if a significant issue is found. Support is commercially available.
- Correct implementation of Cubic congestion control implementation to align with spec.
- Add support to the C media driver for session-specific and multi-destination subscriptions (MDS), plus complete the functionality so the C media driver can support
Archive
. - Support using
0
for port onendpoint
orcontrol
so OS assigns the port without conflict and then make it available onPublication
orSubscription
via each getting a newlocalSocketAddresses()
method. Local socket addresses also get their own counters. - Reduced CPU time spent scanning for loss in Java and C drivers so they can scale to a larger number of connections.
- Apply consistent approach to merge window for
ReplayMerge
, Archive replication, and Cluster catchup. - Add the ability to stop a recording by recording identity when the recording id is known.
- Use CRC if configured and any possible data to help recover last fragments in a recording that may straddle a OS page after an unclean Archive shutdown.
- Support common short name alias for idle strategies in config for both Java and C media driver such as
noop
,spin
,yield
, andbackoff
. - Update false sharing protection to support Java 15 class layout and add it to
ExclusivePublication
. - Improve Java and C++ samples so they are up to date and give more consistent performance numbers.
- Java client close operations for publications, subscriptions, and counters now happen asynchronously so the client does not wait for acknowledgement. This allows for more rapid close of resources.
- Add notifications for client heartbeat counters becoming available and unavailable so Aeron clients can be tracked.
- Allow for race in creating a new recording in catalog and first segment being written which can happen when a replay is set up right after a recording starts.
- Upgrade to javadoc-links 5.1.0.
- Upgrade to ByteBuddy 10.10.1.
- Upgrade to JUnit 5.6.2.
- Upgrade to Gradle 6.4.1.
- Upgrade to SBE 1.18.1.
- Upgrade to Agrona 1.5.0.
- Drivers can be named and names are gossiped between drivers so that they can be used to simplify configuration for endpoints. Driver Name Resolution.
- Fix header file dependencies for C++ archive client.
- Spy subscriptions can now match on channel tag for publications.
- Multicast flow control is selected when using manual or dynamic MDC (Multi-Destination-Cast).
- Add
tryStopRecording
methods to the archive clients so they can be called without raising an exception if no recording is active. - Add a counter for the number of active control session on the archive.
- Add
autoStop
overload when starting a recording in the archive so it is automatically cleaned up when the first matching recordings stops. - Resend recording progress events after back pressure to detect tail progress.
- Improve URI channel parsing validation. Issue #887.
- Reduce allocation when churning publications.
- Add CentOS 7 build to CI.
- Upgrade to BND 5.0.1.
- Upgrade to Junit 5.6.1.
- Upgrade to Gradle 6.3.
- Upgrade to SBE 1.17.0.
- Upgrade to Agrona 1.4.1.
- Add correlation-id to
ArchiveException
and provide the ability to get the last used correlation-id inAeronArchive
client. - Add re-resolution of endpoints when they timeout and become unconnected which can happen when machines migrate in a cloud environment to the Java driver.
- Add
TaggedMulticastFlowControl
and ability to configure flow control via URI params for Java and C media drivers. - Deprecate
PreferredMulticastFlowControl
. - Fix mutexes for the C media driver on Windows. PR #867.
- Fix handling of sockets the C media driver on Windows. PR #866.
- Fix thread handling for the C media driver on Windows. PR #864.
- Fix mmap on Windows for the C media driver. PR #865.
-
SetWaitableTimer
expects a duration in 100-nanosecond intervals on Windows in C media driver. PR #868. - Fix NPE when
-checksum
flag is not used, and validation Checksum classname if it is used withArchiveTool
. - Deal with asynchronous errors from the archive when replicating or Replay Merge.
- Fixes for Windows C driver. PR #861.
- Warnings clean up in native code.
- Fix socket close on Windows for C driver. PR #857.
- Fix getting a random value in C driver on Windows. PR #854.
- Reduce allocation of direct buffers in the archive to minimum of what is required depending on configuration.
- Improve archive behaviour from unexpected outcomes of file read operations.
- Migrate to Gradle maven-publish plugin.
- Improve closing of resources in aborted or interrupted operation for Java client and modules.
- Fix unexpected unavailable image which could occur with mixed use of wildcard and session specific subscriptions on the same channel.
- Fix deadlock which could occur in C++ client if destroyed too quickly after creation. Issue #844.
- Improve performance of Archive replay. Gains are 25%-50% depending on message length and platform.
- Add client shared library support to C++ client. PR #836.
- Only use MDS for archive replicate when joining a live stream or using a tagged subscription. This allows for multiple concurrent replication streams of recordings which are not joining live or being tagged.
- Make receiver id channel endpoint specific so multi-destination subscriptions get flow controlled independently as they use different sockets. This results in less loss when using Replay Merge.
- Improve performance of logging agent to file by batching event writes.
- Upgrade to Gradle 6.2.1.
- Upgrade to Versions 0.28.0.
- Upgrade to Mockito 3.3.0.
- Upgrade to HdrHistogram_c 0.9.13.
- Upgrade to BND 5.0.0.
- Upgrade to SBE 1.16.3.
- Upgrade to Agrona 1.4.0.
- Log to ring buffer with zero copy semantics for improved logging performance. PR #831.
- Retain file handle after establishing mapping in Windows C++ client. Issue #826.
- Improve encoding performance of logging to file.
- Log all events in a consistent manner with standard header.
- Be consistent with the use of positional reads and writes in the archive for supported OS synchronisation and slightly improved performance.
- Configure Java
DistinctErrorLog
to be US-ASCII rather than UTF-8 for compatibility with native driver. - Run slow tests daily in CI.
- add
GNU_SOURCE
to clock for native builds on CentOS. - Upgrade to Agrona 1.3.0.
- Upgrade to SBE 1.16.1.
- Upgrade to JUnit 5.6.0.
- Where possible only weave in logging hooks when enabled in the Java driver. This can help performance for those who are only logging a few events.
- Add ability to log the control channel responses from the Archive.
- Fix issue with truncating recordings when truncate position equals stop position and start of segment to ensure file is deleted.
- Fix issue with unaligned access to fields in
LossReport
. - Introduce interceptor bind framework to C driver for supporting loss testing, logging, and media layers other than BSD sockets.
- Apply system tests to C driver when running in CI. When apply this a number of bugs got fixed in the C media driver.
- Move CI from Travis to GitHub Actions and test on Windows, Linux, and OSX.
- Support for agent logging in the C driver to file to match Java with the
aeron.event.log.filename
. - Support for adding checksums to archive recordings as CRCs which can be verified to detect file corruption.
- Add support for applying and verifying checksums to recordings via
ArchiveTool
. - Add support for fixing recordings after after a system crash running an Archive.
- Improve crash recovery for the archive when restarting.
- Add cached clocks to C media driver to reduce the overhead of clock calls and improve performance, especially in cloud environments. Issue #606.
- Fix thread local storage for Windows C media driver. PR #795.
- Fixes for Windows C media driver. PR #794.
- Improve EOS reporting in
Image.toString()
method. PR #792. - Fix recovery of stop position in crashed archive when start position was non-zero.
- Provide API for for features that existed in
CatalogTool
in newArchiveTool
. - Don't linger replay publications in
ReplayMerge
so resources can be reclaimed sooner. - Default warning of Aeron direction existing on media driver start to false.
- Add poll support to C media driver on Windows. PR #784.
- Name log buffers based on correlation id.
- Provide timestamp with stacktraces in default client error logger. PR #774.
- Reject concurrent publications that specify
init-term-id
,term-id
, andterm-offset
. PR #773. - Add sample illustrating how to build an index and basic time series on a recording that is also replicated in
IndexedReplicatedRecording
. - Improve performance for getting
Header.position()
in Java fragment handler. - Add
BasicAuthenticator
to C++ archive client samples. - Fix issue with configuring threading mode in C media driver. Issue #785.
- Improve validation when extending recordings in the archive.
- Add
taggedReplicate
operation to the archive for replicating a stream with provided tags so an external subscription can follow along. - Don't update the recording position in the archive if an exception occurs during a write. Previous behaviour could have erroneously reported progress when disk was full or underlying storage failure.
- Fix issue in C media driver when a subscription could have go away yet the publication considered it was still connected.
- Fix issue with incremental build dependencies. PR #762.
- Fix recording events enabled property name.
- Add authentication support to C++ archive client.
- Upgrade to Agrona 1.2.0.
- Upgrade to SBE 1.16.0.
- Upgrade to JUnit 5.6.0-RC1.
- Upgrade to Checkstyle 8.28.
- Upgrade to HdrHistogram 2.1.12.
- Upgrade to ByteBuddy 1.10.5.
- Upgrade to Gradle 6.0.1.
- Upgrade to javadoc-links 4.1.6.
- Upgrade to Mockito 3.2.0.
- Upgrade to gtest 1.10.0.
- Upgrade to HdrHistogram_c 0.9.12.
- Add bi-directional version identification to the archive network protocol
- Add support for authenticated sessions to the archive.
- Support setting of session-id on publications in the C media driver. PR #623.
- Fix setting of initial position on an exclusive publication in the C driver when the initial position is beyond the first term. PR #750.
- Allow for archive error log to be stored in archive mark file when running out of process from a media driver.
- Trim down unneeded dependencies in agent and all shadow JARs.
- Clean up allocated resources in C++ and Java clients when URI errors occur.
- Add
boundedPoll
toImage
for C++ and Java. PR #744. - Only include what is used in C++ publication headers. PR #743.
- Provide unique type ids to error counters. PR #741.
- Add new archive control messages to agent logging and improve overall agent performance.
- Fix pointcut for Archive control message logging. PR #740.
- Close files in Windows C++ client to prevent memory leak. Issue #737.
- Improve the performance for MDC dynamic mode in the Java driver.
- Set javadoc encoding to UTF-8.
- Improve validation of channel URIs for endpoint, control, tags, and distinguishing characteristics in both C and Java drivers.
- Fix calculation for archive truncate when offset is beyond first term in a segment.
- Check for reentrant calls when in Archive callbacks and throw an exception if detected.
- Change sample scripts to use the aeron-all JAR as a better example.
- Upgrade to javadoc-links 4.1.4.
- Upgrade to Build Scan 3.0.0.
- Upgrade to Shadow 5.2.0.
- Upgrade to ByteBuddy 1.10.2.
- Upgrade to SBE 1.15.0.
- Upgrade to Agrona 1.1.0.
- Correct bug when setting
MediaDriver.Context.rejoinStream
which setreliableStream
property by mistake and update configuration output dump. - Add bind address and port to channel endpoint counter label to help with debugging connections.
- Fix narrowing type conversion in C++ client for subscription images. PR #726.
- Add progress checks to
ReplayMerge
and a new terminal state ofFAILED
which is entered on exception or lost connection to the archive. - Track close following connections with MDS without timing them out which can help with
ReplayMerge
. - Support manual control on MDC not requiring the control address:port to be specified so it can be automatically assigned.
- Add ability to disable the recording events publication in the archive to save resources when it is not required.
- Add protocol version of the server to the connect response for archive clients.
- Upgrade to SBE 1.14.1.
- Upgrade to Agrona 1.0.11.
- Support the separate configuration of idle strategies for the replay and recording agent in the archive when running dedicated threading mode.
- Improve ownership tracking for subscriptions and images in C++ client.
- Improve matching of tagged channels.
- Increase archive storage version to 2.0.0 which requires the use of migration tool for existing archives.
- Add operations to purge and restore the history of a recording in the archive.
- Add the ability to query start position for a recording.
- Add Image specific fragment assemblers for C++ client.
- Reduce cacheline padding to save on memory footprint.
- Fix double delete in Aeron destructor. Issue #717.
- C++ client refinements. PR #716.
- Upgrade to javadoc-links 4.1.3.
- Upgrade to Gradle 5.6.3.
- Upgrade to Checkstyle 8.25.
- Upgrade to SBE 1.14.0.
- Upgrade to Agrona 1.0.9.
- Fix command message validation which failed to take account of message offset.
- Address some false sharing issues in the Java and C++ clients which can add 50ns of latency to RTT.
- Provide original channel URI in error message when parsing fails to port for an endpoint address. PR #714.
- Rewrite messages from older clients to the archive to allow for gradual upgrade of clients to the new archive. This support will last for only one minor version.
- Separate versioning schema for network protocol from file formats for the archive to allow them to evolve independently.
- Only check concurrent recording limits upfront in the archive to avoid later asynchronous errors.
- Reclaim mapped memory for IPC publications as soon as ref count is 0 and drained by subscriptions without going into 10 second linger.
- This release increases the major version on the archive wire protocol and file format. To upgrade it is necessary to update all archive clients and the archive at the same time. Also an archive migration is required by running the
CatalogTool
with the migrate option. Be sure to backup the archive first before doing a migrate. - Add recording signal reporting on the control stream for an archive. The
RecordingSignalAdapter
can be used to track signals of operations happening to recordings such as START, STOP, EXTEND, REPLICATE, MERGE, etc. - Improved Javadoc for archive configuration.
- Improved checking for clashing session-ids for manually configured publications.
- Reduce heartbeat updates to mark files to once per second to reduce IO traffic.
- Reclaim mapped memory for images by not lingering when the last subscription is closed. This can reclaim the mapped memory 10 seconds sooner by default.
- Fix ref counting to send channel endpoints which could cause a stream to get stopped early when multiple publications use the same channel.
- Add Archive replication feature which replicate a recording from one archive to another with the option of merging with a live multicast stream and continuing to support multiple redundant recordings.
- Reduce Java memory footprint of Archive client.
- Reduce default max concurrent recordings and replay in the archive from 50 to 20.
- Improve consistency of error codes and command validation to both Java and C Media Drivers.
- Add
Image.activeTransportCount()
to track active transports when using MDS which can be used to makeReplayMerge
more reliable. - Add correlation id to
RegistrationException
to help with debugging. - Allocate non-sparse files in Java media driver at safepoint to help avoid Time-To-SafePoint (TTSP) issues.
- Add the ability to configure congestion control as a channel URI param with the
cc=static
orcc=cubic
options. - Handle channel endpoint errors in the C++ client.
- Add support to the Java client for adding an removing destinations to publications and subscriptions asynchronously.
- Catch errors when opening receive destinations and report them to the client.
- Clean up bound ports on Windows when destinations are removed from MDS Subscripitons.
- Improve error messages on channel conflicts.
- Add
rejoin
URI param to channels so that when an image gets timed out to configure if it should stream or not. - Don't try to send archive client close messages when publication is not connected to avoid exceptions.
- Improve reliability of counter active and reuse checks.
- Clean up pending setup messages when a channel when endpoints are closed.
- Use heartbeat timestamp counters to indicate client liveness rather than command messages. This gives more stable behaviour on configurations with multiple clients sending many commands.
- Reworking of C Media Driver internals to more easily accommodate other media APIs such as ef_vi and DPDK.
- Add option to delete the
aeron.dir
on shutdown of the media drivers. - Make
MediaDriver.close()
idempotent. - Abort further reading of archive control stream once listed descriptors have been read so further messages are not missed.
- Improve reliability and precision of
ReplayMerge
. - Update session-id in catalog entries when an archive recording is extended.
- Add 'group' URI param to indicate if receiver group semantics, e.g. multicast NAK semantics, can be applied to Multi-Destination-Cast.
- More efficient and less allocating IP address dissection in logging agent.
- Change Java
RecordingReader
andCatalogTool
so they can read active recordings. - Improve handling of thread interrupt in Java client and archive client.
- Add INVOKER option and config check to C media driver.
- Add Java client
Aeron.Context.awaitingIdleStrategy()
configuration option for what to use when making a synchronous call to the driver. - Add log started event with timestamp when logging is enabled.
- Add cncVersion to configuration print on driver start.
- Fix potential out of bounds access for bytes received update in C media driver.
- Upgrade to Checkstyle 8.24.
- Upgrade to Mockito 3.1.0.
- Upgrade to javadoc-links 4.1.2.
- Upgrade to Gradle 5.6.2.
- Upgrade to build-scan 2.4.2.
- Upgrade to SBE 1.13.3.
- Upgrade to Agrona 1.0.8.
- Add client close handler to C++ client which can be used to detect close after client/driver timeouts for cleanup. It will be called on any client close including the shared pointer going out of scope and will be called only once.
- Try to call all the unavailable handlers in Java and C++ clients regardless of how the client gets closed, especially in the case of timeouts.
- Add addition relevant fields to
Image.toString()
method for debugging. - Correct log position in some cluster event messages which had the potential to be wrong.
- Upgrade to SBE 1.13.2.
- Enable timestamping of sequenced events in cluster at greater precision than milliseconds.
- Avoid seg faults by not incrementing error counters when starvation timeouts occur.
- Check for thread interrupt in spin loops so agents can be interrupted and tests can abort more cleanly.
- Coordinate agent shutdown in the Archive on timeout for a more clean shutdown.
- Detect timeouts due to resource starvation in Archive and Cluster and terminate to avoid seg faults.
- Allow the setting of an Aeron client in the Archive without an invoker.
- Upgrade to javadoc-links 3.8.4.
- Upgrade to SBE 1.13.1.
- Upgrade to Agrona 1.0.7.
- Complete work on IPC messages from services and reliable timers for Cluster.
- Simplify logic in
Image.controlledPeek
methods and correct return value on closed on Java client. - Support OSGi manifest headers. PR #690.
- Add missing Javadoc to public classes.
- Add
AeronException.Category
enum to Aeron exceptions for Java and C++ by separating warn, error, and fatal categories of exception. - Ability to add and remove close handlers on Java Aeron client to avoid use after close and segfaults due to unmapped files.
- Check for Aeron client close in Archive due to starvation to avoid segfaults.
- Remove layer of indirection from
Subscription
toImage
in C++ client to help avoid cache misses. - Update licence references to https.
- Upgrade to Mockito 3.0.0.
- Upgrade to Checkstyle 8.23.
- Upgrade to javadoc-links 3.8.1.
- Upgrade to Gradle 5.5.1.
- Upgrade to SBE 1.13.0.
- Upgrade to Agrona 1.0.6.
- Add pre-touch option for memory mapped files in clients to fault in pages to reduce latency spikes on first cycle around log buffers.
- Rework C++ client to share log buffer mappings between publications and subscriptions when possible to reduce mapping costs.
- Rework C++ client to manage images via
shared_ptr
to help avoid memory reclamation issues. Issue #467. - Rework management of C++ client resources to be more O(1) when reacting to events from the driver.
- Set error handler on Aeron client in Archive client when set. PR #687.
- Check for reentrant calls to Aeron client from C++ callbacks and call error handler if detected.
- Add
watch
option to JavaAeronStat
and separate out counter reader. PR #684. - Tighten up logic to avoid use of C++ Aeron client after close.
- Replace division operations with shifts where possible to improve performance on startup in Java implementation.
- Support adding and removing counters availability handlers after client connect and to support multiple handlers in Java and C++ clients.
- Add the ability to do archive replays bounded by a counter to Java and C++ implementations.
- Improve and expand Java and C++ samples.
- Some fixes for the Windows native build which is still experimental.
- Remove unused linger feature from flow control to reduce footprint and improve performance.
- Fix concurrency issue with buffer cleaning which could result in a stream locking up and being in a permanent back-pressure state.
- Add
stopAllReplays
operation to the Archive which can be on a specific recording or wildcarded to all recordings. - Don't treat explicit client closes as timeouts. PR #681.
- Limit resend window to half a term length to avoid unnecessary under runs.
- Add observed values in timeout exception messages. PR #680.
- Reduce allocation when using logging agent and improve its performance.
- Use more efficient clock implementations in C media driver.
- Use correct message encoder when sending a
stopReplay
to the archive. PR #676. - Correctly initialise send and clean positions on publications when a non-zero start is required in the C driver.
- Add semantic versioning to CnC files. Issue #624.
- Fix
Subscription::isConnected()
in C++ client. - Many improvements to Cluster which is not yet GA.
- Upgrade to Mockito 2.28.2.
- Upgrade to javadoc-links 3.7.5.
- Upgrade to Checkstyle 8.22.
- Upgrade to Shadow 5.1.0.
- Upgrade to SBE 1.12.8.
- Upgrade to Agrona 1.0.3.
- Fix logging of cluster election events due to class loading clash with aeron agent.
- Fix issue with snapshot on leader not recording which service message it had appended up to.
- Upgrade to SBE 1.12.7.
- Add support for clustered services to send reliable ingress back into the cluster over IPC to be sequenced into the log.
- Added
aeron.client.close.linger.duration
which can be set greater than the default of 0 to help clients in resource constrained environments, or long GC pause applications, from experiencing seg faults from unmapped files. - Close Java
Aeron
client when timeouts occur and delay before unmapping files to help prevent seg faults. - Add
dump
option toCatalogTool
to dump out full details including recorded stream data. PR #669. - Provide read only view of Archive Catalog. PR #668.
- Protect against contexts being used in multiple drivers or clients. PR #666.
- Add support for channel tags to C media driver. Issue #622.
- Make socket buffer config settings public for driver context.
- Upgrade to javadoc-links 3.6.4 to allow for offline build.
- Upgrade to hamcrest 2.1.
- Upgrade to SBE 1.12.6.
- Upgrade to Agrona 1.0.1.
- Various improvements and fixes for Aeron Cluster. Elections are becoming robust and the API is firming up.
- Fix counter resource leak when cycling UDP publications with back pressure counters. PR #663.
- Add
tether
support for local flow control to Java and C media drivers. This allows a subscription to declare itself as a tether or not on local flow control. If not a tether then it can be left behind if it stops or cannot keep up. - C and Java media drivers can optionally print their configuration on startup.
- Greatly improved configuration for C media driver allowing configuration via properties files fetched from local file system or over HTTP, or via API for context. The list of files can be passed on the command line to
aeronmd
. - Include
session-id
in archive recording key regardless of using tags or not. - Abort Archive sessions if the response publication fails after successfully connected or when any other exception occurs during processing. This avoid infinite failure loop in the archive on some conditions.
- Improve Javadoc.
- Add channel param for indicating if
EOS
should be sent on close of publication or not. - Improve build of native driver on Windows.
- Output from ctest on failure for native build.
- Upgrade to io.freefair.javadoc-links 3.2.1.
- Upgrade to Checkstyle 8.20.
- Upgrade to ByteBuddy 1.9.10.
- Upgrade to Mockito 2.27.0.
- Upgrade to Gradle 5.4.1.
- Upgrade to SBE 1.12.5.
- Upgrade to Agrona 1.0.0.
- Add a
BacklogStat
tool for inspecting the backlog in bytes for consumers of streams. PR #650. - Faster startup and build time on Windows.
- Reduce module warnings on Java 11.
- Change default archive recording progress event channel to dynamic MDC to avoid
PortUnreachableException
s that cause garbage. - Add the ability to terminate a Java or C media driver by sending it a command. Useful for testing in a mixed language environment.
- Fix issue with multi-destination subscriptions that are set up in advance. PR #649.
- Add sender back-pressure event counters per stream for monitoring.
- Improve
AsyncConnect
for cluster and archive clients and use it wrapped in the sync implementation. - Add
min
flow control strategy to the C Media Driver. - Improve
toString()
methods in Aeron wire protocol flyweights for reduced allocation and better performance. - Complete work on C++ Archive client.
- Change Stats tools to use read only file mapping for safety and better NUMA performance.
- Add back-off idle strategy to the C Media Driver.
- Add C++
LossStat
application. - Fix issues with Archive Catalog failing to load due to old version.
- Clean up native build for Windows.
- Complete work on
ReplayMerge
for both Java and C++ Archive clients. - Add
writev
support to SPSC C ring buffer. PR #644. - Add basic validation to configuration for client and unblock timeouts.
- Add event codes and support logging in archive and cluster for major events. PR #646.
- Upgrade Checkstyle DTD and apply checkstyle to method Javadoc.
- Use HTTPS in generated Javadoc links and fix warnings under Java 11 and OpenJDK.
- Report version numbers in conflict when connecting to archive or cluster.
- Upgrade to Mockito 2.25.1.
- Upgrade to Gradle 5.3.1.
- Upgrade to SBE 1.12.4.
- Upgrade to Agrona 0.9.35.
- Remove deprecated Subscription
EOS
handlers andExclusiveBufferClaim
from Java and C++. - Provide the ability to ignore warning for Aeron direction existence in C Media Driver. Issue #639.
- Fix memory leak when adding and removing destinations for the C Media Driver.
- Include links when generating Javadoc.
- Add
ChannelUri
andChannelUriStringBuilder
to C++ for ease of use when with channels. - Add C++
BackoffIdleStrategy
. - Reduce object allocation in Java media driver startup.
- Provide the ability for the driver to print out all it configuration properties on start via
aeron.print.configuration=true
. - Clean up Java driver configurations so that all properties can be configured from system properties or context.
- Provide the ability to list active recording subscriptions on the archive so they can be tracked and closed when clients do not clean up gracefully.
- Handle race conditions when establishing control session connections to the archive when the same stream id and channel are used by multiple clients or over multicast.
- Check that messages are for the correct protocol communicating with the archive and cluster.
- Add experimental C++ client for Aeron Archive.
- Add experimental
ReplayMerge
for catching up to a live stream from an archive. - Add semantic version support to archive and cluster protocols.
- Upgrade to Mockito 2.25.0.
- Upgrade to Shadow 5.0.0.
- Upgrade to Gradle 5.2.1 and remove used of the now deprecated OSGI plugin.
- Upgrade to Checkstyle 8.18.
- Upgrade to SBE 1.12.3.
- Upgrade to Agrona 0.9.34.
- Cluster refinements.
- Reset padding value on failed claim to C ring buffer. Issue #636.
- Add clang 6 debug build to Travis CI.
- Correct issue with archive and cluster mark files version so it is not tied to SBE schema and instead uses semantic versioning. Issue #638.
- Deal with multiple archive clients connecting on the same stream id and channel. PR #634.
- Cleanup of C++ client. PR #631.
- Upgrade to SBE 1.12.2.
- Upgrade to Agrona 0.9.33.
- Fix issue with length of channel not being taken into account when parsing commands in the C Media Driver which could lead to corrupt URI channel params.
- Add
Automatic-Module-Name
to help support modular Java applications using Aeron. Issue #627. - Reduce cache missing when handling loss in Java Media Driver.
- Relocate Byte Buddy in fat JAR to avoid clash with other needs for Byte Buddy such as Mockito.
- Upgrade to Byte Buddy 1.9.7.
- Upgrade to Mockito 2.24.0.
- Upgrade to SBE 1.12.1.
- Upgrade to Agrona 0.9.32.
- Use unique schema ids to help detect stream miss-configuration between archive and cluster protocols, and allow for future extension.
- Additional validation of publication and subscription URI channel params.
- Experimental build of C Media Driver for Windows. PR #610.
- Fix segment length recorded in the Archive Catalog when term length greater than segment length. PR #619.
- Add support to C Media Driver for initialising a publication for replay at a given position.
- Shadow JAR in the correct versions of HdrHistogram and ByteBuddy into
aeron-all
. - Upgrade to Checkstyle 8.17.
- Upgrade to HdrHistogram_c 0.9.8.
- Upgrade to SBE 1.12.0.
- Upgrade to Agrona 0.9.31.
- Correct order of
correlationId
andrelevantId
when reporting asynchronous messages back from the Archive. - Add
INVALID_EXTENSION
error code for requests to extend an existing recording when a new image that does not match. - Add checks to Archive client for use after close.
- Update Archive Catalog with
controlSessionId
andcorrelationId
when an existing recording is extended. - Have ownership for contexts pass to agent after successful start so the agent is responsible for close to help about seg faults with unmapped files. Contexts are now not
AutoClosable
so people are not tempted to use them in try-with-resources idioms. Issue #612. - Add checks to C++ client for use after close.
- Detect a broadcast buffer wrap when the client has been unresponsive for a long time and close the client.
- Notify client if the driver times them out so they can gracefully shutdown. Issue #611.
- Add support for human readable short form properties for configuring the C media driver. Issue #603.
- Parse interface and address URI parmas without using regex for portability and better performance in C media driver.
- Parse interface URI param without using regex for less allocation and better performance in Java media driver.
- Validate initial position when constructing a Channel URI.
- Upgrade to HdrHistogram 2.1.11.
- Upgrade to Shadow 4.0.4.
- Upgrade to SBE 1.11.0.
- Upgrade to Agrona 0.9.30.
- Lots of testing on Cluster in preparation for release.
- Avoid using unmapped files during an interrupt close which can cause seg faults. Issue #607.
- Store full channel URI in counters labels for the C media driver.
- Default build to include the C media driver with C++ client.
- Enable the setting of
mtu
,term-length
,sparse
,linger
, andreliable
as channel URI params for the C media driver. - Macro guard against missing PROJECT_SOURCE_DIR. PR #601.
- Clean up of warnings in native code.
- Fix unblocking of full ring buffers when clients die.
- Fix clearing of
linger
andsparse
params inChannelUriStringBuilder
. - Fix issue with stripped channel keeping
linger
andsparse
params that would make stop recording fail. - Fix 'extended alignment' error in Visual Studio 2017 (>15.8). Issue #599.
- Add definition for Add definition for PROJECT_SOURCE_DIR to aeron_client's C++ public interface. PR #596.
- Upgrade to Google Benchmark 1.4.1.
- Upgrade to Checkstyle 8.16.
- Upgrade to SBE 1.10.2.
- Upgrade to Agrona 0.9.29.
- Improve throughput and latency of archive replay, significantly so on Windows and OS X.
- Add OS X to Travis for native build.
- Improve handling of compilation flags with CMake. PR #572.
- Fix Page Size constants on Mac OS. Issue #592.
- Fix contains match when listing recordings by URI and improve its performance.
- Add assertions and cleanup C++ client. PR #582.
- Support
alias
in URI parameters. - Fix disparity between Java and C++ for handling of raw tail on publications. Issue #589.
- Deprecate Subscription
pollEndOfStreams
as Images should be used directly. - Use relative source location in C++ exceptions. PR #588.
- Account for EOS when using multi-destination subscriptions.
- Keep an Image around while draining without a timeout to facilitate slow consumers.
- Add the ability to linger an archive replay publication to better cope will lossy networks.
- Refine parsing of tags to allocate less.
- Eliminate unnecessary direct buffer allocations when allocating publication and images.
- Warnings clean up in code base.
- Fix some false sharing issues between agent threads.
- Upgrade to Checkstyle 8.15.
- Upgrade to Gradle 4.10.3.
- Upgrade to SBE 1.10.1.
- Upgrade to Agrona 0.9.28.
- Add
ErrorHandler
toAeronArchive
client to better handle async errors from the archive. - Deprecate use of Java
ExclusiveBufferClaim
now that baseBufferClaim
has all the necessary functionality. - Numerous performance improvements.
- Add a configurable connection timeout for response and replay streams from the archive.
- Fix overflow in partition index calculation based on position resulting in SIGSEGV in C++ client. Issue #579.
- More consistent use of
const
and formatting in C++ client. - Fix case of a recording failing with messages > 2MB at the end of term or on unblocking.
- Force close Java publications and subscriptions on channel errors. Issue #456.
- Add monitoring counter for Aeron clients which have timed out.
- Improve
Image
block polling to better handle padding frames. - Add more checks to input parameters on publications, images, and the archive to catch usage errors.
- Add 2 buffer offer to Java
Publication
to avoid need for vectored offer. - Add context to C++
Header
in fragment callback for access to theImage
. - Upgrade to Shadow 4.0.3.
- Upgrade to Byte Buddy 1.9.3.
- Upgrade to Mockito 2.23.4.
- Upgrade to SBE 1.10.0.
- Upgrade to Agrona 0.9.27.
- This is the first release that supports building and running on Java 11.
- Account for negative initialisation or wrap on
System.nanoTime()
when checking deadlines. - Add a build matrix for C/C++ and Java on Travis.
- Revert failing C++ build due to PR #555. Issue #560.
- Make the C/C++ build more modular. PR #561.
- Preserve original segment file length when extending a recording.
- Update Java samples showing how IPv4 can give better performance.
- Add support for multi-destination subscriptions to the C++ client. PR #568.
- Correct typos in documentation.
- Add the ability to query the
stopPosition
of a recording in a single call. - Add the ability to query for the last recording matching
channel/stream-id/session-id
in the archive. - Don't attempt replay of archive entries that are marked invalid.
- Set better example with list recordings samples from the Archive. Issue #565.
- Detect and handle thread interrupt in Aeron, Archive, and Cluster Java clients.
- Initialise variable in
StringUtil.h
. Issue #562. - Allow replay from the current recorded position in the Archive.
- Add missing copy assignment operators to C++ client.
- Upgrade to Byte Buddy 1.9.0.
- Upgrade to Mockito 2.23.0.
- Upgrade to Checkstyle 8.14.
- Upgrade to SBE 1.9.0.
- Upgrade to Agrona 0.9.26.
- Fix formatting of counter label for stream positions that would corrupt channel and join position. Issue #558.
- Fix potential race condition with updating receiver window for Java and C Media drivers. Impact likely to be more significant on 32-bit platforms.
- Allow accepting of subscriptions when image is in the INIT state. It could be observed that 2 or more subscriptions could see only the first succeed when done in quick succession on a dedicated thread mode.
- Fix potential overflow in position calculations when updating the Archive Catalog after an unclean shutdown.
- Use relative source for exceptions in C++ client. PR #555.
- Fix SHARED_NETWORK threading mode for C Media Driver.
- Upgrade to Shadow 4.0.0.
- Upgrade to SBE 1.8.9.
- Upgrade to Agrona 0.9.25.
- Fix issue with polling images in a controlled or peek fashion when padding was inserted at the end of a term.
- Add ability to set file sync level on the archive catalog separate from segment files.
- Upgrade to Gradle 4.10.1.
- Account for message header when listing descriptors from the archive. Issue #550.
- Reduce allocation of exceptions when network issues occur.
- Delete log buffers that are no longer referenced in C media driver.
- Add system counter for failed attempted deletes of log buffers on Windows.
- Fix build of C++ client on Clang 3.9, 4, and 5. Issue #544.
- Prepare for building on Java 11.
- Finally get Aeron working on Sparc. Issue #359.
- Allow for debug logging in the Java driver via the
aeron.event.log.filename
system property and reduce allocation when using this. - Upgrade to Byte Buddy 1.8.21.
- Upgrade to Mockito 2.22.0.
- Upgrade to Gradle 4.10.
- Upgrade to SBE 1.8.8.
- Upgrade to Agrona 0.9.24.
- Fix issue with running Java version of Aeron on Sparc CPUs due to endianess.
- Improve efficiency and significantly reduce allocation when performing agent logging in Java driver.
- Fix regression to agent logging for Java driver due to broken pointcut.
- Make resource linger configurable in the Java client.
- Fix buffer overrun in C driver with system counters.
- Reduce footprint for Java subscriptions in the client.
- Fix issue with adding and removing destination to manual MDC publications when publication is not original.
- Improve resource lingering code in Java and C++ clients to narrow the possibility of a seg fault when buffers are unmapped in a low resource utilisation scenario.
- Fix bug when 1GB term lengths are used in Java driver.
- Apply
const
to methods and arguments in C++ client where appropriate. - Added a busy spin idle strategy to C driver that uses X86 PAUSE and make the noop idle strategy truly no op.
- Added
NoOpIdleStrategy
to C++ client. - Fix overflow in large reassembly buffers. PR #538.
- Set
ByteBuffer
s in Java client to be little endian for underlying log buffers. - Upgrade to Checkstyle 8.12.
- Upgrade to Byte Buddy 1.8.15.
- Upgrade to Mockito 2.21.0.
- Upgrade to SBE 1.8.7.
- Upgrade to Agrona 0.9.23.
- Small improvement to startup time for Java Media Driver.
- Fix C++ compile for Visual Studio on Windows. PR #528.
- Fix issue causing a seg fault on Windows when connections are closed but clients are holding subscriptions.
- Set control channel to be sparse for faster connections with the archive.
- Allow setting of log buffers to be sparse on a per connection basis by using
sparse=true
as a channel URI param. - Cluster refinements in preparation for beta.
- Upgrade to SBE 1.8.6.
- Upgrade to Agrona 0.9.22.
- Upgrade to Mockito 2.20.1.
- Upgrade to Gradle 4.9.
- Fix bug with printing contents of
DistinctErrorLog
on start-up with existing errors. - Upgrade to SBE 1.8.5.
- Reduce allocation when setting up new connections.
- Allow multiple replays for the same session in the archive if using tag references.
- Change response publication from concurrent to exclusive so that a connect response is not missed due to another connection on the same channel and stream id. Issue #525.
- Guard against the Aeron client being used in a re-entrant fashion from a image or counter callback. Issue #524.
- Enqueue new image events in the archive to avoid race when sending a control action. Issue #524.
- Return subscription id when starting an archive recording so it can be stopped by id as an alternative to channel and stream id.
- Upgrade to Byte Buddy 1.8.13.
- Upgrade to Mockito 2.19.1.
- Upgrade to SBE 1.8.4.
- Upgrade to Agrona 0.9.21.
- Change default send to status message poll ratio from 4 to 6 for more efficient message sending.
- Check send and receive channel endpoint UDP datagram channels are open and connected to avoid exception allocation.
- Move authentication interfaces from the cluster module to aeron client module under the security package.
- Fix issue with missing passing of relevant id for error codes in response from the archive. Issue #523.
- Upgrade to Checkstyle 8.11.
- Upgrade to Gradle 4.8.1.
- Upgrade to SBE 1.8.3.
- Upgrade to Agrona 0.9.20.
- Preparing for beta release of Aeron Cluster.
- Add the ability to query the
Image.sourceIdentity()
from aRecordingPos
counter. - Support the extending of an existing recording with a different
session-id
. - Add error codes to Archive exceptions so that they can be programatically handled.
- Provide Aeron specific exceptions where relevant.
- Add the ability to add and remove endpoint destinations to subscriptions so they can receive from multiple sources to the same
Subscription
. Details of usage can be found on the Wiki. - Add support for tagging subscriptions and publications so they can be referrenced.
- Expand
ChannelUri
andChannelUriStringBuilder
to reflect new channel configuration params. - Simplify local function call with compile time polymorphic semantics for C++. PR #514.
- CMake improvements for native build. PR #507.
- Reset Thread interrupted flag in Aeron client after catching an
InterruptedException
. - Allow the setting of the thread name on C++
AgentRunner
. PR #509. - Upgrade Google Benchmark to 1.4.0.
- Upgrade to Checkstyle 8.10.1.
- Upgrade to Byte Buddy 1.8.10.
- Upgrade to Mockito 2.19.0.
- Upgrade to SBE 1.8.2.
- Upgrade to Agrona 0.9.19.
- Fix for overflow in calculation on
Image.boundedControlledPoll()
that can result in not progressing past the end of a term.
- Add
SegmentInspector
tool for debugging archive segment files. - Fix calculation for replaying a recording when the start is not the beginning of the stream and the position is greater than the start.
- Address race condition between
File.length()
andFileChannel.size()
when starting Aeron client before Media Driver. - Fix extend recording so offset calculation is correct after the first term.
- Initialise archive recording before first data so that a replay can start before data is transmitted.
- Fix
CatalogTool
to correctly deal with filenames. - Upgrade to Checkstyle 8.10.
- Upgrade to Shadow 2.0.4.
- Upgrade to SBE 1.8.1.
- Upgrade to Agrona 0.9.18
- Set HTML 5 version in javadoc if building under Java 10 to avoid warning.
- Use
Constructor
to avoid Java 10 deprecation warning when dynamically creating new object instances from a class. - Fix catalog refresh for Archive on restart for recordings that did not finish gracefully. PR #502.
- Upgrade to Gradle 4.7.
- Upgrade to SBE 1.8.0.
- Upgrade to Agrona 0.9.17.
- Add a new exclusive publication handler for the C++ client. Issue #500.
- Add a
StreamStat
tool for give a summary of all counters grouped by stream for monitoring. - Provide a publication position counter that is updated once per second for tracking the publishers for monitoring.
- Provide monitoring counters for each Aeron client with their heartbeat timestamp.
- Support the compressing or deletion of archive recordings by external scripts by giving a warning that the files are not available rather than giving an error.
- Refinements to the Catalog tool.
- Provide a limit on the range which can be reserved for session ids. Issue #498.
- Fix race with an Aeron client mapping a CnC file which has not been set the the correct length yet by the driver. Issue #499.
- Add the ability to truncate a recording in the archive whereby truncating to the stop position effectively deleting the recording.
- Add the ability to query the current recorded position of an stream from a remote archive.
- Add support for async connect to an archive.
- Make archive catalog file length configurable and use a smaller default.
- Fix write of data frame header for 32 bit machines, e.g. Raspberry Pi.
- Faster and less garbage version for parsing URI endpoints. Issue #491.
- Sample code for doing large file transfer with Aeron. Issue #237.
- Add startup timestamp and PID to CnC file for monitoring. Issue #480.
- Fix Javadoc generation so the index page references all classes.
- Upgrade to Shadow 2.0.3.
- Upgrade to Mockito 2.18.3.
- Upgrade to Gradle 4.6.
- Upgrade to Checkstyle 8.9.
- Upgrade to SBE 1.7.10.
- Upgrade to Agrona 0.9.16.
- Fix vectored offer of large messages with offsets that do not start at zero. PR #475.
- Replace hot switch statements in the Java driver so the JIT can make better optimisation decisions.
- Fix race issue with archive replaying an active recording and the existing recording stopping mid replay.
- Add support for large pages to C driver.
- Improve error reporting in C driver startup.
- Upgrade to SBE 1.7.9.
- Upgrade to Agrona 0.9.15.
- Guard against a counter being added with no key in C++ client.
- Change move to copy semantics for C++ client to avoid concurrency issues while iterating images under a subscription. Issue #472.
- Simplify archive recording so it works more consistently on all platforms. This has resulted in a better throughput and latency profile.
- Rework archive to better handle concurrent usage of recording catalog and aeron client.
- Handle case of removing a subscription that has not seen any traffic that could result in a
NullPointerException
. - Update archive samples to be a better illustration of usage.
- Perform a contains operation for matching channel URIs rather than an exact match when listing recording descriptors.
- Upgrade to SBE 1.7.8.
- Upgrade to Agrona 0.9.14.
- Fix idle config for C driver from env vars.
- Add low-latency config to Java sample scripts.
- Allow all context types to be
Cloneable
in Java for easier config in testing. - Add
YeildingIdleStrategy
to C++ client and add aidle()
method without args to the all strategies. - Bring C++ client inline with Java by providing simpler methods for checking channel endpoint status.
- Add a
MarkFile
so a second archive cannot be started accidentally in the same directory plus provide metadata about the running instance. - Improve the
RecordingPos
counter support from tracking the progress of a local recording in the archive. - Add the ability to change the linger timeout for a publication to wait around after close in case the receiver needs to recover loss. Issue #452.
- Support human friendly values such as
term-length=64m
andmtu=4k
in channel URI params. - Improve system tests to be more suitable example code.
- Improve C++ client performance when built for Windows.
- Improve the archive client for more robust connection to the archive.
- Improve the error reporting from the driver when network errors are experienced.
- Add the ability to extend an existing archive recording as a continuous immutable log. This requires the new publication to have the existing session id which can be provided via the channel URI.
- Performance improvements to manual multi-destination-cast from Java driver.
- Improve error reporting from the archive to the control sessions on invalid requests.
- Numerous small archive performance improvements.
- Add the ability to the archive to record specific publication sessions and update client API to support this.
- Reduce the cost of querying the archive for recordings. The listing of a recording descriptor by recording id is now allocation free.
- Reduce allocation when setting up new connections so connections can be cycled more efficiently.
- Add a second local subscription to the archive in listening for control sessions. This can be over IPC for greater efficiency.
- Add the ability to subscribed to a specific
session-id
by adding it as a channel URI param. Useful to isolating a single session rather than wild-carding on channel and stream id. - Add the ability to choose a publication
session-id
by setting it as a channel URI param. A default range from low to high value can be configured for the driver to not use in automatic assignment. - Add cooldown option on counters so they are not reused for a timeout which defaults to 0.
- Change congestion control to more accurately measure RTT before feeding into calculations.
- Reduce default MTU to 1408 to avoid loss on Google Cloud and inter-region AWS. This will reduce throughput on local un-contended networks and a larger value can be configured.
- Cache clock values during duty cycle to save on system calls in Java driver.
- Adjust idle constants so backoff will sleep and thus reduce CPU usage. Newer kernels spin on
LockSupport.parkNanos()
with a low value. - Handle unaligned strings used in commands to the C driver.
- Reduce Hamcrest dependency from all to library.
- Upgrade to Mockito 2.15.0.
- Upgrade to Gradle 4.5.1.
- Upgrade to JShadow 2.0.2.
- Upgrade to Checkstyle 8.8.
- Upgrade to SBE 1.7.7.
- Upgrade to Agrona 0.9.13.
- Bring the C++ client up to par with the Java client on feature set.
- Use
fallocate
on Linux with the C driver if supported. - Allow
spiesSimulateConnection
to be configured for anArchivingMediaDriver
, it was previously hard coded totrue
. - Add
RecordingPos
utility to find the counter for recordings so it can be tracked and consumption of Images can be bounded to ensure they are recorded. - Move Images out of the active state as soon as they have reached end-of-stream and rebuilt so that they don't get spuriously captured by new Subscriptions.
- Don't elicit SETUP messages once a Publication has reached end-of-stream to avoid spurious connections.
- Reduce allocation in the Archive when setting up recordings or replays.
- Rework client close so it is constant time regardless of the number of publications and subscriptions.
- Additional simplified API for adding counters that only require a label.
- Improve efficiency of MDC (Multi-Destination-Cast) connections.
- Clean up C/C++ build warnings. PR #440.
- Provide an API to start and stop an Archive recording without creating a Subscription.
- Fixes for building C++ with Visual Studio. PR #384.
- Add counter available and unavailable callbacks to the Aeron client.
- Add the ability to set the Aeron directory to the Archive context.
- Simplify API for getting channel endpoint status on a
Publication
orSubscription
this is a breaking change to the feature introduced in the 1.6.0 release. - Add bounded control poll to
Image
so they can be conditionally consumed. - Fix samples for simple publication and subscription for timing issues. Issue #435.
- Upgrade to Checkstyle 8.5.
- Upgrade HdrHistogram for C/C++.
- Upgrade to Mockito 2.13.0.
- Upgrade to JShadow 2.0.1.
- Upgrade to Gradle 4.4.
- Upgrade to SBE 1.7.6.
- Upgrade to Agrona 0.9.12.
- Change Aeron client ownership in
AeronArchive
so that the archive client will default to not owning the Aeron client unless it creates it. - Detection of network channel setup errors, e.g. port already in use, and propagation back to the client public the addition of channel status indications of publications and subscriptions.
- Reduced use of capturing lambdas to allow for faster warmup and reduced allocation on connection setup.
- Simplify client command API to allow for easier development of clients in other languages.
- Provide an API for allocating
Counter
s which are managed by the media driver and can be read withAeronStat
- Small performance improvements and faster warm up of Multi-Destination-Cast publications.
- Decouple Archive from media driver so that the Java Archive can be used with the C media driver.
- Tidy up logic in archive client to better handle unexpected messages from the archive and not read messages in pollers after message may have been released.
- Modify unblock logic so that it does not apply to exclusive publications except in when past the active state.
- Correct Javadoc for
Publication
s and concurrent semantics. Issue #429. - Update to SBE 1.7.5.
- Update to Agrona 0.9.11.
- Update to Byte Buddy 1.7.9.
- Update to Mockito 2.12.0.
- Update to Checkstyle 8.4.
- Improve failure detection in
AeronArchive
client for when connections drop. Issue #427. - Add a lock to
AeronArchive
client so it is threadsafe. - Fix issue introduced with unblock logic that would not unblock when term had rotated ahead of blocked position. Issue #424.
- Remove use of PAGE_SIZE in C++ client to avoid compilation errors on Android. Issue #405.
- Have
Publication
andExclusivePublication
have a common super class of Publication in Java so they can be interchanged. - Add
isConnected()
method to Subscription. - Check window length against socket buffer on start up fail fast. Issue #420.
- Support units (k, m, and g) for size such as
16k
for a 16 KB buffer length, and (s, ms, us, ns) for durations in system properties. - Add generated codecs to aeron-archive.jar. Issue #416.
- Upgrade to Mockito 2.11.0.
- Fix bug with segment filename generation for the Archive. #414.
- Further validation of publication parameters to the C media driver.
- Duplicate ByteBuffers for terms in client so independent subscriptions can operate in a threadsafe manner if they require access to the underlying ByteBuffers for onwards IO.
- Remove
aeron.term.buffer.max.length
property as it is not needed given current usage. - Fix socket options and bind address for multicast in C driver.
- Upgrade to SBE 1.7.3.
- Upgrade to Agrona 0.9.9.
- Separate
sendmmsg
andrecvmmsg
detection in C driver to support older kernels. Issue #412. - Better support the cycling of connections so they consume less resources when lingering.
- Provide more immediate detection of an unconnected IPC stream.
- Allow spy subscriptions to simulate a network connection so a network publication can progress when no receivers are present. Issue #393.
- Add configuration for page size so the files can be on storage using huge pages to reduce TLB misses. Issue #387
- Limit maximum message length to be 16MB to encourage better design and streaming in chunks.
- Change concurrent publication algorithm to better handle the case of thread starvation with publishers and allow other publishers to help rotate a log if another publication is interrupted, or dies, while it is rotating the log. Issue #377.
- Add
Subscription.isConnected()
. - Add event logging for events which result in an exception.
- Add the ability to set JVM_OPTS from an environment variable in command line scripts.
- Pass through command line arguments and system properties in sample scripts.
- Allow for mapped buffers to be ref counted and shared across publications and subscriptions within a client. Issue #365.
- Add vectored IO for publishing a number of buffers which make up a message as a single operation. Issues #401, #394.
- Reduce copying, interface scanning, and object allocation for channel setup in Java.
- Upgrade to SBE 1.7.2.
- Upgrade to Agrona 0.9.8.
- Upgrade to Checkstyle 8.3.
- Upgrade to Mockito 2.10.0.
- Upgrade to Byte Buddy 1.7.4.
- Update the aeron-archive module to Maven Central as part of the build.
- Improve C driver build when OS features are not available so it degrades more gracefully.
- Tidy up of media driver configuration and added Javadoc. Note that if you used
Context.dirsDeleteOnStart(boolean)
for testing that it has been renamed toContext.dirDeleteOnStart(boolean)
. - Add configuration option so that high resolution timers can be enabled on Windows.
- Fix issue with term buffer length of 1GB. Issue #388.
- Native media driver written in C for greater throughput on Linux. Get started here....
- Archive service in Java to support the recording and replay of message streams. Get started here....
- Improved
AeronStat
for running in a Windows console and addeddelay
param for controlling the update cycle in seconds. - Handle race condition on driver start when checking if it is active yet. Issue #385.
- Update
FlowControl
API to support ashouldLinger
method and EOS parameters to allow for faster clean up of resources. - Encapsulate buffer and length into a struct for atomic update of images under a subscription in C++. Issue #383.
- Add
Subscription::controlPoll
to C++ client. - C++ client was not going round robin on the images under a subscription. Issue #381.
- Added
AgentInvoker
to C++ client for removing the need for the client conductor thread. - Fix issues with publications being unblocked when clients are closing and creating new subscriptions. Issue #377.
- Add
Publication.MAX_POSITION_EXCEEDED
return code from offer for whentermId
can wrap due to using small term buffer lengths. - Add
shared_ptr
for LogBuffer saved inPublication
andExclusivePublication
to keep mapping around while in scope. Issue #371. - Rework Java client to better support timeouts and closing from other threads to help avoid segfaults. Issue #371.
- Publications store
registrationId
andcorrelationId
with the ability to callPublication.isOriginal()
to determine if this is the first publication added for a given channel and streamId on a driver. - Move access to the
AgentInvoker
fromAeron.Context
toAeron
on the client andMediaDriver.Context
toMediaDriver
. - Fix alignment types in native buffers for supporting processors which require alignment. Issue #359.
- Counters now have labels in US_ASCII rather than UTF-8 to be more efficient as they are restricted to US_ASCII anyway.
-
Image.joiningPosition()
has been renamed toImage.joinPosition()
. -
ByteBuffer
s inSubscription
andImage
polling callbacks now have offsets the same as termBuffer offsets so theByteBuffer
can be used directly. - Stricter validation of channel URI parameters and new classes for client usage in
ChannelUri
andChannelUriStringBuilder
. - Reduce object allocations when cycling streams in Java media driver.
- Add
Image.isEndOfStream()
for fast detection of stream end. - Add direct
ByteBuffer
support toFragmentAssembler
s to avoid the need for copying of received messages before sending on the an NIO channel. - Upgrade to Agrona 0.9.7.
- Upgrade to SBE 1.7.1.
- Upgrade to Mockito 2.8.47.
- Upgrade to Byte Buddy 1.6.14.
- Upgrade to Checkstyle 8.1.
- Upgrade to sevntu-checks:1.24.1.
- Upgrade to Gradle 4.1.
- Fixed issue with a Subscription being closed and rapidly reopened on the same streamId and channel.
- Add
ThreadingMode.INVOKER
to the driver so no threads are started and the media driver can be invoked by the client for its duty cycle. Example in the Archiver. - Added system property that allows TCP_MODE to be enabled on CUBIC congestion control.
- Throughput and latency improvements on the receiver side when interacting with NIO
DatagramChannel
s. - Add
Header.context()
member which contains theImage
for use in subscriber callbacks. - Allow for client thread and lock to be elided when used in a low resource environment where the client will invoke the client duty cycle. Example can be found in the Archiver.
- Notify that a new Image is available before it is added to the
Subscription
so setup actions can be taken. - Fix
Ping
samples for messages larger thanMTU
. - Numerous improvements to the performance and GC overhead of creating and releasing connections.
- Add
Image.joiningPosition()
. - Avoid use of
std::function
for performance in C++. - Add
ExclusivePublication
support to C++. - Close file descriptor after mapping an existing file in C++. Issue #339
- Upgrade to Checkstyle 7.7.
- Upgrade to Agrona 0.9.6.
- Make
ExclusiveBufferClaim.flags()
consistent withHeader.flags()
. - Move responsibility for closing channel endpoints on shutdown from the driver-conductor to sender and receiver as appropriate.
- Don't wait for Receiver to close a channel endpoint within the driver conductor. Issue #338.
- Remove use of
RandomAccessFile
so that onlyFileChannel
s are used to reference files. - Improve efficiency of startup.
- Upgrade to Agrona 0.9.5.
- Upgrade to Mockito 2.7.22.
- Upgrade to Gradle 3.5.
- Update CnC file to accommodate counters with labels up to 380 bytes. This bumps the CNC_VERSION so all clients and the driver on the same machine need to be upgraded at the same time.
- Truncate log buffers when a stream is closed so space can be reclaimed as soon as possible.
- Add
Image.controlledPeek
to peek ahead into a stream without advancing the subscriber position. - Improve the efficiency of Aeron startup and shutdown.
- Back pressure a Publication as soon as its timeout is triggered due to not receiving status messages so that low throughput connections signal NOT_CONNECTED sooner. Also applies to IPC when all Subscriptions are closed.
- Add per
Subscription
callbacks for notification ofImage
availability and unavailability. - Make media driver conductor more efficient so it can deal with a higher number of connected streams.
- Add the ability to create an
ExclusivePublication
with initial state set so it can replay existing archived data. - Add
ExclusivePublication
for higher throughput single threaded publisher. - Allow the Aeron client to start before the media driver when an existing CnC file is present.
- Clean up
Aeron.Context
if an error occurs when trying to establish a connection with the media driver. - Add
Aeron.nextCorrelationId()
. - Avoid a recursive close in the Aeron Java client when a timeout occurs.
- Upgrade to sevntu-checks 1.23.1.
- Upgrade to Checkstyle 7.6.1.
- Upgrade to Byte Buddy 1.6.12.
- Upgrade to Mockito 2.7.21.
- Upgrade to Agrona 0.9.4.
- Fix bug with managing the lifetime of
Subscription
s to IPCPublication
s when a publication is added and removed multiple times in quick succession without break on the same stream id. - Improvements to C++ client for building on Windows.
- Fix bug with IPC
Subscription
s being linked multiple times whenPublication
added from different clients that match. - Transition IPC
Publication
s to inactive as soon as the Publication reference count reaches zero.
- Fix bug with IPC
Publication
s goingNOT_CONNECTED
with very slow subscribers that are still connected. - Grow fragment assembly buffer using the golden ratio rather than doubling on a
Subscription
. - Improve Javadoc for Multi-Destination-Cast params.
- Upgrade to SBE 1.6.0.
- Multi-Destination-Cast for multicast like semantics over multiple UDP unicast streams.
- Linger networked
Publication
s that have spies until the spies have caught up with the sender position. - Manage
Image
availability andSubscription
s for IPC in the same way as for network streams. - Performance improvements from fixing a few false sharing issues.
- Fix network frame logging with debugging agent.
- Fix intermittent system tests on Windows.
- Make the Java client more robust to misuse.
- Windows C++ compilation fixes.
- Be more strict with validating client parameters before mutating state.
- Ability to set the MTU length on individual UDP streams.
- Upgrade to Checkstyle 7.5.1.
- Upgrade to Byte Buddy 1.6.9.
- Upgrade to Mockito 2.7.9.
- Upgrade to Agrona 0.9.3.
- Numerous small performance improvements.
- Fix bug with publications going NOT_CONNECTED because nano time was used rather than the epoch clock.
- Ability to subscribe to a stream as
reliable=false
which enables gaps being filled without invoking the reliable recovery process. - Loss Reporting by stream for monitoring.
- OSI layer 4 congestion control/avoidance service which can optionally be enabled on congested networks.
- Support for running 32-bit C++ clients on ARM.
- Rework of loss detection to make it more efficient.
- Ability to measure RTT added to the network protocol.
- Upgrade to Mockito 2.6.8.
- Upgrade to Checkstyle 7.4.
- Upgrade to sevntu-checks 1.23.0.
- Upgrade to ByteBuddy 1.6.5.
- Upgrade to Agrona 0.9.2.
- Upgrade to Gradle 3.3.
- Use
BusySpinIdleStrategy
for the conductor in theLowLatencyMediaDriver
to reduce some latency outliers at the expense of more CPU usage. - Fix some cases of false sharing on key data structures.
- Remove OSGi plugin from aeron-agent to fix issue that prevented debug logging due to premain not in manifest.
- Allow driver and examples to be shutdown with SIGTERM as well as SIGINT.
- New flow control strategies for minimum and preferred receivers on multicast as additions to existing max strategy.
- Add receiver ID to status messages to enable additional flow control strategies such as preferred receiver.
- Remove deprecated channel URI format support.
- Added warm up cycle to C++ ping pong example.
- Apply
const
where possible in C++ client code. - Fix checks against max message and max payload length in C++ publication claim.
- Upgrade to Mockito 2.2.29.
- Upgrade to Checkstyle 7.3.
- Upgrade to sevntu-checks 1.22.0.
- Upgrade to ByteBuddy 1.5.9.
- Upgrade to Agrona 0.9.1.
- Upgrade to Gradle 3.2.1.
- Detect Aeron Client use after close operations and improve locking strategy.
- Park the client conductor when waiting on the driver response.
- Fix concurrency issues in system tests.
- Added OSGi Gradle plugin.
- Upgrade to Mockito 2.2.16.
- Upgrade to sevntu-checks 1.21.1.
- Upgrade to ByteBuddy 1.5.5.
- Upgrade to Agrona 0.9.0.
- Upgrade to Gradle 3.2.
- Linger client resources before informing driver to remove reference to avoid race when client is suffering resource starvation.
- Sleep for less than a millisecond between polls for completed admin actions to make setup more responsive.
- Remove subscription spies before cleaning up
NetworkPublication
s to avoid potential SIGEV. - Use project based file reference to allow for composable Gradle builds.
- Increase default low file space warning to 10 times term length.
- Upgrade to Mockito 2.2.9.
- Upgrade to Checkstyle 7.2.
- Upgrade to ByteBuddy 1.5.0.
- Upgrade to Agrona 0.5.6.
- Bugfix for double free of counters that resulted in issues #269 and #281.
- Enable high resolution timer on Windows for more regular polling.
- Add sender limit as a counter for extra telemetry.
- Avoid false sharing on
Sender
andSubscription
round robin counters. - Make rebuilding of
PublicationImage
s a pure monotonic function to avoid unlikely but possible regression in state. - Validate flow control window before accepting a heartbeat message.
- Poll for control messages as a ratio to data packets sent, default to 4, to allow for scaling to larger numbers of connections.
- Allow multi-digit version numbering with C++ build.
- Inline logging code when activated rather than delegating for better performance.
- Low file space warnings and error checks for sufficient space in volume holding log files.
- Update to ByteBuddy 1.4.32.
- Update to Gradle 3.1.
- Update to Agrona 0.5.5.
- Adjust incremental cleaning to avoid overwrite of cleaned log in loss scenarios to address Issue #271.
- Correctly initialise cleaning position for late joining a stream. Issue #268.
- Make
clientLivenessTimeout
configurable fromMediaDriver.Context
. - Shadow samples into aeron-all JAR.
- Support creating Agent threads via
ThreadFactory
to enable pinning. - Update to Agrona 0.5.4.
- Update to ByteBuddy 1.4.20.
- Expose maxPayloadLength Publications.
- Added setting clientId to remove messages from clients.
- Reduce the number of data dependent loads and indirections for reduced cache missing and memory footprint.
- Simplified Java version of DriverProxy.
- Driver is more conservative about sending available images for new Subscriptions with images in LINGER.
- NAKs must be within a single term length of the sender to cause a retransmission.
- Incremental cleaning of log buffers to smooth out pauses by the driver conductor.
- Java client API will yield the current thread when spinning waiting for response from the driver.
- Java and C++ APIs will attempt to make sure driver is completed startup before using CnC file.
- Per term metadata moved into the log metadata section of the logbuffer.
- C++ API term appending now safer for unblock operation.
- Simplification and reorg of Javadoc and Configuration.
- Update to Agrona 0.5.3
- Update to Checkstyle 7.0.
- Update to ByteBuddy 1.4.9.
- Numerous small improvements for performance and consistent latency.
- Default the rebuilt images to be read-only when memory-mapped in the Java client.
- Move signalling of term buffers for cleaning from the client to the driver. This reduces latency on rotation.
- Enable the
LogInspector
to scan over gaps in the term buffers. - Constrain in-flight windows that have been miss-configured to fit within term buffers.
- Counters for receiver rebuild position and joining position on streams for subscribers.
- Reduce memory footprint and processing for NAKs.
- Fix for stream sticking when loss occurred on the last frame in a term. Issue #245
- Removed the need for
aeron.rcv.buffer.length
as a configuration option. - The ability to set the reserved value field in the header when calling a overloaded
Publication.offer()
taking a function that returns the value. Useful for applying a checksum or timestamp. - The ability to spy on an outgoing network stream as an IPC Subscription. Useful for archiving. Issue #230
- Warn when old style channel URIs are used and update all samples to use the new format.
- Ability to set term length as a URI param to the channel when adding a stream. This overrides the defaults for
aeron.term.buffer.length
andaeron.ipc.term.buffer.length
. Issue #193 - Target minimum of CMAKE 3.0.2 for c++.
- Validate MTU lengths are correct multiples and within range.
- Change
Image.filePoll()
toImage.rawPoll()
and allow access to the underlyingUnsafeBuffer
. This is useful for adding things like a checksum to header reserved value field. - Update to Agrona 0.5.2.
- Update to HdrHistogram 2.1.9.
- Update to ByteBuddy 1.4.5.
- Update to Checkstyle 6.19.
- Update to Gradle 2.14.
- Change samples to use aligned buffers for better copy performance.
- Ability to check the status of a channel to see socket connect status. This can be seen in the counters with
AeronStat
. - Change low-latency samples to not use sparse files to avoid page faults.
- Move channel endpoint dispatch logic to the pollers for simplicity. This is breaking for existing channel endpoint extensions that overrode dispatch.
- Use only one copy of send buffers and flyweights for control messages from
NetworkPublication
s to allow for better scalability of streams. - Pull out receive buffers and flyweights from channels into
TransportPoller
s so only one instance of each is used regardless of how many channels are used. - Use aligned
ByteBuffer
s for receiving data and control messages. - Add Bash and Windows batch file scripts for running samples.
- Update to ByteBuddy 1.3.19.
- Update to Agrona 0.5.1.
- Update to Checkstyle 6.18.
- Update to Gradle 2.13.
- Change
Header.position()
in Subscription callback to include alignment in calculated resulting position. - Added
Subscription.forEachImage()
to allow allocation free iteration over active Images on a Subscription.
- Frame alignment changed to 32 bytes. Improving Java IPC byte throughput. (Not backwards compatible).
- Frame header increased to 32 bytes. Allowing inclusion of a 64-bit user supplied value when sending. (Not backwards compatible).
- Simplified network publication cleanup.
- Update to Agrona 0.5.
- Update to Checkstyle 6.17
- Migrate package layout from
uk.co.real_logic.aeron
->io.aeron
(Breaking for existing code). - Allow flow control flexibility to vary by distinct streams.
- Change event logging to be Java agent based using byte code weaving.
- Ability to globally set TTL for multicast by system property or channel specific by uri param.
- Expand
AeronStat
to allow filtering based on counters metadata. - Update to Agrona 0.4.13.
- Update to Checkstyle 6.16.1.
- Update to Gradle 2.12.
- Bug fix for NPE due to errorLog not being setup correctly.
- Save errors from existing CnC file on
MediaDriver
restart. - Extract loss generation to be cleanly separated for use in debug channel endpoints.
- Fix bug with negative sessionIds that can be generated with higher numbers of streams.
- A number of changes the enable faster creation of Publications and Subscriptions to support 100s of streams.
- Add new DistinctErrorLog to CnC buffer so errors don't go to the event log. This error log can be read with ErrorStat.
- Remove the need for the template file allowing for faster driver startup and save space in /dev/shm.
- Add
Publication.initialTermId()
accessor. - Call
onUnavailableImage()
for all Images contained in a Subscription when it is closed. - Support new channel syntax for endpoints.
- Added the ability to load multiple configuration files and merge them.
- Allow the creation of term buffers in a log to be sparse for faster connection setup.
- Add
Publication.isConnected()
method to check if a publication is currently connected to a subscriber. - Return
NOT_CONNECTED
from Publication when offer fails and no recent status messages have been seen. - Linger client Java Publications to help avoid segfaults when used after being closed.
- Update to Agrona 0.4.12.
- Update to Checkstyle 6.15.
- Update to Gradle 2.11.
- Add tracking of last status message (SM) to publications for knowing if they are no longer connected to a receiver.
- Removal of unused configuration parameters.
- Fix event reader output for unavailable image reporting.
- Fix control protocol flyweights to use native rather than little endian byte ordering.
- Configurable supplier support for channel endpoints for introducing channel specific behaviour such as debugging, monitoring, loss introduction, etc.
- Update to Agrona 0.4.11.
- Iterate SelectedKeys taking account of size when transport count greater than
ITERATION_THRESHOLD
. - Don't keep attempting to log an exception when event buffer is full.
- Return
ADMIN_ACTION
rather thanBACK_PRESSURED
from publication offer when another thread is rotating the log so that the publisher knows to retry.
- Added Checkstyle for indentation and fixed violations.
- Align C++ and Java on feature set for client API.
- Rework flyweights to use less memory and avoid one layer of indirection.
- Simplify the event logging code to be more efficient.
- Modified network publication to always send an initial SETUP upon first send() duty cycle. Adding tracking of last sent SETUP to effectively handle SM with SETUP implosion. Fixed sending of SETUP when triggered by SM with SETUP flag when position has not been advanced at all yet.
- Correct bug with incorrect minimum on producer position for unblocking a dead publication.
- Reduce memory footprint on quite a few data structures and reduced code path for setting up connections to speed startup time.
- Update to Agrona 0.4.10.
- Improved unblocking code to deal with more cases of clients dying.
- Reworked publication algorithm to be more robust on machines with limited CPU resource.
- Reduced the potential for cache missing when publication logs rotate.
- Updated IdleStrategies.
- Added ControlledFragmentHandler to control the polling progress and position reporting from within the handler.
- Simplified API for Image availability notification.
- Added position tracking accessors to the Publication.
- Improved error messages for insufficient buffer sizes related to sockets.
- Minor performance improvements for consistency.
- Update to Gradle 2.10.
- Update to Checkstyle 2.14.1.
- Update to HdrHistogam 2.1.8.
- Update to Agrona 0.4.9.
- Add ADMIN_ACTION as return from publication.
- Migrate client liveness timeout to CNC file.
- Bugfix for fragmentLimit on Subscription.poll().
- Ignore CloseChannelException on Publication.
- Small performance improvement to publication.
- Update to Agrona 0.4.8.
- Update to Gradle 2.9.
- Bugfix to RingBuffer for sending client commands.
- Update to Agrona 0.4.7.
- Ability to detect and fix a blocked Publication or command ring buffer.
- Configuration via a properties file that is in classpath, local directory, or on a webserver.
- Meta data available on Publications and Images for term length, source identity, stream details.
- Detection of closed Images on poll.
- Ability to abort a BufferClaim.
- Update to Gradle 2.8, Agrona 0.4.6 and Checkstyle 6.11.2.
- Latency and throughput improvements in the Publication path for both networked and IPC comms.
- Reduced workload on the conductor thread to better support larger number of streams or when running with shared mode for agents when threads need to be conserved.
- Apply Eclipse plugin to build files.
- Update to Agrona 0.4.5
- IPC transport for high throughput and low-latency messaging inter thread and process.
- Added configuration support via a properties file.
- Added IPC samples.
- Improved latency and throughput for
Publication
s. - Improved performance for receiving status and NAK messages.
- Update Gradle to 2.7, Agrona to 0.4.4, HdrHistogram to 2.1.7
- Experimentation IPC transport via shared memory.
- Major bugfix for handling late joining publications in other Aeron clients on the same driver.
- Added the ability to configure client keep-alives.
- Rename client callbacks and provide access to underlying images.
- Update to Agrona 0.4.3.
- Rename connections to images.
- Add NOT_CONNECTED semantics.
- Close all publications and subscriptions with an Aeron client when it is closed.
- Update to Agrona 0.4.2.
- Fix bug with FileChannel being closed on Image which preventing pollFile()
- Migrated assignment of publication session ids from client to the driver to easy the sharing of publications across processes.
- Update Gradle to 2.5, Checkstyle to 6.8.1, and Agrona to 0.4.1.
- Initial Beta Release