All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Added Kafka Raft for the Kafka-standalone app.
Removed zookeeper from Kafka-standalone app.
Update spring framework verison 6.1.14 to avoid vulnerability in webflux
Partial support of Active Profile using the "spring.profiles.active" parameter
Feature to ping a function without payload and headers.
- OSS update - vertx 2.5.10, spring boot 3.3.4
- Simplified api-playground application
N/A
N/A
- Updated documentation
- AsyncHttpRequest - remove the class type variable in the data model
- MultiLevelMap - update the removeElement method
- OSS update - GSON 2.11.0, classpath 4.8.174, guava 33.3.0-jre, vertx 2.5.9, spring boot 3.3.3, spring framework 5.3.39
This version supercedes 3.0.13 and 3.0.14 due to updated data structure for static content handling.
- Added optional static-content.no-cache-pages in rest.yaml
- AsyncHttpClientLoader
N/A
- Updated data structure for static-content section in rest.yaml
- Fixed bug for setting multiple HTTP cookies
- Unified configuration file prefix "yaml."
N/A
N/A
Updated syntax for static-content-filter section in rest.yaml
Added optional static content HTTP-GET request filter in rest.yaml
N/A
Updated guava to version 33.1.0-jre
N/A
N/A
Enhanced OptionalService annotation
Backport KernelThreadRunner from Mercury Composable 3.1
KernelThreadRunner annotation
CoroutineRunner annotation
-
Set default execution strategy to "coroutine". To tell the system to run a function using kernel thread pool, add the
KernelThreadRunnerannotation. -
Update PersistentWsClient to use vertx WebSocketClient.
-
Upgrade netty to version 4.1.109.Final.
Added "app-config-reader.yml" file in the resources folder so that you can override the default application configuration files.
N/A
- Open sources library update (Spring Boot 3.2.4, Vertx 4.5.5)
- Improve AppConfigReader and ConfigReader to use the app-config-reader.yml file.
- AutoStart to run application as Spring Boot if the rest-spring-3 library is packaged in app
- Bugfix: removed websocket client connection timeout that causes the first connection to drop after one minute
- Open sources library update (Spring Boot 3.2.2, Vertx 4.5.3 and MsgPack 0.9.8)
- Rename application parameter "event.worker.pool" to "kernel.thread.pool"
- Support user defined serializer with PreLoad annotation and platform API
N/A
ActiveMQ and Tibco cloud connectors
Updated AsyncHttpRequest and CryptoApi
Print out basic JVM information before startup for verification of base container image.
Removed Maven Shade packager
Updated open sources libraries to address security vulnerabilities
- Spring Boot 2/3 to version 2.7.18 and 3.2.1 respectively
- Tomcat 9.0.84
- Vertx 4.5.1
- Classgraph 4.8.165
- Netty 4.1.104.Final
- slf4j API 2.0.9
- log4j2 2.22.0
- Kotlin 1.9.22
- Artemis 2.31.2
- Hazelcast 5.3.6
- Guava 33.0.0-jre
Enhanced Benchmark tool to support "Event over HTTP" protocol to evaluate performance efficiency for commmunication between application containers using HTTP.
N/A
Updated open sources libraries
- Spring Boot 2/3 to version 2.7.17 and 3.1.5 respectively
- Kafka-client 3.6.0
Support two executable JAR packaging system:
- Maven Shade packager
- Spring Boot packager
Starting from version 3.0.5, we have replaced Spring Boot packager with Maven Shade. This avoids a classpath edge case for Spring Boot packager when running kafka-client under Java 11 or higher.
Maven Shade also results in smaller executable JAR size.
N/A
Updated open sources libraries
- Spring-Boot 2.7.16 / 3.1.4
- classgraph 4.8.163
- snakeyaml 2.2
- kotlin 1.9.10
- vertx 4.4.6
- guava 32.1.3-jre
- msgpack 0.9.6
- slj4j 2.0.9
- zookeeper 3.7.2
The "/info/lib" admin endpoint has been enhanced to list library dependencies for executable JAR generated by either Maven Shade or Spring Boot Packager.
Improved ConfigReader to recognize both ".yml" and ".yaml" extensions and their uses are interchangeable.
N/A
N/A
Updated open sources libraries
- Spring-Boot 2.7.14 / 3.1.2
- Kafka-client 3.5.1
- classgraph 4.8.161
- guava 32.1.2-jre
- msgpack 0.9.5
- File extension to MIME type mapping for static HTML file handling
N/A
- Open sources library update - Kotlin version 1.9.0
N/A
N/A
- Consistent exception handling for Event API endpoint
- Open sources lib update - Vertx 4.4.4, Spring Boot 2.7.13, Spring Boot 3.1.1, classgraph 4.8.160, guava 32.0.1-jre
In this release, we have replace Google HTTP Client with vertx non-blocking WebClient. We also tested compatibility up to OpenJDK version 20 and maven 3.9.2.
When "x-raw-xml" HTTP request header is set to "true", the AsyncHttpClient will skip the built-in XML serialization so that your application can retrieve the original XML text.
Retire Google HTTP client
Upgrade maven plugin versions.
This is a major release with some breaking changes. Please refer to Chapter-10 (Migration guide) for details. This version brings the best of preemptive and cooperating multitasking to Java (version 1.8 to 19) before Java 19 virtual thread feature becomes officially available.
- Function execution engine supporting kernel thread pool, Kotlin coroutine and suspend function
- "Event over HTTP" service for inter-container communication
- Support for Spring Boot version 3 and WebFlux
- Sample code for a pre-configured Spring Boot 3 application
- Remove blocking APIs from platform-core
- Retire PM2 process manager sample script due to compatibility issue
- Refactor "async.http.request" to use vertx web client for non-blocking operation
- Update log4j2 version 2.20.0 and slf4j version 2.0.7 in platform-core
- Update JBoss RestEasy JAX_RS to version 3.15.6.Final in rest-spring
- Update vertx to 4.4.2
- Update Spring Boot parent pom to 2.7.12 and 3.1.0 for spring boot 2 and 3 respectively
- Remove com.fasterxml.classmate dependency from rest-spring
N/A
N/A
- Improved load balancing in cloud-connector
- Filter URI to avoid XSS attack
- Upgrade to SnakeYaml 2.0 and patch Spring Boot 2.6.8 for compatibility with it
- Upgrade to Vertx 4.4.0, classgraph 4.8.157, tomcat 9.0.73
- standalone benchmark report app
- client and server benchmark apps
- add timeout tag to RPC events
N/A
- Updated open sources dependencies
- Netty 4.1.86.Final
- Tomcat 9.0.69
- Vertx 4.3.6
- classgraph 4.8.152
- google-http-client 1.42.3
- Improved unit tests to use assertThrows to evaluate exception
- Enhanced AsyncHttpRequest serialization
In this version, REST automation code is moved to platform-core such that REST and Websocket service can share the same port.
- AsyncObjectStreamReader is added for non-blocking read operation from an object stream.
- Support of LocalDateTime in SimpleMapper
- Add "removeElement" method to MultiLevelMap
- Automatically convert a map to a PoJo when the sender does not specify class in event body
N/A
- REST automation becomes part of platform-core and it can co-exist with Spring Web in the rest-spring module
- Enforce Spring Boot lifecycle management such that user apps will start after Spring Boot has loaded all components
- Update netty to version 4.1.84.Final
In this version, websocket notification example code has been removed from the REST automation system. If your application uses this feature, please recover the code from version 2.5.0 and refactor it as a separate library.
N/A
Simplify REST automation system by removing websocket notification example in REST automation.
- Replace Tomcat websocket server with Vertx non-blocking websocket server library
- Update netty to version 4.1.79.Final
- Update kafka client to version 2.8.2
- Update snake yaml to version 1.33
- Update gson to version 2.9.1
New Preload annotation class to automate pre-registration of LambdaFunction.
Removed Spring framework and Tomcat dependencies from platform-core so that the core library can be applied to legacy J2EE application without library conflict.
- Bugfix for proper housekeeping of future events.
- Make Gson and MsgPack handling of integer/long consistent
Updated open sources libraries.
- Eclipse vertx-core version 4.3.4
- MsgPack version 0.9.3
- Google httpclient version 1.42.2
- SnakeYaml version 1.31
Support more than one event stream cluster. User application can share the same event stream cluster for pub/sub or connect to an alternative cluster for pub/sub use cases.
N/A
Cloud connector libraries update to Hazelcast 5.1.2
Add tagging feature to handle language connector's routing and exception handling
Remove language pack's pub/sub broadcast feature
- Update Spring Boot parent to version 2.6.8 to fetch Netty 4.1.77 and Spring Framework 5.3.20
- Streamlined language connector transport protocol for compatibility with both Python and Node.js
N/A
- Remove swagger-ui distribution from api-playground such that developer can clone the latest version
- Update application.properties (from spring.resources.static-locations to spring.web.resources.static-locations)
- Update log4j, Tomcat and netty library version using Spring parent 2.6.6
Enhanced AsyncRequest to handle non-blocking fork-n-join
N/A
Upgrade Spring Boot from 2.6.3 to 2.6.6
Add support of queue API in native pub/sub module for improved ESB compatibility
N/A
N/A
N/A
N/A
- Update Vertx to version 4.2.4
- Update Tomcat to version 5.0.58
- Use Tomcat websocket server for presence monitors
- Bugfix - Simple Scheduler's leader election searches peers correctly
N/A
N/A
- Update copyright notice
- Update Vertx to version 4.2.3
- Bugfix - RSA key generator supporting key length from 1024 to 4096 bits
- CryptoAPI - support different AES algorithms and custom IV
- Update Spring Boot to version 2.6.3
- Transaction journaling
- Add parameter
distributed.trace.aggregationin application.properties such that trace aggregation may be disabled.
N/A
- Update JBoss RestEasy library to 3.15.3.Final
- Improved po.search(route) to scan local and remote service registries. Added "remoteOnly" selection.
- Fix bug in releasing presence monitor topic for specific closed user group
- Update Apache log4j to version 2.17.1
- Update Spring Boot parent to version 2.6.1
- Update Netty to version 4.1.72.Final
- Update Vertx to version 4.2.2
- Convenient class "UserNotification" for backend service to publish events to the UI when REST automation is deployed
- User defined API authentication functions can be selected using custom HTTP request header
- "Exception chaining" feature in EventEnvelope
- New "deferred.commit.log" parameter for backward compatibility with older PowerMock in unit tests
N/A
- Improved and streamlined SimpleXmlParser to handle arrays
- Bugfix for file upload in Service Gateway (REST automation library)
- Update Tomcat library from 9.0.50 to 9.0.54
- Update Spring Boot library to 2.5.6
- Update GSON library to 2.8.9
Callback function can implement ServiceExceptionHandler to catch exception. It adds the onError() method.
N/A
Open sources library update - Vert.x 4.1.3, Netty 4.1.68-Final
- User defined PoJo and Generics mapping
- Standardized serializers for default case, snake_case and camelCase
- Support of EventEnvelope as input parameter in TypedLambdaFunction so application function can inspect event's metadata
- Application can subscribe to life cycle events of other application instances
N/A
- Replace Tomcat websocket server engine with Vertx in presence monitor for higher performance
- Bugfix for MsgPack transport of integer, long, BigInteger and BigDecimal
- Multicast - application can define a multicast.yaml config to relay events to more than one target service.
- StreamFunction - function that allows the application to control back-pressure
"object.streams.io" route is removed from platform-core
- Elastic Queue - Refactored using Oracle Berkeley DB
- Object stream I/O - simplified design using the new StreamFunction feature
- Open sources library update - Spring Boot 2.5.2, Tomcat 9.0.50, Vert.x 4.1.1, Netty 4.1.66-Final
Vert.x is introduced as the in-memory event bus
- ActiveMQ and Tibco connectors
- Admin endpoints to stop, suspend and resume an application instance
- Handle edge case to detect stalled application instances
- Add "isStreamingPubSub" method to the PubSub interface
- Event Node event stream emulator has been retired. You may use standalone Kafka server as a replacement for development and testing in your laptop.
- Multi-tenancy namespace configuration has been retired. It is replaced by the "closed user group" feature.
- Refactored Kafka and Hazelcast connectors to support virtual topics and closed user groups.
- Updated ConfigReader to be consistent with Spring value substitution logic for application properties
- Replace Akka actor system with Vert.x event bus
- Common code for various cloud connectors consolidated into cloud core libraries
Version 1.13.0 is the last version that uses Akka as the in-memory event system.
- A simple websocket notification service is integrated into the REST automation system
- Seamless migration feature is added to the REST automation system
Legacy websocket notification example application
N/A
- "kafka.pubsub" is added as a cloud service
- File download example in the lambda-example project
- "trace.log.header" added to application.properties - when tracing is enabled, this inserts the trace-ID of the transaction in the log context. For more details, please refer to the Developer Guide
- Add API to pub/sub engine to support creation of topic with partitions
- TypedLambdaFunction is added so that developer can predefine input and output classes in a service without casting
N/A
- Decouple Kafka pub/sub from kafka connector so that native pub/sub can be used when application is running in standalone mode
- Rename "relay" to "targetHost" in AsyncHttpRequest data model
- Enhanced routing table distribution by sending a complete list of route tables, thus reducing network admin traffic.
If predictable topic is set, application instances will report their predictable topics as "instance ID" to the presence monitor. This improves visibility when a developer tests their application in "hybrid" mode. i.e. running the app locally and connect to the cloud remotely for event streams and cloud resources.
N/A
N/A
N/A
N/A
Improved Kafka producer and consumer pairing
New presence monitor's admin endpoint for the operator to force routing table synchronization ("/api/ping/now")
N/A
Improved routing table integrity check
Event stream systems like Kafka assume topic to be used long term. This version adds support to reuse the same topic when an application instance restarts.
You can create a predictable topic using unique application name and instance ID. For example, with Kubernetes, you can use the POD name as the unique application instance topic.
N/A
N/A
Automate trace for fork-n-join use case
N/A
N/A
N/A
N/A
Improved distributed trace - set the "from" address in EventEnvelope automatically.
N/A
N/A
Application life-cycle management - User provided main application(s) will be started after Spring Boot declares web application ready. This ensures correct Spring autowiring or dependencies are available.
Bugfix for locale - String.format(float) returns comma as decimal point that breaks number parser. Replace with BigDecimal decimal point scaling.
Bugfix for Tomcat 9.0.35 - Change Async servlet default timeout from 30 seconds to -1 so the system can handle the whole life-cycle directly.
- new "search" method in Post Office to return a list of application instances for a service
- simple "cron" job scheduler as an extension project
- add "sequence" to MainApplication annotation for orderly execution when more than one MainApplication is available
- support "Optional" object in EventEnvelope so a LambdaFunction can read and return Optional
N/A
- The rest-spring library has been updated to support both JAR and WAR deployment
- All pom.xml files updated accordingly
- PersistentWsClient will back off for 10 seconds when disconnected by remote host
-
Payload segmentation
For large payload in an event, the payload is automatically segmented into 64 KB segments. When there are more than one target application instances, the system ensures that the segments of the same event is delivered to exactly the same target.
-
PersistentWsClient added - generalized persistent websocket client for Event Node, Kafka reporter and Hazelcast reporter.
N/A
- Code cleaning to improve consistency
- Upgraded to hibernate-validator to v6.1.5.Final and Hazelcast version 4.0.1
- REST automation is provided as a library and an application to handle different use cases
N/A
N/A
For security reason, upgrade log4j to version 2.13.2
Use RestEasy JAX-RS library
For security reason, removed Jersey JAX-RS library
- Updated RestLoader to initialize RestEasy servlet dispatcher
- Support nested arrays in MultiLevelMap
N/A
For simplicity, retire route-substitution admin endpoint. Route substitution uses a simple static table in route-substitution.yaml.
N/A
N/A
SimpleRBAC class is retired
- Improved ConfigReader and AppConfigReader with automatic key-value normalization for YAML and JSON files
- Improved pub/sub module in kafka-connector
N/A
Retired proprietary config manager since we can use the "BeforeApplication" approach to load config from Kubernetes configMap or other systems of config record.
- Added "isZero" method to the SimpleMapper class
- Convert BigDecimal to string without scientific notation (i.e. toPlainString instead of toString)
- Corresponding unit tests added to verify behavior
N/A
N/A
Kafka-connector will shutdown application instance when the EventProducer cannot send event to Kafka. This would allow the infrastructure to restart application instance automatically.
N/A
N/A
- Kafka-connector now supports external service provider for Kafka properties and credentials. If your application implements a function with route name "kafka.properties.provider" before connecting to cloud, the kafka-connector will retrieve kafka credentials on demand. This addresses case when kafka credentials change after application start-up.
- Interceptors are designed to forward requests and thus they do not generate replies. However, if you implement a function as an EventInterceptor, your function can throw exception just like a regular function and the exception will be returned to the calling function. This makes it easier to write interceptors.
- Expose "async.http.request" as a PUBLIC function ("HttpClient as a service")
N/A
- Improved Hazelcast client connection stability
- Improved Kafka native pub/sub
- Rest-automation will transport X-Trace-Id from/to Http request/response, therefore extending distributed trace across systems that support the X-Trace-Id HTTP header.
- Added endpoint and service to shutdown application instance.
N/A
- Updated SimpleXmlParser with XML External Entity (XXE) injection prevention.
- Bug fix for hazelcast recovery logic - when a hazelcast node is down, the app instance will restart the hazelcast client and reset routing table correctly.
- HSTS header insertion is optional so that we can disable it to avoid duplicated header when API gateway is doing it.
Feature to disable PoJo deserialization so that caller can decide if the result set should be in PoJo or a Map.
N/A
- Simplified key management for Event Node
- AsyncHttpRequest case insensitivity for headers, cookies, path parameters and session key-values
- Make built-in configuration management optional
Added HTTP relay feature in rest-automation project
N/A
- Improved hazelcast retry and peer discovery logic
- Refactored rest-automation's service gateway module to use AsyncHttpRequest
- Info endpoint to show routing table of a peer
- Simple configuration management is added to event-node, hazelcast-presence and kafka-presence monitors
- Added
BeforeApplicationannotation - this allows user application to execute some setup logic before the main application starts. e.g. modifying parameters in application.properties - Added API playground as a convenient standalone application to render OpenAPI 2.0 and 3.0 yaml and json files
- Added argument parser in rest-automation helper app to use a static HTML folder in the local file system if
arguments
-html file_pathis given when starting the JAR file.
N/A
- Kafka publisher timeout value changed from 10 to 20 seconds
- Log a warning when Kafka takes more than 5 seconds to send an event
- getRoute() method is added to PostOffice to facilitate RBAC
- The route name of the current service is added to an outgoing event when the "from" field is not present
- Simple RBAC using YAML configuration instead of code
N/A
Updated Spring Boot to v2.2.1
Multi-tenancy support for event streams (Hazelcast and Kafka). This allows the use of a single event stream cluster for multiple non-prod environments. For production, it must use a separate event stream cluster for security reason.
N/A
- logging framework changed from logback to log4j2 (version 2.12.1)
- Use JSR-356 websocket annotated ClientEndpoint
- Improved websocket reconnection logic
- Distributed tracing implemented in platform-core and rest-automation
- Improved HTTP header transformation for rest-automation
N/A
language pack API key obtained from environment variable
N/A
rest-core subproject has been merged with rest-spring
N/A
- Periodic routing table integrity check (15 minutes)
- Set kafka read pointer to the beginning for new application instances except presence monitor
- REST automation helper application in the "extensions" project
- Support service discovery of multiple routes in the updated PostOffice's exists() method
- logback to set log level based on environment variable LOG_LEVEL (default is INFO)
N/A
Minor refactoring of kafka-connector and hazelcast-connector to ensure that they can coexist if you want to include both of these dependencies in your project.
This is for convenience of dev and testing. In production, please select only one cloud connector library to reduce memory footprint.
Add inactivity expiry timer to ObjectStreamIO so that house-keeper can clean up resources that are idle
N/A
- Disable HTML encape sequence for GSON serializer
- Bug fix for GSON serialization optimization
- Bug fix for Object Stream housekeeper
By default, GSON serializer converts all numbers to double, resulting in unwanted decimal point for integer and long. To handle custom map serialization for correct representation of numbers, an unintended side effect was introduced in earlier releases.
List of inner PoJo would be incorrectly serialized as map, resulting in casting exception. This release resolves this issue.
- Store-n-forward pub/sub API will be automatically enabled if the underlying cloud connector supports it. e.g. kafka
- ObjectStreamIO, a convenient wrapper class, to provide event stream I/O API.
- Object stream feature is now a standard feature instead of optional.
- Deferred delivery added to language connector.
N/A
N/A
- Route substitution for simple versioning use case
- Add "Strict Transport Security" header if HTTPS (https://tools.ietf.org/html/rfc6797)
- Event stream connector for Kafka
- Distributed housekeeper feature for Hazelcast connector
System log service
Refactoring of Hazelcast event stream connector library to sync up with the new Kafka connector.
Language-support service application for Python, Node.js and Go, etc. Python language pack project is available at https://github.com/Accenture/mercury-python
N/A
- replace Jackson serialization engine with Gson (
platform-coreproject) - replace Apache HttpClient with Google Http Client (
rest-spring) - remove Jackson dependencies from Spring Boot (
rest-spring) - interceptor improvement
N/A
N/A
- Move safe.data.models validation rules from EventEnvelope to SimpleMapper
- Apache fluent HTTP client downgraded to version 4.5.6 because the pom file in 4.5.7 is invalid
Added retry logic in persistent queue when OS cannot update local file metadata in real-time for Windows based machine.
N/A
pom.xml changes - update with latest 3rd party open sources dependencies.
platform-core
- Support for long running functions so that any long queries will not block the rest of the system.
- "safe.data.models" is available as an option in the application.properties. This is an additional security measure to protect against Jackson deserialization vulnerability. See example below:
#
# additional security to protect against model injection
# comma separated list of model packages that are considered safe to be used for object deserialization
#
#safe.data.models=com.accenture.models
rest-spring
"/env" endpoint is added. See sample application.properties below:
#
# environment and system properties to be exposed to the "/env" admin endpoint
#
show.env.variables=USER, TEST
show.application.properties=server.port, cloud.connector
N/A
platform-core
Use Java Future and an elastic cached thread pool for executing user functions.
N/A
Hazelcast support is added. This includes two projects (hazelcast-connector and hazelcast-presence).
Hazelcast-connector is a cloud connector library. Hazelcast-presence is the "Presence Monitor" for monitoring the presence status of each application instance.
platform-core
The "fixed resource manager" feature is removed because the same outcome can be achieved at the application level. e.g. The application can broadcast requests to multiple application instances with the same route name and use a callback function to receive response asynchronously. The services can provide resource metrics so that the caller can decide which is the most available instance to contact.
For simplicity, resources management is better left to the cloud platform or the application itself.
N/A
N/A