All notable changes to this project will be documented in this file, which follows the guidelines on Keep a CHANGELOG. This project adheres to Semantic Versioning.
- Fixed spelling mistake in OversizeMessageGuard error message
- New method 'payloadIsNull()' on DefaultJsonEnvelope, to check if the payload is
JsonValue.NULL
ornull
- Fix where null payloads of JsonEnvelopes get converted to
JsonValue.NULL
and cause a ClassCastException - All JsonEnvelopes that have null payloads will now:
- return
JsonValue.NULL
ifgetPayload()
is called - throw
IncompatibleJsonPayloadTypeException
ifgetPayloadAsJsonObject()
is called - throw
IncompatibleJsonPayloadTypeException
ifgetPayloadAsJsonArray()
is called - throw
IncompatibleJsonPayloadTypeException
ifgetPayloadAsJsonString()
is called - throw
IncompatibleJsonPayloadTypeException
ifgetPayloadAsJsonNumber()
is called
- return
- Fixed the percentage of times that HIGH, MEDIUM and LOW priority jobs are run
- Fixed test library accidentally put on compile scope
- Update framework-libraries to 17.6.4 in order to
- Improve the fetching of jobs by priority from the jobstore by retrying with a different priority if the first select returns no jobs
- Update framework libraries to 17.6.3 in order to:
- Refactor of File Store to merge file store 'metadata' table into the 'content' table.
- File Store now only contains one table
- Update framework libraries to 17.6.2 in order to:
- Update jobstore to process tasks with higher priority first
- Fix for Jackson single argument constructor issue inspired from FasterXML/jackson-databind#1498
- The catchup process can now whitelist event sources to catchup
- New Jndi value can be set to
ALLOW_ALL
to allow all
- New Jndi value
java:global/catchup.event.source.whitelist
for a comma separated list of whitelisted event-sources for catchup.
- Fetch of PublishedEvents during catchup now correctly uses MultipleDataSourcePublishedEventRepository
- All events pulled from the event queue by the message driven bean now
check the size of the message, and will log an error if the number of bytes
is greater than a new jndi value
messaging.jms.oversize.message.threshold.bytes
- All rest http parameters in the generated rest endpoints are now encoded using owasp to protect against cross site scripting
- Adds created_at column to snapshot table and populate this while saving new snapshot
- Fix transaction problem in ReplayEventToEventListerCommands by moving transaction boundry one layer down
- All streams now only have one (current) snapshot stored in the database. All older snapshots are deleted (CPI-890)
- Merged in release-17.x.x branch to keep master up to date
- Update framework to 17.6.0
- Add REPLAY_EVENT_TO_EVENT_LISTENER and REPLAY_EVENT_TO_EVENT_INDEXER system command handlers to replay single event
- Add maven-sonar-plugin to pluginManagement (through maven-parent-pom)
- Update framework to 17.5.0 for:
- JmsMessageConsumerClientProvider now returns JmsMessageConsumerClient interface rather than the implemening class
- Add method to SystemCommanderMBean interface to invoke system command without supplying CommandRunMode (through micro-service-framework changes)
- Break dependency on framework-command-client in test-utils-jmx library (by micro-service-framework changes)
- Add REPLAY_EVENT_TO_EVENT_LISTENER and REPLAY_EVENT_TO_EVENT_INDEXER system command handlers to replay single event
- Invoke eventBufferProcessor in transaction while replaying REPLAY_EVENT_TO_EVENT_LISTENER/REPLAY_EVENT_TO_EVENT_INDEXER system commands
- Release microservice-framework changes for adding junit CloseableResource that manages closing jms resources
- Release micro-service-framework changes for adding jms message clients for effective management of jms resources in integration tests
- Add jobstore retry migration liquibase script via framework-libraries
- Add retry mechanism to jobstore via framework-libraries
- Update framework to 17.4.0
- Update common-bom to 17.2.1
- Added dependencies required by various contexts to the framework-libraries-bom
- Added unifiedsearch-core and unifiedsearch-client-generator dependencies to framework-bom
- Centralise all generic library dependencies and versions into maven-common-bom
- Update common-bom to 17.2.0
- Removed dependency on apache-drools as it's not used by any of the framework code
- Update common-bom to fix various security vulnerabilities in org.json, plexus-codehaus, apache-tika and google-guava
- Fixed IndexOutOfBoundsException in ProcessedEventStreamSpliterator during catchup
- Remove
clientId
from the header of all generated Message Driven Beans
- Update junit to 5, surefire and failsafe plugins
- Fix Logging of missing event ranges to only log on debug
- Limit logging of MissingEventRanges logged to sensible maximum number.
- New JNDI value
catchup.max.number.of.missing.event.ranges.to.log
- Update org.json to version 20230227 to fix security vulnerability CVE-2022-45688 Detail: https://nvd.nist.gov/vuln/detail/CVE-2022-45688
- Fix batch fetch of processed events to load batches of 'n' events into a Java List in memory
- Update framework-libraries to 17.0.1 in order to:
- Remove unnecessary logging of 'skipping generation' message in pojo generator
- Update to Java 17
- Pojo generator fixed to handle null for additionalProperties
- Update common-bom to 17.0.0 in order to:
- Add byte-buddy 1.12.22 as a replacement for cglib
- Downgrade h2 to 1.4.196 as 2.x.x is too strict for our tests
- Update framework-libraries to 17.0.0 in order to:
- Change 'additionalProperties' Map in generated pojos to HashMap to allow serialization
- Remove illegal-access argument from surefire plugin from plugin management (through maven-parent-pom 17.0.0-M6)
- Remove illegal-access argument (not valid for java 17) from sure fire plugin
- Removed unnecessary indexes from event_log table
- Downgraded maven minimum version to 3.3.9 until the pipeline maven version is updated
- Updated to Java 11
- Bumped version to 11 to match new framework version
- Update to JEE 8
- Updated JdbcBasedEventRepository to insert eventIs into the pre_publish_queue table directly
- Provide constant for artemis healthcheck name
- Provide capability to override destination names while performing artemis healthcheck
- Upgrade framework to access messaging-jms dependency through framework-bom
- Updated slf4j/log4j bridge jar from slf4j-log4j12 to slf4j-reload4j
- MessageProducerClient and MessageConsumerClient are now idempotent when start is called
- Moved healthcheck database table checker into microservices-framework
- Update liquibase to 4.10.0
- New healthcheck module
- Healthchecks for viewstore, eventstore, jobstore and filestore
- Added new healthcheck for the system database.
- Added healthcheck modules into the event store bom
- Add artemis health check implementation which gets automatically registered by existing healthcheck mechanism
- Removed log4j-over-slf4j as it is now replaced by slf4j-reload4j
- Removed all old hamcrest libraries
- Removed strict checking of liquibase.properties files
- Removed the trigger from the event_log table and all related classes and commands
- Removed dependency on liquibase jars from test-utils-persistence
- Update hibernate version to 5.4.24.Final
- Update jackson.databind version to 2.12.7.1
- Update jackson libraries to 2.12.7
- Update wildfly to version 26.1.2.Final
- Update artemis to version 2.20.0
- Update resteasy-client to version 4.7.7.Final
- Updated framework to version 7.2.2
- Updated framework to version 7.2.1
- Added support for FeatureControl toggling by annotating service component handler methods with @FeatureControl
- Moved timer bean utilities to framework-libraries
- Updated framework to version 7.2.0
- Updated framework-libraries to version 7.1.5
- Builders of generated pojos now have a
withValuesFrom(...)
method to allow the builder to be initialised with the values of another pojo instance
- Security updates to apache.tika, commons.beanutils, commons.guava and junit in common-bom
- Updated common-bom to 7.1.1
- The context name is now used for creating JMS destination name if the event is an administration event
- Updated framework-libraries to version 7.1.1
- Updated parent maven-framework-parent-pom to version 2.0.0
- Updated framework to version 7.1.0
- Moved to new Cloudsmith.io repository for hosting maven artifacts
- Updated encrypted properties in travis.yaml to point to cloudsmith
- Update framework-libraries to 7.0.11
- Changed the removal of the event_log trigger to be called by a ServletContextListener to fix call happening after database connections destroyed
- Updated framework to 7.0.10
- Updated framework to 7.0.9
- indexes added to stream_id and position_in_stream in event_log table
- DefaultEventStoreDataSourceProvider changed to a singleton, so the caching of the DataSource works properly
- Test util class DatabaseCleaner has an additional method 'cleanEventStoreTables(...)' for truncating specified tables in the event-store
- Update framework to 7.0.8
- Update framework to 7.0.7
- Updated travis.yaml security setting for bintray to use the new cjs user
- Update framework to 7.0.6
- Update framework to 7.0.5
- Update framework to 7.0.4
- jboss-ejb3-ext-api from dependency-management
- Changed dependencies on only depend on framework 7.0.3
- Bumped version to 7.0.0 to match other framework libraries
- Update framework to 6.4.2
- Github issues
- Added a test DataSource for the file-service database
- Added indexes to processed_event table
- microservice-framework -> 6.4.1
- Fail cleanly if exception occurs while accessing subscription event source
- Inserts into the event-buffer no longer fails if there is a conflict; it just logs a warning
- Event store now works with multiple event sources
- Event store now compatible with contexts that do not have a command pillar
- Extracted all command pillar SystemCommands into their own module
- Catchup for multiple components now run in order of component and subscription priority
- Added event source name to catchup logger output
- Fixed catchup error where catchup was marked as complete after all subscriptions rather than all components
- Remove mechanism to also drop/add trigger on SUSPEND/UNSUSPEND as it causes many strange ejb database errors
- Added mechanism to also drop/add trigger to event_log table on SUSPEND/UNSUSPEND commands
- Fixed potential problem of a transaction failing during catchup causing catchup to never complete
- Backpressure added to the event processing queues during catchup
- Verification completion log message now correctly logs if verification of Catchup or of Rebuild
- Catchup Log message to show that all active events are now waiting to be consumed
- Now batch inserting PublishedEvents on rebuild to speed up the command
- Changed batch size on PublishedEvent rebuild to 1,000 per batch
New SystemCommand VERIFY_REBUILD to verify the results of of the rebuild
- Verifies that the number of active events in event_log matches the number of events in published_event
- Verifies that each event_number in published_event correctly links to an existing previous_event
- Verifies that each active stream has at least one event
- SHUTTER command renamed to SUSPEND
- UNSHUTTER command renamed to UNSUSPEND
- The database trigger for publishing on the event_log table is now added on application startup and removed on application shut down
- Updated to framework 6.4.0
- removed rogue logging of payload during event validation
- Added event_id to the processed_event table to aid debugging of publishing
- Event-Store SystemCommands moved into this project to break the dependency on framework
- New command 'ValidatePublishedEventsCommand' and handler for validating all events in event_log against their schemas
- Updated framework to 6.2.5
- Updated Json Schema Catalog to 1.7.6
- Improved the event_log query to determine if the renumber of events is complete. Changed to use select MAX rather than count(*)
- Removed asynchronous bean to run catchup queue and replaced with ManagedExecutor
- During catchup each event is now processed in a separate transaction
- Catchup range processing
- Pre publish and publish timer beans now run in a separate thread.
- New JNDI boolean values of 'pre.publish.disable' and 'publish.disable' to disable the running of PrePublisherTimerBean and PublisherTimerBean respectively
- Error message of event linking verification now gives more accurate error messages based on whether the problem is in published_event or processed_event
- New SystemCommands EnablePublishingCommand and DisablePublishingCommand for enabling/disabling the publishing beans
- Catchup will check for all missing events in the processed_event table and Catchup only the missing event ranges
- VERIFY_CATCHUP now correctly marks its status as COMMAND_FAILED if any of the verification steps fail. Verification warnings are considered successful
- New table in System database 'system_command_status' for storing state of commands
- Updated framework to 6.2.0
- All system commands now store their state in the system_command_status table in the system database. This is to allow the JMX client to wait until the command has completed or failed before it exits.
- Now using CatchupCommand to determine if we are running Event or Indexer catchup
- Converted ShutteringExecutors to use the new ShutteringExecutor interface
- Added commandId to all SystemEvents
- All SystemCommand handlers now take a mandatory UUID, commandId.
- Moved MdcLogger to framework 'jmx-command-handling' module
- Updated to framework 6.1.1
- All SystemCommand handlers now take a mandatory UUID, commandId.
- New system event 'CatchupProcessingOfEventFailedEvent' fired if processing of any PublishedEvent during catchup fails
- All system events moved into their own module 'event-store-management-events'
- Unsuccessful catchups now logged correctly in catchup completion.
- Subscriptions are no longer run asynchronously during catchup. Change required for MI catchup.
- Event catchup and Indexer catchup now run the same code
- Fix single event rebuilding of published_event table
- Fix issue where more than 1000 inactive events stops the rebuild process
Run the renumbering of events in a batch
- Updated framework to 6.0.17
- Catchup verification logging now runs in MDC context
- New SystemCommand to verify the results of running catchup
- Verifies that the number of active events in event_log matches the number of events in published_event
- Verifies that the number of events in published_event matches the number of events in processed_event
- Verifies that the stream_buffer table is empty
- Verifies that each event_number in published_event correctly links to an existing previous_event
- Verifies that each event_number in processed_event correctly links to an existing previous_event
- Verifies that each active stream has at least one event
- Fixed name of publish queue trigger
- New SystemCommands AddTrigger and RemoveTrigger to manage the trigger on the event_log table
- MDC logging of the service context for JMX commands
- Use DefaultEnvelopeProvider in MetadataEventNumberUpdater directly to fix classloading errors during rebuild
- Add logging to catchup processing, log every 1000th event
- Moved the conversion of PublishedEvent to JsonObject for the publishing of catchup inside the multi-threaded code
- Use DefaultJsonEnvelopeProvider in EventConverted directly to fix classloading errors during rebuild
- The event-source-name is now always calculated from the name of the event and never from its source field
- Renumbering of events during rebuild is now run in batches to allow shorter transactions
- Process rebuild in pages of events
- Changed transaction type of EventCatchupProcessorBean to NEVER to fix timeouts for long running transactions
- Reduced the maximum runtime for each iteration of the publishing beans to 450 milliseconds
- Long running transaction during rebuild broken into separate transactions
- Update framework to 6.0.14
- Update framework to 6.0.12
- Events are now renumbered according to their date_created order during rebuild
- If the event source name in the JsonEnvelope is missing is calculated from the event name rather than throwing an exception
- Update framework to 6.0.11
- Update framework to 6.0.10
- Update framework to 6.0.9
- Update framework to 6.0.8
- Update framework to 6.0.7
- Update framework to 6.0.6
- Update framework to 6.0.5
- Update framework to 6.0.4
- Update framework to 6.0.3
- Update framework to 6.0.2
- Update framework to 6.0.1
- Event Catchup is now Observable using the JEE event system
- Observers for the shutter then catchup process
- Add Shuttering implementation to PublisherTimerBean
- Added an Observer for PublishedEvent Rebuild
- CommandHandlerQueueChecker to check the command handler queue is empty when shuttering
- Added implementation for updating StreamStatus table with component column
- Add Published Event Source
- Add event linked tracking for processed events
- Subscription prioritisation
- Catchup now callable from JMX bean
- Support to for linked events after stream transformation
- Java API to retrieve all stream IDs (active and inactive ones)
- truncate() method to the EventSource interface
- populate() method to the EventSource interface
- Added component to the event buffer to allow it to handle both the event listener and indexer
- Added component column to processed_event table to allow Event Listener and Indexer to create unique primary keys
- Catchup and rebuild moved to a new 'event-store-management' module
- Catchup and rebuild now use the new System Command jmx architecture
- Shuttering executors no longer register themselves but are discovered at system startup and registered
- Implement usage of new system database
- Pre-publish and publish timer beans run for a given timer max runtime. The max time can be set with JNDI values "pre.publish.timer.max.runtime.milliseconds" and "event.dequer.timer.max.runtime.milliseconds".
- Renamed subscription-repository to event-tracking
- Add event-buffer-core and event-subscription-registry pom entries
- Moved shuttering from the publishing timer bean into the command api in framework
- Events for catchup and rebuild moved in from framework-api
- Merged event-publisher modules into one
- Event-Buffer functionality with changes to StreamBuffer & StreamStatus tables to support EventIndexer
- PrePublishBean now limits the maximum number of events published per run of the timer bean.
- No longer passing around event store data source JNDI name. Using default name instead
- Simplify datasource usage and setup
- Moved Subscription domain, parsing classes and builders to Framework
- Create concurrent catchup process, replays events on different streams concurrently
- Catchup now returns events from published_event rather than from event_log
- Replaced BackwardsCompatibleJsonSchemaValidator with DummyJsonSchemaValidator in Integration Tests
- Catchup, Rebuild and Index no longer call shutter/unshutter when running
- Improved utility classes for getting Postgres DataSources
- Renamed TestEventInserter to EventStoreDataAccess as an improved test class
- Added a findEventsByStreamId() method to EventStoreDataAccess
- Moved system commands to framework jmx-api
- Updated framework-api to 4.0.1
- Updated framework to 6.0.0
- Updated common-bom to 2.4.0
- Updated utilities to 1.20.1
- Updated test-utils to 1.24.3
- Update utilities to 1.16.4
- Update test-utils to 1.22.0
- Update framework to 5.1.1
- Update framework-domain to 1.1.1
- Update common-bom to 1.29.0
- Update framework-api to 3.2.0
- JNDI configuration variable to enable/disable Event Catchup
- Event Catchup on startup, where all unknown events are retrieved from the EventSource and played
- An event-number to each event to allow for event catchup
- Added a new TimerBean 'PrePublishBean'
- Added a new auto incrementing column event_number to event_log table
- Subscription event interceptor to update the current event number in the subscriptions table
- New util module
- Subscription repository
- Subscription liquibase for subscription table
- Better logging for event catchup
- A new transaction is started for each event during event catchup
- Event catchup only runs with event listener components
- Indexes to 'name' and 'date_created' columns in the event_log table
- New util module
- Better logging for event catchup
- An event-number to each event to allow for event catchup
- Added a new TimerBean 'PrePublishBean'
- Added a new auto incrementing column event_number to event_log table
- Subscription liquibase for subscription table
- Subscription repository
- Subscription event interceptor to update the current event number in the subscriptions table
- Updated publish process to add events into a pre_publish_queue table
- Renamed the sequence_number column in event_stream to position_in_stream
- Renamed the sequence_id column in event_log to position_in_stream
- Tightened up transaction boundaries for event catchup so that each event is run in its own transaction
- Event catchup now delegated to a separate session bean to allow for running in RequestScope
- Current event number is initialised to zero if it does not exist on app startup
- Tightened up transaction boundaries for event catchup so that each event is run in its own transaction
- A new transaction is started for each event during event catchup
- Updated publish process to add events into a pre_publish_queue table
- Renamed the sequence_id column in event_log to position_in_stream
- Renamed the sequence_number column in event_stream to position_in_stream
- Updated framework-api to 3.0.1
- Updated framework to 5.0.4
- Updated framework-domain to 1.0.3
- Added a page size when reading stream of events
- Removed hard coded localhost from test datasource url
- Update framework to 5.0.3
- Update framework to 5.0.2
- Update framework to 5.0.0
- Renamed DatabaseCleaner method to match table name
- Removed the need to have event-source.yaml
- Remove requirement to have a subscription-descriptor.yaml on the classpath
- event-publisher-process to event-store-bom
- New test-utils-event-store module to hold TestEventRepository moved from framework
- Reverted names of event buffer tables to their original names: subscription -> stream_status, event-buffer -> stream buffer
- Moved test-utils-persistence into this project from Framework
- Extracted project from all event store related modules in Microservices Framework 5.0.0-M1: https://github.com/CJSCommonPlatform/microservice_framework