Pull change event producer

The Pull based change event producer is an alternative to the log-mining approach for sourcing data changes. This producer too however remains true to design principles of Aesop described in Aesop Architecture.

The pull producer makes one big tradeoff though - that of being able to detect changes in near real-time. This constraint is due to the fact that this producer works on the approach of taking periodic snapshots of the entire source data and performing diffs between snapshots to detect changes. The granularity of detected changes is still at attribute-level for individual instances of a type (or) record. Snapshot creation and diff evaluation is done using Netflix Zeno

The Pull change event producer is implemented as two Maven modules comprising:

• Snapshot creator - Available in the runtime-snapshot-serializer module. Snapshot creation is designed as a cron-like batch job implemented on based on Trooper Batch. Provides components to create the Netflix Zeno State Engine and append data to it. Also manages creation of serialized snapshots and deltas. Refer to the Zeno documentation on Transporting Data for understanding snapshots and deltas
• Diff based Change event producer - Available in the diff-producer module.