Change Propagation Approach Chinese

Aesop uses the log mining approach of detecting data changes as described by the LinkedIn Databus project. It also uses the infrastructure components of Databus, mostly for serving change events. The concept of Event Producer, Relay, Event Buffer, Bootstrap, Event Consumer and System Change Number(SCN) are quite appealing and used mostly as-is in Aesop.

Aesop实现了基于 NGDATA hbase-sep的Event Producer ，这样就支持Hbase数据库的变更数据监测。对于MySQL，直接是基于Databus中已经存在的producer实现。 Aesop extends support for change detection on HBase data store by implementing an Event Producer based off NGDATA hbase-sep. For MySQL, Aesop builds on the producer implementation available in Databus.

log-mining producer的方式利用了数据库中已有的主从复制的原理。此类producer可以称之为push producer，因为变更是从master 推送到slave的。 The log mining producer implementations leverage master-slave replication support available in databases. Such producers may be called "Push" producers where changes are pushed from the master to slave (the Aesop Event Producer).

如果数据散落在多个数据库的表中(更新操作不是一个单独的事务时)log-mining的方式是存在缺陷的，很难将多次更新关联成单个变更。如果数据分布在多种不同类型数据存储中，比如RDBMS和文档数据库。为了解决此类问题，Aesop中实现了一个 Pull producer，其中使用系统提供的Iterator API来周期性的扫描整个数据存储，检测不同扫描周期之间的变更，这个producer是基于Netflix Zeno实现的。

The log mining approach has limitations if data is distributed across database tables (when updates are not part of a single transaction) where-in it is hard to correlate multiple updates to a single change. It also has limitations when data is distributed across types of data stores - for e.g. between an RDBMS and a Document database. Aesop addresses this problem by implementing a "Pull" producer that uses an application-provided "Iterator" API to periodically scan the entire datastore and detect changes between scan cycles. This implementation is based off Netflix Zeno.

变更迁移同时用到了 "Push" and "Pull" producers:

Change propagation employing both "Push" and "Pull" producers:

Pull Producer                        Streaming Client 1       Slow/Catchup client1
(Zeno based)    \                   /                        /
                 \_____ Relay _____/___ Bootstrap __________/
                 /    (Databus)    \    (Databus)           \
                /                   \                        \
Push Producer                        Streaming Client 2       Slow/Catchup client 2  
(e.g. HBase WAL edits listener,
 e.g. MySQL Replication listener)