Document replication

For the document [replication](http://en.wikipedia.org/wiki/Replication_(computing\)) feature two types of eXist-db instances are involved: The instance in which a document is added or modified (the producer or 'master' instance) and the instances that shall receive a copy of the document(change): the consumer or 'slave' instances.

The JMS broker receives the document from the producer instance and distributes the data to one or more consumer instances. Theoretically there is no limit on the number of consumer instances, the performance of the producer instance is independent of the number of consumer instances.

During the XML Prague 2013 pre-conference a presentation was given about the replication feature. The slides are available and there is a YouTube recording.

Producer (Master)

The replication extension uses eXist-db triggers to detect when document has been changed. Triggers must defined in collection.xconf files on a per collection basis.

The following XML example shows to connect to an ActiveMQ server on the myserver.local host, the topic name is dynamicTopics/eXistdb:

<collection xmlns="http://exist-db.org/collection-config/1.0">
    <triggers>
        <trigger class="org.exist.jms.replication.publish.ReplicationTrigger">
            <!-- 
                Class name of the initial context provider, default value 
                for ActiveMQ
                see javax.naming.Context#INITIAL_CONTEXT_FACTORY
            -->
            <parameter name="java.naming.factory.initial" 
                value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
            <!-- 
                URL of the message broker, default value for ActiveMQ 
                see javax.naming.Context#PROVIDER_URL
            --> 
            <parameter name="java.naming.provider.url" 
                          value="tcp://myserver.local:61616"/>
            <!-- 
                Lookup connection factory
                see javax.naming.InitialContext#lookup(String) 
            -->
            <parameter name="connection-factory" value="ConnectionFactory"/>
            <!--
                Lookup destination (topic)
                see javax.naming.InitialContext#lookup(String) 
            -->
            <parameter name="destination" value="dynamicTopics/eXistdb-replication-example"/>
        </trigger>
    </triggers>
</collection>

Consumer (Slave)

For receiving documents, a document receiver (JMS listener) must be started at each consumer instance. This can be done via two routes:

1. Define a StartupTrigger in`conf.xml'
2. Call an XQuery function

Both are described in the following paragraphs.

conf.xml

For this route the configuration file conf.xml must be extended with the following XML fragment; when finished the database must be restarted to effectuate the change.

The example connects with the same configuration paramaters to the same server the producer example has been connected with.

<!--
    Start JMS listener for listener of the clustering feature. 
-->
<trigger 
    class="org.exist.jms.replication.subscribe.ReceiverStartupTrigger">
    <!-- 
        Class name of the initial context provider, default value for ActiveMQ
        see javax.naming.Context#INITIAL_CONTEXT_FACTORY
    -->
    <parameter name="java.naming.factory.initial" 
        value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
    
    <!-- 
        URL of the message broker, default value for ActiveMQ
        see javax.naming.Context#PROVIDER_URL
    -->
    <parameter name="java.naming.provider.url" 
        value="tcp://myserver.local:61616"/>
    
    <!-- 
        Lookup connection factory
        see javax.naming.InitialContext#lookup(String) 
    -->
    <parameter name="connection-factory" value="ConnectionFactory"/>
    
    <!--
        Lookup destination (topic)
        see javax.naming.InitialContext#lookup(String) 
    -->
    <parameter name="destination" value="dynamicTopics/eXistdb-replication-example"/>
    
    <!-- 
        Set the name used to identify this subscription
        see JMS javax.jms.TopicSession#createDurableSubscriber(Topic,String) 
    -->
    <parameter name="subscriber.name" value="SubscriptionId"/>

    <!-- 
        Set the client identifier for the connection
        see JMS javax.jms.Connection#setClientID(String) 
    -->
    <parameter name="connection.client-id" value="ClientId"/>
    
</trigger>

Xquery function

Starting the replication listener can be as easy as calling the replication:register($jmsConfiguration) function, with the same parameter set as discussed earlier:

xquery version "3.0";

import module namespace replication="http://exist-db.org/xquery/replication" 
              at "java:org.exist.replication.xquery.ReplicationModule"; 

let $jmsConfiguration := map {
        "java.naming.factory.initial" 
                := "org.apache.activemq.jndi.ActiveMQInitialContextFactory",
        "java.naming.provider.url" := "tcp://myserver.local:61616",
        "connection-factory" := "ConnectionFactory",
        "destination" := "dynamicTopics/eXistdb-replication-example",
        "subscriber.name" := "SubscriptionId",
        "connection.client-id" := "ClientId"
    }
              
return
    replication:register($jmsConfiguration)

To have the function started at database startup, please consult the XQueryStartupTrigger documentation.

Note that this functions is available starting v0.7 of the extension.