Channel Index - Feed Partitioning

[adamcfraser]

Milestone 1 work for Feed Partitioning.

• Understand cbdatastore
• Nail down requirements, check out cbmirror
• Partition feed based on partition definition in config file
• #969 Handle DCP stream end/reconnection
• Buffer incoming feed to minimize rollback risk -- we need to support rollback. As long as we're rolling back to a point that hasn't been indexed yet, it's safe since clients haven't seen it. Assuming the rollback comes faster than our indexing.

Design notes here: https://github.com/couchbase/sync_gateway/blob/feature/distributed_index/docs/design/channel_index/feed_management.md

[tleyden]

Current status:

• Bulk of coding is done on the https://github.com/couchbase/sync_gateway/tree/feature/distributed_index_issue_922 branch
• Running into Couchbase Server problems on AWS (memcached errors in logs, buckets are yellow, sporadic inability to connect to Couchbase Server)
• Need to try on our bare metal performance cluster to compare behavior.

[tleyden]

Today we got a good "test run" on the bare metal cluster and verified correct partitioning of the dcp feed.

What's remaining:

• Doesn’t work when a bucket password is set
• Rename DCPCBGT -> DCP_SHARD
• Use uuid for bind_http and remove from config
• Check for invalid configurations -- if using a DistributedIndexWriter, then it MUST be configured to use DCPSHARD or else it should throw an error
• Not setting the vbucket when the TapEvent created in SyncGatewayPIndex.DataUpdate/DataDelete
• Start DCP from a different starting point -- load stable sequence from the bucket and use that as a starting point
• Try to test with IndexWriter distributed index
• Clean up code for PR submission (rebase, squash again)