Sync optimizations for changes and proposeChanges #6190
Conversation
snej
force-pushed
the
fix/optimize-changes
branch
from
35e4cd5 to
302b6ae
Compare
|
Note that I fixed the test failure by having RevsDiff treat ErrXattrNotFound the same as NotFound ... I'm not sure that's correct. Should the ErrXattrNotFound be handled differently? |
The approach you've got makes sense for this use case - GetXattr differentiates between ErrNotFound and ErrXattrNotFound for use cases that want to handle them differently. |
|
Changes look good, but we should run integration tests before merging to make sure everything is clean when running against Couchbase Server. There were some fixes on master specifically around integration tests. @snej can you rebase on master and then kick off integration tests? |
When a client pushes a doc and `proposeChanges` sees that the docID doesn't exist in the bucket, the `rev` handler will write the doc to the bucket as new without trying to get an existing copy first. This will usually succeed, saving time. If it fails, it's handled like any other CAS mismatch, by retrying and getting the existing doc. - Added a field BlipSyncContext.pendingInsertions, a set of docIDs. - RevDiff returns a new status `ProposedRev_OK_IsNew`. - When the `proposeChanges` handler gets ProposedRev_OK_IsNew, it treats it like ProposedRev_OK and adds the docID to pendingInsertions. - The `rev` handler then checks if the docID is in pendingInsertions; if so, it passes an empty BucketDoc with cas=0 to PutExistingRev as the `existingDoc`. - WriteUpdateWithXattr now allows a `previous` doc with cas=0, interpreting it as the document not previously existing.
- Added a new DocUnmarshalLevel constant, DocUnmarshalHistory, which unmarshals just the history, current rev and CAS. - Added GetDocSyncDataNoImport, which is like GetDocSyncData but does not check whether on-demand import is needed (this lets it avoid reading the doc body) and takes a DocumentUnmarshalLevel parameter. - RevDiff and CheckProposedRev call this instead. RevDiff uses the level DocUnmarshalHistory, and CheckProposedRev uses DocUnmarshalRev unless it might need the history.
Also added a blipSyncCollectionContext reference to blipHandler, since several methods were looking it up.
- Limit size of pendingInsertions - GetDocSyncDataNoImport now upgrades a doc to xattrs
Also cleaned up GetDocSyncDataNoImport a bit
snej
force-pushed
the
fix/optimize-changes
branch
from
6569eac to
2e313e9
Compare
|
@adamcfraser Unfortunately Jenkins says "snej is missing the Job/Build permission"... |
I've granted you permission on jenkins. |
|
I just kicked off a build, but it looks like Adam already ran the integration tests Friday and they passed. |
CBG-2851
Describe your PR here...
proposeChangessees that the docID doesn't exist in the bucket, therevhandler will write the doc to the bucket as new without trying to get an existing copy first. This will usually succeed, saving time. If it fails, it's handled like any other CAS mismatch, by retrying and getting the existing doc.ProposedRev_OK_IsNew.proposeChangeshandler gets ProposedRev_OK_IsNew, it treats itlike ProposedRev_OK and adds the docID to pendingInsertions.
revhandler then checks if the docID is in pendingInsertions; if so,it passes an empty BucketDoc with cas=0 to PutExistingRev as the
existingDoc.previousdoc with cas=0, interpreting itas the document not previously existing.
just the history, current rev and CAS.
check whether on-demand import is needed (this lets it avoid reading the doc
body) and takes a DocumentUnmarshalLevel parameter.
DocUnmarshalHistory, and CheckProposedRev uses DocUnmarshalRev unless it
might need the history.
Pre-review checklist
fmt.Print,log.Print, ...)base.UD(docID),base.MD(dbName))docs/apiDependencies (if applicable)
Integration Tests
GSI=true,xattrs=truehttps://jenkins.sgwdev.com/job/SyncGateway-Integration/000/