-
Notifications
You must be signed in to change notification settings - Fork 97
TokuMX Hot Indexing
John Esmet edited this page Aug 1, 2013
·
2 revisions
- Vanilla MongoDB supports background indexing. The client thread builds the index using regular indexed insertions and yields the DBWrite lock perodically. This makes the operation online.
- TokuDB for MySQL supports hot indexing which has a similar user experience to background indexing.
- TokuMX should be able to support background indexing by initiating the index build in a DBWrite lock and performing the actual build in a DBRead lock. Committing the hot index will take the DBWrite lock.
- Break up index builds into three stages:
- Write locked: Prep (validation, setup, mark the ns as containing a hot index)
- Read locked: Build (build the index)
- Write locked: Commit (finalize the index, mark the ns as not containing a hot index)
- Capture system.indexes inserts higher in the stack
- Allows for the above locking scheme.
- Implementation notes:
- _indexBuildInProgress is the hot index marker.
- _indexes.size() is the number of total indexes
- _nIndexes is the number of usable indexes
- _indexes[_nIndexes] is the hot index, if in progress.
- If a hot index aborts, the nsindex must rollback and close the affected ns.
- Initiating a hot index build:
- Cannot already have a hot index in progress for the ns.
- ...plus all the normal rules for creating an index.
- During a hot index build:
- Inserts/updates/deletes should work.
- Cannot dropIndexes/dropDatabase if a hot index is in progress.
- Cannot run optimize on a hot index.
- Cannot force the query optimizer to use a hot index.
- Cannot update by _id if one of the fields is in a hot index key.
- After a hot index build:
- Should be possible to drop the index.
- Should be possible to query the index.