Add general tombstones or more specific indexes #2673
Labels
theme/internal-cleanup
Used to identify tech debt, testing improvements, code refactoring, and non-impactful optimization
type/enhancement
Proposed improvement or new feature
Milestone
In working through the details of #2671 I realized that there's still quite a bit of room for improvement to blocking queries, even with fine-grained watches. In particular, because we don't have tombstones for anything other than KV, we use max full-table indexes for other types of queries. The fine-grained watches mean we don't wake up unless needed, but then the coarse-grained indexes mean that we don't have good feedback if something actually changed.
We should look at a more general tombstone mechanism or a more general index tracking mechanism that lets us give better-scoped indexes in response to queries. It seems like if we managed the max Raft index in a radix tree alongside the memdb indexes we could do the right thing, even without tombstones. When items are deleted we'd need to propagate the max index from the subtree up to the node above. Need some more thinking and vetting on this, but it seems like a good line to investigate.
The text was updated successfully, but these errors were encountered: