Internal transactionality / ACID compliance

[joepio]

The Commits model is transactional, which means that it either succeeds and applies the entire Commit, or fails and doen't apply anything at all.

However, internally, the indexes are updated atom by atom. This means that if atom_2 fails, atom_1 will already be applied and atom_3 will not be applied. I'm pretty confident that this scenario is unlikely, but hardware can always fail. This will not result in data loss, but it could result in incorrectly built indexes, which means a --rebuild-index is needed.

So ideally, we'd implement the entire Commit as a single Transaction. Luckily for me, sled supports transactions. However, I think that it might lead to quite a big change in the architecture of atomic_lib::Db.

Approach 1: add an apply_commit function to Store, drop the add_atom_to_index public API

Currently, the indexing logic is done on a per_atom basis. Storelike has add_atom_to_index functions. I think maybe this API was a mistake. When should a user of this API have to worry about indexing at the atom level?

I could refactor the logic, remove the add_atom_to_index and remove_atom_from_index functions.
We'll get an apply_commit function that does everything in a single transaction. It creates a sled transaction, which is passed around in the various indexing update logic.

Is there a different approach? I don't know.

[joepio]

Sled transactions seem to work through lambdas.
Right now I have a add_atom_to_index and remove_atom_from_index function, both of which call a bunch of other functions that update the respective indexes.

Ideally, I'd pass a mutable transaction or something to these functions, which I can commit / apply after all of these functions have been called.

Maybe I should create my own Transaction struct which keeps track of inserts and deletions, per tree. Idk.

[joepio]

I'm seeing amazing performance improvements :)

Like.. 20x faster?