You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, we do not have a fast, low-memory, BlockStore implementation that runs on all platforms. The current implementations have the following limitations:
MemBlockStore is totally ephemeral - it's state doesn't survive node container shutdown. Useful mainly for testing and for use by a standalone JS editor.
FileBlockStore uses acid-state, so it persists its state, but keeps the entire store in memory. It also is apparently somewhat slow? I'm guessing the performance could be improved with tuning of how we use acid-state, but keeping the entire store in memory is not going to work long-term.
LevelDbStore uses LevelDB, which works nicely but is painful to build on windows. We want people to be able to easily run Unison nodes on their windows machines, too, so that is a problem (unless we can figure out a good story for building on windows).
So, this task is to investigate a SQLite-backed implementation of BlockStore. SQLite runs on pretty much everything (Windows phone 8 anyone??) and I'm guessing would be easy to integrate into our build. For implementation strategy, it seems like it would work to represent the BlockStore with one two-column table, with columns (hash, blob), with an index on the hash column. Then one more table for tracking the series - (seriesid, hash), with an index on seriesid.
If there's another good embedded key-value store option on Windows, that would be fine also.
There seem to be a couple sqlite bindings on hackage. Not sure which is best.
For questions, @sfultong is something of an expert on BlockStore (he produced the three existing implementations). I'm also available for general support. @sfultong if you decide you'd like to implement this, feel free to claim it - just assign it to yourself. But this might be a good project for someone else to get involved with as a way to get their feet wet with the project.
One additional requirement - the implementation strategy should be obviously safe against SQL injection attacks. No manually gluing raw strings together to construct queries, obviously. Tests should include some 'Bobby tables' style queries.
The text was updated successfully, but these errors were encountered:
At the moment, we do not have a fast, low-memory,
BlockStore
implementation that runs on all platforms. The current implementations have the following limitations:MemBlockStore
is totally ephemeral - it's state doesn't survive node container shutdown. Useful mainly for testing and for use by a standalone JS editor.FileBlockStore
uses acid-state, so it persists its state, but keeps the entire store in memory. It also is apparently somewhat slow? I'm guessing the performance could be improved with tuning of how we use acid-state, but keeping the entire store in memory is not going to work long-term.LevelDbStore
uses LevelDB, which works nicely but is painful to build on windows. We want people to be able to easily run Unison nodes on their windows machines, too, so that is a problem (unless we can figure out a good story for building on windows).So, this task is to investigate a SQLite-backed implementation of
BlockStore
. SQLite runs on pretty much everything (Windows phone 8 anyone??) and I'm guessing would be easy to integrate into our build. For implementation strategy, it seems like it would work to represent theBlockStore
with one two-column table, with columns (hash, blob), with an index on the hash column. Then one more table for tracking the series - (seriesid, hash), with an index on seriesid.If there's another good embedded key-value store option on Windows, that would be fine also.
There seem to be a couple sqlite bindings on hackage. Not sure which is best.
For questions, @sfultong is something of an expert on
BlockStore
(he produced the three existing implementations). I'm also available for general support. @sfultong if you decide you'd like to implement this, feel free to claim it - just assign it to yourself. But this might be a good project for someone else to get involved with as a way to get their feet wet with the project.One additional requirement - the implementation strategy should be obviously safe against SQL injection attacks. No manually gluing raw strings together to construct queries, obviously. Tests should include some 'Bobby tables' style queries.
The text was updated successfully, but these errors were encountered: