-
Notifications
You must be signed in to change notification settings - Fork 474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use the RocksDB WriteBatchWithIndex to implement the read-your-own-writes in transaction #1287
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't review it carefully, so maybe I made a mistake, but my point is that this changes too much.
Assume that a transaction is execution, it doesn't block all execution threads, so, the things would be trickey:
Storage (which is shared among system) would be mark as txn. My point is that if multiple thread call exec, which would win?When an non-txn get executing when txn is executing, what would it happens?
I suggest that rename it to ctx, and rather than save txn flag to storage, encapsulate it into a per-request context. And rather than transaction, this maybe better called a batch, because maybe no conflict detection here.
Updated:
Seems that exclusive
flag would make exec holds and write lock. But still argue that would a exception cause the txn abort.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other LGTM. Though I still think put txn_write_batch_
in storage in a bit trickey, can we add some description here, for it can only used by exclusive
command?
Thanks for your kind review. Agreed to add more comments to explain why the current implementation is right. I'm also thinking about how to allow multi transactions to improve performance because it's unnecessary to be exclusive command if we can promise the commit all or none. But can retrospect this after this PR and the watch/unwatch feature are ready. |
Just parallel exec can be put into a "Session.Context" rather than Though it's ok, but it needs detail considering. For know, I think txn related is all under exclusive mode, because they only used by "Exec", which is exclusive. |
Yes, the transaction state indeed is a connection state, but it needs some refactoring work to support binding transaction conetext in underlying storage type implementation. We won't do that until this PR and watch/unwatch are supported. |
In general, LGTM. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other LGTM here, but I still think we need more log and comments.
read-your-own-writes in transaction By default, the writes cannot be seen its own writes if the WriteBatch wasn't committed in transaction. To mitigate this issue, RocksDB offers the WriteBatchWithIndex to make the uncommitted writes can be visited. For more information can see: https://rocksdb.org/blog/2015/02/27/write-batch-with-index.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems that incoming changes is that write
with txn_batch_mode_
changed to writeToDb
, and don't rely on txn_batch_mode_
.
Currently I've no problem, but I hope we can moke error response to test this in the future. Maybe you can use sync_point in rocksdb ( I've port one of it: https://github.com/mapleFU/sync_point , but you can use it in rocksdb) to mock partial exec error in testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, a clear implementation.
Thanks all, merging… |
By default, the writes cannot be seen in transactions if the WriteBatch
wasn't committed. To mitigate this issue, RocksDB offers the WriteBatchWithIndex
to make the uncommitted writes visible if needed.
For more information can see: https://rocksdb.org/blog/2015/02/27/write-batch-with-index.html
This PR looks not easy to review at first glance, but it will be easier if review commits one by one:
GetWriteBatch
in storage to wrap the WriteBatch for the write operations, making it possible to group and write at once in the EXEC command