Fix life time of memtable_write_group
#171
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
yiwu-arbug
reviewed
Apr 28, 2020
db/db_impl/db_impl_write.cc
Outdated
| @@ -215,6 +215,7 @@ Status DBImpl::MultiBatchWriteImpl(const WriteOptions& write_options, | |||
| // and it would not notify the threads in this WriteGroup. So we must make someone in | |||
| // this WriteGroup to complete it and leader thread is easy to be decided. | |||
| if (is_leader_thread) { | |||
| assert(write_thread_.CompleteParallelMemTableWriter(&writer)); | |||
There was a problem hiding this comment.
From what I remember assert will be optimize out in release build. Do something like the following:
bool ok __attribute__((__unused__)) = write_thread_.CompleteParallelMemTableWriter(&writer);
assert(ok);
There was a problem hiding this comment.
I'm confused. Without calling CompleteParallelMemTableWriter before, how do the leader make sure all other parallel writers in the group has finished at this point?
There was a problem hiding this comment.
Because leader_thread will be blocked in the loop until all followers have finished their tasks.
There was a problem hiding this comment.
So in release build, we do not need to run CompleteParallelMemTableWriter.
There was a problem hiding this comment.
I prefer calling CompleteParallelMemTableWriter anyway, and return error if the call says the leader is not the last writer to exit from the group. That's the contract with WriteThread.
|
Please describe in the summary what's the issue without the change. Thanks. |
Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
Little-Wallace
added a commit
that referenced
this pull request
Apr 30, 2020
tabokie
pushed a commit
to tabokie/rocksdb
that referenced
this pull request
May 12, 2022
* fix life time of memtable_write_group Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie
added a commit
that referenced
this pull request
May 12, 2022
* add multibatch write into memtable (#131) Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Support write multiple WriteBatch into RocksDB together. These WriteBatch will be assigned sequence number in order and pushed into queue. If a thread is waiting for some state, it could steal some job from work queue. Signed-off-by: tabokie <xy.tao@outlook.com> * Improve Multi Batch Write (#154) * perform like normal pipelined write Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * pass enable_multi_thread_write to BuildDBOptions (#170) Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * Fix life time of `memtable_write_group` (#171) * fix life time of memtable_write_group Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * Commit pipeline when write a WriteBatch for linearizability (#267) * support commit pipeline Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * format Signed-off-by: tabokie <xy.tao@outlook.com> * remove useless code Signed-off-by: tabokie <xy.tao@outlook.com> Co-authored-by: Wallace <bupt2013211450@gmail.com>
tabokie
added a commit
that referenced
this pull request
May 25, 2022
* add multibatch write into memtable (#131) Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Support write multiple WriteBatch into RocksDB together. These WriteBatch will be assigned sequence number in order and pushed into queue. If a thread is waiting for some state, it could steal some job from work queue. Signed-off-by: tabokie <xy.tao@outlook.com> * Improve Multi Batch Write (#154) * perform like normal pipelined write Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * pass enable_multi_thread_write to BuildDBOptions (#170) Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * Fix life time of `memtable_write_group` (#171) * fix life time of memtable_write_group Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * Commit pipeline when write a WriteBatch for linearizability (#267) * support commit pipeline Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Signed-off-by: tabokie <xy.tao@outlook.com> * format Signed-off-by: tabokie <xy.tao@outlook.com> * remove useless code Signed-off-by: tabokie <xy.tao@outlook.com> Co-authored-by: Wallace <bupt2013211450@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I created
memtable_write_groupin a wrong position, so it was dropped before this thread began runningSTATE_PARALLEL_MEMTABLE_WRITER. If there are more than one thread trying to insert keys toRocksDBand every of them have more than oneWriteBatch, the process will crash because of accessing an invalid memory address.