-
Notifications
You must be signed in to change notification settings - Fork 6.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider removing StorageEmbeddedRocksDB. #18655
Comments
We are using rocksdb table engine as a kv store for now since we don't wanna introduce another kv store dependency. If we get this removed, may i ask if there is any other alternative in CH (ReplacingMergeTree etc may not be as convenient as kv store) ? |
@chenziliang Thank you! Now the chance it will be removed is low. |
But we have to fix bugs, otherwise it will make constant noise in our CI system. |
@sundy-li Maybe you want to help fixing the remaining use-after-free error? facebook/rocksdb#7821 |
Sure. BTW I was not aware that rocksdb may introduce lots of bugs to broke ClickHouse. Because I thought there were many newsql database built base on rocksdb (cockroachdb, tidb, yugabytedb), it proves that ClickHouse has better tests systems than other databases . I am not rocksdb expert, but I'm willing to help it work better in ClickHouse. |
Maybe none of them have CI testing with fuzzing and all four sanitizers? |
All bugs has been fixed. |
Not sure if it is correct to write here, but curiosity and this is Open Source :). I have a question to ClickHouse team regarding usage of RocksDB. Guys, have you faced with RocksDB's memory problems ? I mean unlimited memory growth during long period of insertion ? I receive more and more complains during last 3 years in this issue: facebook/rocksdb#4112. |
@toktarev It's not actively used yet and we have no reports about this issue. There are no tests for long period of insertion - maximum are fuzz and stress tests that last only one hour (for every commit). There is still a chance that we will notice something if it exists... |
@alexey-milovidov , I see , thanks. |
Bugs have been found:
facebook/rocksdb#7711
facebook/rocksdb#7821
facebook/rocksdb#7778
Are there use cases that will motivate against removal?
The text was updated successfully, but these errors were encountered: