Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION]when will rocksdb.write_buffer_size be changed to 16 #286

Closed
stutiredboy opened this issue Jun 2, 2021 · 4 comments · Fixed by #289
Closed

[QUESTION]when will rocksdb.write_buffer_size be changed to 16 #286

stutiredboy opened this issue Jun 2, 2021 · 4 comments · Fixed by #289

Comments

@stutiredboy
Copy link

stutiredboy commented Jun 2, 2021

image

Version: 2.0.1 @5f41145a

I set target_file_size_base and rocksdb.write_buffer_size to 128 in kvrocks.conf then use config get to double check the value was set to 128.

But, some minutes later, the value was changed to 16, log in kvrocks.INFO

I0602 11:43:43.800396 11653 server.cc:1022] [server] Resize rocksdb.target_file_size_base from 128 to 16, average_kv_size: 15, total_size: 104606264039, total_keys: 6620970652, result: ok
I0602 11:43:45.627482 11653 server.cc:1038] [server] Resize rocksdb.write_buffer_size from 16 to 16, average_kv_size: 15, total_size: 104606264039, total_keys: 6620970652, result: ok

We can find that rocksdb.target_file_size_base was changed to 16, then rocksdb.write_buffer_size was changed to the same as rocksdb.target_file_size_base.
And the most interesting is from 16 to 16 but not from 128 to 16, I'm quite sure the value for rocksdb.write_buffer_size in kvrocks.conf is 128 before kvrocks started.

Questions:

  1. when will this happen or this is a bug?
  2. what also can I do for your tracing
@git-hulk
Copy link
Member

git-hulk commented Jun 2, 2021

hi @stutiredboy Thanks for your feedback, kvrocks would auto-adjust the write_buffer_size by calculating the key-value size, see the related issue: #118

As the second issue(the write buffer size from 16 to 16), we'll investigate and feedback here.

cc @karelrooted

@ShooterIT
Copy link
Member

I think we maybe should provide one config to enable auto-adjust of the write_buffer_size and memtable, instead of always enable, because, users maybe have their own method or optimization strategy such as munual config

@git-hulk
Copy link
Member

git-hulk commented Jun 2, 2021

yeah, also the second issue was a bug that we didn't update the config. I'll submit a new issue for @karelrooted .

@karelrooted
Copy link
Contributor

karelrooted commented Jun 2, 2021

the old write_buffer_size in the log is a typo(use the newly setted value as old value), the auto-resize-block-and-sst config directive is added in PR: #289

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants