Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libroach: Set bytes_per_sync to smooth out disk usage #20352

Merged
merged 1 commit into from
Dec 4, 2017
Merged

libroach: Set bytes_per_sync to smooth out disk usage #20352

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 8 additions & 12 deletions c-deps/libroach/db.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1537,10 +1537,9 @@ rocksdb::Options DBMakeOptions(DBOptions db_opts) {
// Use the rocksdb options builder to configure the base options
// using our memtable budget.
rocksdb::Options options;
// Increase parallelism for compactions based on the number of
// cpus. This will use 1 high priority thread for flushes and
// num_cpu-1 low priority threads for compactions. Always use at
// least 2 threads, otherwise compactions won't happen.
// Increase parallelism for compactions and flushes based on the
// number of cpus. Always use at least 2 threads, otherwise
// compactions and flushes may fight with each other.
options.IncreaseParallelism(std::max(db_opts.num_cpu, 2));
// Enable subcompactions which will use multiple threads to speed up
// a single compaction. The value of num_cpu/2 has not been tuned.
Expand All @@ -1554,14 +1553,11 @@ rocksdb::Options DBMakeOptions(DBOptions db_opts) {
options.statistics = rocksdb::CreateDBStatistics();
options.max_open_files = db_opts.max_open_files;
options.compaction_pri = rocksdb::kMinOverlappingRatio;
// Periodically sync the WAL to smooth out writes. Not performing
// such syncs can be faster but can cause performance blips when the
// OS decides it needs to flush data.
options.wal_bytes_per_sync = 256 << 10; // 256 KB

// Periodically sync sstables during compaction to smooth out
// writes. Experimentally this has had no effect.
// options.bytes_per_sync = 4 << 20;
// Periodically sync both the WAL and SST writes to smooth out disk
// usage. Not performing such syncs can be faster but can cause
// performance blips when the OS decides it needs to flush data.
options.wal_bytes_per_sync = 512 << 10; // 512 KB
options.bytes_per_sync = 512 << 10; // 512 KB

// The size reads should be performed in for compaction. The
// internets claim this can speed up compactions, though RocksDB
Expand Down