Skip to content

Commit

Permalink
Remove default max concurrent compaction limit
Browse files Browse the repository at this point in the history
PR #9204 introduced a maximum default concurrent compaction limit of 4.
The idea was to reduce IO utilisation on large systems with many cores,
and high write load. Often on these systems, disks were not scaled
appropriately to to the write volume, and while the write path could
keep up, compactions would saturate disks.

In #9225 work was done to reduce IO saturation by limiting the
compaction throughput. To some extent, both #9204 and #9225 work towards
solving the same problem.

We have recently begun to notice larger clusters to suffer from
situations where compactions are not keeping up because they have been
scaled up, but the limit of 4 has stayed in place. While users can
manually override the setting, it seems more user friendly if we remove
the limit by default, and set it manually in cases where compactions are
causing too much IO on large boxes.
  • Loading branch information
e-dard committed Jul 18, 2018
1 parent 55ffeb5 commit 95db829
Showing 1 changed file with 0 additions and 5 deletions.
5 changes: 0 additions & 5 deletions tsdb/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -217,11 +217,6 @@ func (s *Store) loadShards() error {
if lim == 0 {
lim = runtime.GOMAXPROCS(0) / 2 // Default to 50% of cores for compactions

// On systems with more cores, cap at 4 to reduce disk utilization
if lim > 4 {
lim = 4
}

if lim < 1 {
lim = 1
}
Expand Down

0 comments on commit 95db829

Please sign in to comment.