Skip to content

Commit

Permalink
GC: Consider size of value while rewriting (#1357)
Browse files Browse the repository at this point in the history
The existing code doesn't consider the size of the value while
calculating the size of the entry batch in rewrite method.
This PR fixes it.

Fixes #1292
  • Loading branch information
Ibrahim Jarif authored Jun 9, 2020
1 parent b2267c2 commit 14386ac
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions value.go
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,11 @@ func (vlog *valueLog) rewrite(f *logFile, tr trace.Trace) error {

ne.Value = append([]byte{}, e.Value...)
es := int64(ne.estimateSize(vlog.opt.ValueThreshold))
// Consider size of value as well while considering the total size
// of the batch. There have been reports of high memory usage in
// rewrite because we don't consider the value size. See #1292.
es += int64(len(e.Value))

// Ensure length and size of wb is within transaction limits.
if int64(len(wb)+1) >= vlog.opt.maxBatchCount ||
size+es >= vlog.opt.maxBatchSize {
Expand Down

0 comments on commit 14386ac

Please sign in to comment.