Skip to content

Commit

Permalink
Avoid a 32 kB file allocation on every bitBucketFilePutter.Put
Browse files Browse the repository at this point in the history
io.Copy usually allocates a 32kB buffer, and due to the large
number of files processed by tar-split, this shows up in Go profiles
as a very large alloc_space total.

It doesn't seem to actually be a measurable problem in any way,
but we can allocate the buffer only once per tar-split creation,
at no additional cost to existing allocations, so let's do so,
and remove the distraction.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
  • Loading branch information
mtrmac committed Aug 21, 2021
1 parent 05e7c39 commit 8d76363
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion tar/storage/getter.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,12 @@ func NewDiscardFilePutter() FilePutter {
}

type bitBucketFilePutter struct {
buffer [32 * 1024]byte // 32 kB is the buffer size currently used by io.Copy, as of August 2021.
}

func (bbfp *bitBucketFilePutter) Put(name string, r io.Reader) (int64, []byte, error) {
c := crc64.New(CRCTable)
i, err := io.Copy(c, r)
i, err := io.CopyBuffer(c, r, bbfp.buffer[:])
return i, c.Sum(nil), err
}

Expand Down

0 comments on commit 8d76363

Please sign in to comment.