Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingestion performance tuning #87

Merged
merged 16 commits into from
Aug 21, 2023
Merged

Ingestion performance tuning #87

merged 16 commits into from
Aug 21, 2023

Conversation

jwilder
Copy link
Collaborator

@jwilder jwilder commented Aug 21, 2023

This series of commit removes the majority of allocations from the write path which reduces GC overhead, CPU and memory utilization by 2x-3x.

The optimizations are pretty straight-forward:

  • Avoid excessive type casting for []byte to string
  • Hold onto and re-use buffers vs allocating new ones each time
  • Change some pool sizes to better match concurrency/inflight requests

bytes.ToLower allocates a slice and since this func is in the
hot path for write, it creates a lot of garbage.

This switches to a case-insensitive sort func that does not allocate.
This converts the marshalTS func to not allocate during marhsalling
which doubles the throughput and removes excess memory allocated
that needs to be garbage collected.

benchmark                  old ns/op     new ns/op     delta
BenchmarkMarshalCSV-10     2117          1141          -46.10%

benchmark                  old allocs     new allocs     delta
BenchmarkMarshalCSV-10     20             0              -100.00%

benchmark                  old bytes     new bytes     delta
BenchmarkMarshalCSV-10     1548          0             -100.00%
Bettter conveys that this is a one-time operation.
This sacrifices a little bit of disk for faster compression time
and lower CPU util.
The rings account for a lot of allocations and heap space.  Re-using
them cuts memory usage and GC work for the runtime.
These were too low for larger number of clients and cause new ones
to be allocated frequently.
ingestor/transform/csv.go Show resolved Hide resolved
@jwilder jwilder merged commit a0e15e2 into main Aug 21, 2023
4 checks passed
@jwilder jwilder deleted the jwilder/csv branch August 21, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants