receive: Optimized receive; fixed long term overallocation #3334
Commits on Oct 17, 2020
-
receive: Added benchmark for writes.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
-
Optimized write: AddFast and debug prints.
GOROOT=/home/bwplotka/.gvm/gos/go1.15 #gosetup GOPATH=/home/bwplotka/Repos/thanosgopath #gosetup /home/bwplotka/.gvm/gos/go1.15/bin/go test -c -o /tmp/___BenchmarkHandlerReceiveHTTP_in_github_com_thanos_io_thanos_pkg_receive github.com/thanos-io/thanos/pkg/receive #gosetup /tmp/___BenchmarkHandlerReceiveHTTP_in_github_com_thanos_io_thanos_pkg_receive -test.v -test.bench ^\QBenchmarkHandlerReceiveHTTP\E$ -test.run ^$ goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/receive BenchmarkHandlerReceiveHTTP BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_single_sample BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/OK BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/OK-12 66613 17432 ns/op 6382 B/op 47 allocs/op BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/conflict_errors BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/conflict_errors-12 65179 20922 ns/op 9392 B/op 90 allocs/op BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/OK BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/OK-12 51 22250679 ns/op 14286524 B/op 5118 allocs/op BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/conflict_errors BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/conflict_errors-12 31 47359942 ns/op 45323821 B/op 500126 allocs/op BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/OK BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/OK-12 33147 30462 ns/op 10578 B/op 49 allocs/op BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/conflict_errors BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/conflict_errors-12 53385 24087 ns/op 13606 B/op 92 allocs/op BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/OK BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/OK-12 49 24593572 ns/op 15436524 B/op 5141 allocs/op BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/conflict_errors BenchmarkHandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/conflict_errors-12 25 52976766 ns/op 45329167 B/op 500129 allocs/op BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/OK BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/OK-12 60 27261861 ns/op 32538875 B/op 54 allocs/op BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/conflict_errors BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/conflict_errors-12 78 15315152 ns/op 32540624 B/op 97 allocs/op BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/OK BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/OK-12 33 35595450 ns/op 47171972 B/op 5100 allocs/op BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/conflict_errors BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/conflict_errors-12 21 58917325 ns/op 78629564 B/op 500125 allocs/op PASS Process finished with exit code 0 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
-
Tried DeepCopy but not worth it: ``` benchstat -delta-test=none _dev/diffc.out _dev/diffd.out name old time/op new time/op delta HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/OK-12 18.5µs ± 0% 17.8µs ± 0% -3.89% HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/conflict_errors-12 22.5µs ± 0% 20.9µs ± 0% -7.03% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/OK-12 25.0ms ± 0% 24.3ms ± 0% -2.62% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/conflict_errors-12 54.2ms ± 0% 52.3ms ± 0% -3.35% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/OK-12 33.9µs ± 0% 26.6µs ± 0% -21.46% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/conflict_errors-12 24.3µs ± 0% 24.6µs ± 0% +1.43% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/OK-12 24.4ms ± 0% 24.8ms ± 0% +1.36% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/conflict_errors-12 52.4ms ± 0% 48.8ms ± 0% -7.03% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/OK-12 15.2ms ± 0% 15.0ms ± 0% -1.37% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/conflict_errors-12 15.9ms ± 0% 15.2ms ± 0% -4.08% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/OK-12 38.7ms ± 0% 39.6ms ± 0% +2.28% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/conflict_errors-12 67.7ms ± 0% 64.8ms ± 0% -4.34% name old alloc/op new alloc/op delta HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/OK-12 7.35kB ± 0% 7.66kB ± 0% +4.25% HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/conflict_errors-12 10.4kB ± 0% 10.7kB ± 0% +2.94% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/OK-12 14.9MB ± 0% 15.3MB ± 0% +2.56% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/conflict_errors-12 45.3MB ± 0% 45.3MB ± 0% -0.00% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/OK-12 11.9kB ± 0% 12.2kB ± 0% +2.79% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/conflict_errors-12 14.9kB ± 0% 15.2kB ± 0% +2.18% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/OK-12 15.9MB ± 0% 17.7MB ± 0% +11.38% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/conflict_errors-12 45.3MB ± 0% 45.3MB ± 0% -0.00% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/OK-12 43.0MB ± 0% 43.0MB ± 0% -0.00% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/conflict_errors-12 43.0MB ± 0% 43.0MB ± 0% -0.01% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/OK-12 61.2MB ± 0% 60.8MB ± 0% -0.72% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/conflict_errors-12 89.1MB ± 0% 89.1MB ± 0% -0.00% name old allocs/op new allocs/op delta HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/OK-12 67.0 ± 0% 68.0 ± 0% +1.49% HandlerReceiveHTTP/typical_labels_under_1KB,_single_sample/conflict_errors-12 110 ± 0% 111 ± 0% +0.91% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/OK-12 5.14k ± 0% 5.14k ± 0% +0.02% HandlerReceiveHTTP/typical_labels_under_1KB,_2MB_of_samples/conflict_errors-12 500k ± 0% 500k ± 0% -0.00% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/OK-12 69.0 ± 0% 70.0 ± 0% +1.45% HandlerReceiveHTTP/bigger_labels_over_1KB,_single_sample/conflict_errors-12 112 ± 0% 113 ± 0% +0.89% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/OK-12 5.17k ± 0% 5.18k ± 0% +0.17% HandlerReceiveHTTP/bigger_labels_over_1KB,_2MB_of_samples/conflict_errors-12 500k ± 0% 500k ± 0% +0.00% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/OK-12 57.0 ± 0% 57.0 ± 0% 0.00% HandlerReceiveHTTP/extremely_large_label_value_10MB,_single_sample/conflict_errors-12 100 ± 0% 100 ± 0% 0.00% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/OK-12 5.11k ± 0% 5.11k ± 0% -0.08% HandlerReceiveHTTP/extremely_large_label_value_10MB,_2MB_samples/conflict_errors-12 500k ± 0% 500k ± 0% 0.00% ``` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
-
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
-
Reduced allocs on write append errors.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.