store: Added benchmarks for .Series and expanded matchers. #2070
Commits on Feb 3, 2020
-
store: Added benchmarks for .Series and expanded matchers.
## Changes * Added ExpandedPostingsMatchers benchmarks. This function and only this will be affected by index-header changes. * Added store Series test. This can be later expanded for tests for different StoreAPIs not only Store GW. * Added few interfaces to improve testability to Store GW bucket struct. * Fixed time filtering edge case. Now request mint=0, maxt=0 should succeed. ## ExpandedPostings Script for running those: ``` #!/usr/bin/env bash DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" cd ${DIR}/../ RUN="${1}-exppostings" mkdir -p ${DIR}/bench_outs/${RUN} for TEST in BenchmarkBucketIndexReader_ExpandedPostings/binary_header BenchmarkBucketIndexReader_ExpandedPostings/index_json do echo "Running ${TEST} run ${RUN}" go test -bench=${TEST} -run=^$ -benchmem -memprofile ${DIR}/bench_outs/${RUN}/memprofile${TEST//\//-}.out -timeout 2h -benchtime 60s ./pkg/store | tee ${DIR}/bench_outs/${RUN}/bench${TEST//\//-}.out done sed 's/BenchmarkBucketIndexReader_ExpandedPostings\/binary_header/BenchmarkBucketIndexReader_ExpandedPostings/g' ${DIR}/bench_outs/${RUN}/benchBenchmarkBucketIndexReader_ExpandedPostings-binary_header.out > ${DIR}/bench_outs/${RUN}/bench_new.out sed 's/BenchmarkBucketIndexReader_ExpandedPostings\/index_json/BenchmarkBucketIndexReader_ExpandedPostings/g' ${DIR}/bench_outs/${RUN}/benchBenchmarkBucketIndexReader_ExpandedPostings-index_json.out > ${DIR}/bench_outs/${RUN}/bench_old.out benchcmp ${DIR}/bench_outs/${RUN}/bench_old.out ${DIR}/bench_outs/${RUN}/bench_new.out | tee ${DIR}/bench_outs/${RUN}/bench.out ``` Results: ``` benchmark old ns/op new ns/op delta BenchmarkBucketIndexReader_ExpandedPostings/n="1"-12 3842599 4396053 +14.40% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j="foo"-12 32690723 31569625 -3.43% BenchmarkBucketIndexReader_ExpandedPostings/j="foo",n="1"-12 32505999 33343759 +2.58% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j!="foo"-12 200367800 229696038 +14.64% BenchmarkBucketIndexReader_ExpandedPostings/i=~".*"-12 153160181 188532296 +23.09% BenchmarkBucketIndexReader_ExpandedPostings/i=~".+"-12 380726946 463532800 +21.75% BenchmarkBucketIndexReader_ExpandedPostings/i=~""-12 421439451 512133098 +21.52% BenchmarkBucketIndexReader_ExpandedPostings/i!=""-12 308473739 407513630 +32.11% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",j="foo"-12 136999349 163355090 +19.24% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",i!="2",j="foo"-12 161095504 178444900 +10.77% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!=""-12 123643237 154754514 +25.16% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!="",j="foo"-12 141086986 175977956 +24.73% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",j="foo"-12 212058205 244589372 +15.34% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~"1.+",j="foo"-12 33981603 46508363 +36.86% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!="2",j="foo"-12 268266004 323914937 +20.74% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!~"2.*",j="foo"-12 306779915 347659338 +13.33% benchmark old allocs new allocs delta BenchmarkBucketIndexReader_ExpandedPostings/n="1"-12 73 75 +2.74% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j="foo"-12 108 112 +3.70% BenchmarkBucketIndexReader_ExpandedPostings/j="foo",n="1"-12 108 112 +3.70% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j!="foo"-12 142 148 +4.23% BenchmarkBucketIndexReader_ExpandedPostings/i=~".*"-12 96 98 +2.08% BenchmarkBucketIndexReader_ExpandedPostings/i=~".+"-12 100159 300159 +199.68% BenchmarkBucketIndexReader_ExpandedPostings/i=~""-12 100114 300115 +199.77% BenchmarkBucketIndexReader_ExpandedPostings/i!=""-12 100156 300156 +199.69% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",j="foo"-12 149 155 +4.03% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",i!="2",j="foo"-12 181 193 +6.63% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!=""-12 100174 300176 +199.65% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!="",j="foo"-12 100204 300208 +199.60% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",j="foo"-12 100207 300211 +199.59% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~"1.+",j="foo"-12 11285 33511 +196.95% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!="2",j="foo"-12 100240 300247 +199.53% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!~"2.*",j="foo"-12 111349 333577 +199.58% benchmark old bytes new bytes delta BenchmarkBucketIndexReader_ExpandedPostings/n="1"-12 11379240 11379278 +0.00% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j="foo"-12 23527225 23527336 +0.00% BenchmarkBucketIndexReader_ExpandedPostings/j="foo",n="1"-12 23527224 23527339 +0.00% BenchmarkBucketIndexReader_ExpandedPostings/n="1",j!="foo"-12 90635328 90636435 +0.00% BenchmarkBucketIndexReader_ExpandedPostings/i=~".*"-12 285362375 334956762 +17.38% BenchmarkBucketIndexReader_ExpandedPostings/i=~".+"-12 331578558 384372836 +15.92% BenchmarkBucketIndexReader_ExpandedPostings/i=~""-12 182041232 234834884 +29.00% BenchmarkBucketIndexReader_ExpandedPostings/i!=""-12 331576878 384371194 +15.92% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",j="foo"-12 92242749 141837796 +53.77% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".*",i!="2",j="foo"-12 94340164 143955415 +52.59% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!=""-12 126308942 179103370 +41.80% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i!="",j="foo"-12 138456800 191251255 +38.13% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",j="foo"-12 138458444 191253140 +38.13% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~"1.+",j="foo"-12 33852975 83803000 +147.55% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!="2",j="foo"-12 207664951 260478496 +25.43% BenchmarkBucketIndexReader_ExpandedPostings/n="1",i=~".+",i!~"2.*",j="foo"-12 212691177 315435644 +48.31% ``` ## Series Script: ``` #!/usr/bin/env bash DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" cd ${DIR}/../ RUN="${1}-series" mkdir -p ${DIR}/bench_outs/${RUN} for TEST in BenchmarkSeries/binary_header BenchmarkSeries/index_json do echo "Running ${TEST} run ${RUN}" go test -bench=${TEST} -run=^$ -benchmem -memprofile ${DIR}/bench_outs/${RUN}/memprofile${TEST//\//-}.out -timeout 2h -benchtime 30s ./pkg/store | tee ${DIR}/bench_outs/${RUN}/bench${TEST//\//-}.out done sed 's/BenchmarkSeries\/binary_header/BenchmarkSeries/g' ${DIR}/bench_outs/${RUN}/benchBenchmarkSeries-binary_header.out > ${DIR}/bench_outs/${RUN}/bench_new.out sed 's/BenchmarkSeries\/index_json/BenchmarkSeries/g' ${DIR}/bench_outs/${RUN}/benchBenchmarkSeries-index_json.out > ${DIR}/bench_outs/${RUN}/bench_old.out benchcmp ${DIR}/bench_outs/${RUN}/bench_old.out ${DIR}/bench_outs/${RUN}/bench_new.out | tee ${DIR}/bench_outs/${RUN}/bench.out ``` Results: ``` Running BenchmarkSeries/binary_header run 0-series goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/store BenchmarkSeries/binary_header/1of10000000-12 10 3405980704 ns/op 1577961956 B/op 20076939 allocs/op BenchmarkSeries/binary_header/10of10000000-12 10 3338306252 ns/op 1577902473 B/op 20076885 allocs/op BenchmarkSeries/binary_header/100of10000000-12 10 3381093365 ns/op 1577926803 B/op 20077301 allocs/op BenchmarkSeries/binary_header/1000of10000000-12 10 3346731304 ns/op 1578388380 B/op 20081755 allocs/op BenchmarkSeries/binary_header/10000of10000000-12 9 3387770880 ns/op 1587132363 B/op 20127273 allocs/op BenchmarkSeries/binary_header/100000of10000000-12 10 3509686626 ns/op 1671451827 B/op 20580914 allocs/op BenchmarkSeries/binary_header/1000000of10000000-12 8 4379223614 ns/op 2514264246 B/op 25115266 allocs/op BenchmarkSeries/binary_header/10000000of10000000-12 3 12498205945 ns/op 15882149677 B/op 130614810 allocs/op PASS ok github.com/thanos-io/thanos/pkg/store 481.055s Running BenchmarkSeries/index_json run 0-series goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/store BenchmarkSeries/index_json/1of10000000-12 13 2519404503 ns/op 1437956498 B/op 15076920 allocs/op BenchmarkSeries/index_json/10of10000000-12 14 2570713484 ns/op 1437918778 B/op 15076972 allocs/op BenchmarkSeries/index_json/100of10000000-12 14 2462235340 ns/op 1437948738 B/op 15077374 allocs/op BenchmarkSeries/index_json/1000of10000000-12 15 2409482348 ns/op 1438429929 B/op 15081921 allocs/op BenchmarkSeries/index_json/10000of10000000-12 15 2447382076 ns/op 1447084683 B/op 15127167 allocs/op BenchmarkSeries/index_json/100000of10000000-12 14 2452156730 ns/op 1530657506 B/op 15580889 allocs/op BenchmarkSeries/index_json/1000000of10000000-12 10 3234136594 ns/op 2379163824 B/op 20115223 allocs/op BenchmarkSeries/index_json/10000000of10000000-12 signal: killed FAIL github.com/thanos-io/thanos/pkg/store 569.329s FAIL benchmark old ns/op new ns/op delta BenchmarkSeries/1of10000000-12 2519404503 3405980704 +35.19% BenchmarkSeries/10of10000000-12 2570713484 3338306252 +29.86% BenchmarkSeries/100of10000000-12 2462235340 3381093365 +37.32% BenchmarkSeries/1000of10000000-12 2409482348 3346731304 +38.90% BenchmarkSeries/10000of10000000-12 2447382076 3387770880 +38.42% BenchmarkSeries/100000of10000000-12 2452156730 3509686626 +43.13% BenchmarkSeries/1000000of10000000-12 3234136594 4379223614 +35.41% benchmark old allocs new allocs delta BenchmarkSeries/1of10000000-12 15076920 20076939 +33.16% BenchmarkSeries/10of10000000-12 15076972 20076885 +33.16% BenchmarkSeries/100of10000000-12 15077374 20077301 +33.16% BenchmarkSeries/1000of10000000-12 15081921 20081755 +33.15% BenchmarkSeries/10000of10000000-12 15127167 20127273 +33.05% BenchmarkSeries/100000of10000000-12 15580889 20580914 +32.09% BenchmarkSeries/1000000of10000000-12 20115223 25115266 +24.86% benchmark old bytes new bytes delta BenchmarkSeries/1of10000000-12 1437956498 1577961956 +9.74% BenchmarkSeries/10of10000000-12 1437918778 1577902473 +9.74% BenchmarkSeries/100of10000000-12 1437948738 1577926803 +9.73% BenchmarkSeries/1000of10000000-12 1438429929 1578388380 +9.73% BenchmarkSeries/10000of10000000-12 1447084683 1587132363 +9.68% BenchmarkSeries/100000of10000000-12 1530657506 1671451827 +9.20% BenchmarkSeries/1000000of10000000-12 2379163824 2514264246 +5.68% ``` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.