performance

References

• Community wiki about performance testing: https://github.com/qaload/qaload.github.io

Performance results analysis

• (!) https://github.com/AndreyAkinshin/perfolizer
• Change Point Detection in Software Performance Testing
• Chrome: Regression Detection for Performance Tests

Methodology

• http://www.brendangregg.com/activebenchmarking.html
• http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html
• http://www.brendangregg.com/blog/2014-04-26/the-noploop-cpu-benchmark.html

Tuning environment

• https://perf.readthedocs.io/en/latest/system.html#pin-cpu
• https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application
• How to Benchmark Code Execution Times on Intel® IA-32 and IA-64 Instruction Set Architectures My journey to stable benchmark, part 1 (system)

Microbenchmarks

• low-overhead timers https://github.com/jdmccalpin/low-overhead-timers
• https://github.com/ligurio/microb/
• https://github.com/leo-yuriev/cicro-benchmarks/blob/master/README.md
• https://github.com/jku-ssw/cicro-benchmarks
• http://www.serpentine.com/criterion/fibber.html#grokularation
• https://github.com/bos/criterion
• https://github.com/Gabriel439/bench
• https://docs.python.org/3/library/timeit.html
• https://github.com/sharkdp/hyperfine
• https://github.com/japaric/criterion.rs
• https://blog.anp.lol/rust/2018/09/29/lolbench/
• https://github.com/brendanashworth/bench
• https://github.com/clibs/bench/
• mhz: Anatomy of a micro-benchmark
• https://github.com/softdevteam/softdevbib/blob/master/softdev.bib

Vis

• http://hdrhistogram.org/
• airspeed https://github.com/airspeed-velocity/asv/
• https://wiki.jenkins.io/display/JENKINS/Plot+Plugin
• https://github.com/undera/perf-target-site
• https://sense.blazemeter.com/
• https://overload.yandex.net/login/?next=/
• A Better Way of Reporting Performance Test Results картинка
• https://gist.github.com/hellerbarde/2843375, viz
• jmh?
• BenchmarkDotNet
• Caliper https://microbenchmarks.appspot.com/, https://github.com/google/caliper/wiki/OnlineResults
• criterion.rs пример
• flamescope, https://medium.com/netflix-techblog/netflix-flamescope-a57ca19d47bb
• flamegraphs http://www.brendangregg.com/flamegraphs.html
• benchdraw, benchcmp, benchstat, benchmany

Форматы

• Go
• Jenkins
• JMeter
• Google Benchmark
• criterion.rs, CSV
• https://sense.blazemeter.com/wiki/help:supported_formats/
• CodeSpeed
• BenchmarkAnything
• Chrome Performance Dashboard Data Format
• Skia
• catapult

Анализ, выявление регрессии

• stat https://github.com/thorduri/ministat
• stat https://github.com/nferraz/st
• stat https://github.com/simonccarter/sta
• stat https://github.com/dpmcmlxxvi/clistats
• stat https://sourceforge.net/projects/average/
• stat https://github.com/tonyfischetti/qstats
• stat http://www.numcommand.com/
• stat http://web.cs.wpi.edu/%7Eclaypool/misc/stats/stats.html
• stat https://www.gnu.org/software/datamash/
• ml https://github.com/blue-yonder/tsfresh
• ml https://github.com/lytics/anomalyzer
• ml https://github.com/yzhao062/Pyod
• ml https://github.com/yui0/catseye/
• ml https://github.com/attractivechaos/kann
• ml https://github.com/attractivechaos/sann
• k-means
  • http://bonsai.hgc.jp/~mdehoon/software/cluster/software.htm#source
  • https://github.com/dcasella/k-means/tree/master/C
  • https://github.com/pramsey/kmeans
  • http://homepages.cae.wisc.edu/~brodskye/mr/kmeans/
  • https://github.com/jamiebullock/k-means/blob/master/README.md
• lolbench: automagically and empirically discovering Rust performance regressions
• criterion.rs анализ

Статьи:

• How not to lie with statistics: the correct way to summarize benchmark results
• Detecting Benchmark Regression (слайды)
• https://www.chromium.org/chromium-os/testing/perf-regression-detection
• https://github.com/yzhao062/anomaly-detection-resources
• Automatic Detection of Performance Deviations in the Load Testing of Large Scale Systems -- Haroon Malik, Hadi Hemmati, Ahmed E. Hassan

Benchmarks for anomaly detection

• http://odds.cs.stonybrook.edu/
• http://www.dbs.ifi.lmu.de/research/outlier-evaluation/