Releases: evanwashere/mitata
Releases · evanwashere/mitata
mitata december holidays update
docs now feature writing good benchmarks section to help people with improving their micro-benchmarks
🚀 New Features
concurrency benchmarks
mitata now offers quick and transparent way to test concurrency of asynchronous functions
bench('sleep(1000) x $concurrency', function* () {
yield async () => await sleep(1000);
}).range('concurrency', 1, 1024);
benchmark avg (min … max) p75 p99 (min … top 1%)
------------------------------------------- -------------------------------
sleep(1000) x 1 1.00 s/iter 1.00 s █ █ █
(1.00 s … 1.00 s) 1.00 s █▁███▁▁▁▁▁█▁▁▁▁█▁▁█▁█
sleep(1000) x 8 1.00 s/iter 1.00 s █ █
(1.00 s … 1.00 s) 1.00 s ▆▁▁▆▁▁▁▁▁▁▁█▁▁▁▆▁▆█▁▆
sleep(1000) x 64 1.00 s/iter 1.00 s ██ █
(1.00 s … 1.00 s) 1.00 s █▁▁▁█▁▁▁▁▁▁██▁███▁▁▁█
sleep(1000) x 512 1.00 s/iter 1.00 s █ ▃ █
(1.00 s … 1.00 s) 1.00 s ▆▁▁▁▁▁▁▁▁▆▆█▁█▁▁▁▁▁▁█
sleep(1000) x 1024 1.00 s/iter 1.00 s ▃ █ ▃
(1.00 s … 1.00 s) 1.00 s ▆▁▁▁▁▆▁█▁█▆▆▁▁▁▁▁▁▁▁█mitata early december update
🚀 New Features
do_not_optimize
utility function to prevent jit from optimizing out values in benchmarks
import { bench, do_not_optimize } from 'mitata';
bench(function () {
do_not_optimize(new Array(0));
});hardware counters
bun add @mitata/counters
mitata now supports collecting and displaying hardware counters on macos and linux
------------------------------------------- -------------------------------
Date.now() 27.25 ns/iter 27.75 ns █
(26.32 ns … 32.33 ns) 30.81 ns █▃▂▁▂▅▂▁▂▂▂▂▁▂▂▁▁▁▁▁▁
2.20 ipc ( 0.00% stalls) 99.99% L1 data cache
95.35 cycles 209.98 instructions 25.48% retired LD/ST ( 53.50)computed parameters
benchmarks now can include generated parameters that avoid variables hoisting optimization in loops
bench('a * b', function* () {
yield {
[0]() {
return Math.random();
},
[1]() {
return Math.random();
},
bench(a, b) {
do_not_optimize(a * b);
},
};
});------------------------------------------- -------------------------------
a * b 611.20 ps/iter 600.10 ps █
(559.33 ps … 8.20 ns) 1.43 ns ▂█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
a / b 913.91 ps/iter 895.26 ps █
(844.48 ps … 8.03 ns) 1.87 ns ▁█▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
mitata november update
🛠️ Changes
groups can have names
group(name, () => {
...
});
• math
-------------------------------------------
1 + 1 87.43 ps/iter
(51.03 ps … 9.83 ns)charts/visualizations can be nested and combined
summary(() => {
barplot(() => {
bench(...);
});
barplot(() => { ... });
});
------------------------------------------- -------------------------------
1 + 1 105.54 ps/iter 91.55 ps █ █ !
(61.04 ps … 1.83 µs) 122.07 ps ▁▁▁▁▁▁▁█▁▁█▁▁▁▁▁▁▄▁▁▂
Date.now() 39.61 ns/iter 28.01 ns █
(27.83 ns … 3.19 µs) 153.12 ns █▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
┌ ┐
1 + 1 ┤ 105.54 ps
Date.now() ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 39.61 ns
└ ┘
summary
1 + 1
375.26x faster than Date.now()
------------------------------------------- -------------------------------
2 + 2 117.75 ps/iter 91.55 ps █ ▄ !
(61.04 ps … 3.69 µs) 112.06 ps ▁▁▁▁▁▁▁▁█▁▁▁█▁▁▁▁▁▁▁▁
3 + 3 126.97 ps/iter 91.55 ps █ █ !
(61.04 ps … 4.23 µs) 122.07 ps ▁▁▁▁▁▁▁█▁▁█▁▁▁▁▁▁▄▁▁▂
┌ ┐
2 + 2 ┤ 117.75 ps
3 + 3 ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 126.97 ps
└ ┘
summary
2 + 2
1.08x faster than 3 + 3other notable changes
- support for detecting webcontainers
- benchmark names now have 28 chars of space
- optimized amount of ansi codes outputted in some charts
🚀 New Features
benchmark names can be highlighted
bench(...).highlight('red');
markdown table format
await run({ format: 'markdown' });clk: ~3.17 GHz
cpu: Apple M2 Pro
runtime: bun 1.1.26 (arm64-darwin)
| benchmark | avg | min | p75 | p99 | max |
|---|---|---|---|---|---|
| 1 + 1 | 90.69 ps/iter |
61.04 ps |
91.55 ps |
122.07 ps |
309.74 ns |
| Date.now() | 28.70 ns/iter |
26.70 ns |
28.01 ns |
43.35 ns |
413.05 ns |
option to fine tune garbage collection behavior
// runs gc before each iteration
bench(...).gc('inner');mitata v1 - stable release
mitata
benchmark tooling that loves you ❤️
mitata is a powerful javascript benchmarking library with features like ascii visualizations, automatic garbage collection, universal runtime/engine compatibility, and high-resolution timings.
Quick Start
| javascript | c++ single header |
|---|---|
import { run, bench, boxplot } from 'mitata';
function fibonacci(n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}
bench('fibonacci(40)', () => fibonacci(40));
boxplot(() => {
bench('new Array($size)', function* (state) {
const size = state.get('size');
yield () => Array.from({ length: size });
}).range('size', 1, 1024);
});
await run(); |
#include "src/mitata.hpp"
int fibonacci(int n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}
int main() {
mitata::runner runner;
runner.bench("noop", []() { });
runner.summary([&]() {
runner.bench("empty fn", []() { });
runner.bench("fibonacci", []() { fibonacci(20); });
});
auto stats = runner.run();
} |
breaking changes
simpler run options
await run({
// throw errors immediately instead of handling them quietly
throw: false,
// filter benchmarks with regex
filter: /.*/,
// colored output (respects NO_COLOR)
colors: true,
// output format: quiet, json, or mitata
format: 'mitata',
});groups no longer have names and summary has been moved to dedicated function
import { bench, group, summary } from 'mitata';
group(() => {
bench('...', () => {});
bench('...', () => {});
});
summary(() => {
bench('...', () => {});
bench('...', () => {});
});🚀 New Features
beautiful visualizations right in your terminal
mitata output has been completely revamped to take advantage of its new ascii rendering capabilities while still having zero dependencies.
-------------------------------------- -------------------------------
Bubble Sort 2.05 ms/iter 2.20 ms ▂▇█▄
(1.75 ms … 6.21 ms) 2.61 ms ▂█████▇▄▃▃▄▅█▄▃▃▂▃▂▁▁
Quick Sort 158.22 µs/iter 156.25 µs █
(135.38 µs … 685.29 µs) 545.29 µs ██▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Native Sort 98.09 µs/iter 98.54 µs █▇
(94.21 µs … 136.71 µs) 111.58 µs ▁▄██▆▃▃▃▃▂▂▂▁▂▁▁▁▁▁▁▁
┌ ┐
╷ ┌──┬──┐ ╷
Bubble Sort ├─┤ │ ├──────┤
╵ └──┴──┘ ╵
┬ ╷
Quick Sort │──────┤
┴ ╵
┬
Native Sort │
┴
└ ┘
94.21 µs 1.35 ms 2.61 ms
┌ ┐
1 + 1 ┤■ 83.50 ps
Date.now() ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 27.30 ns
└ ┘
┌ ┐
Array.from($size) ⢠⠊ 2.02 µs
new Array($size) ⢀⠔⠁
⡠⠃
⢀⠎
⡔⠁
⡠⠊
⢀⠜
⡠⠃
⡔⠁
⢀⠎
⡠⠃
⢀⠜
⢠⠊
⡰⠁ ⢀⣀⣀⣀⡠⠤⠤⠤⠒⠒
⣀⣀⣀⠤⠜ ⢀⣀⡠⠤⠒⠒⠉⠉⠁
⣤⣤⣤⣤⣤⣤⣤⣤⣤⣤⣤⣤⣔⣒⣒⣊⣉⠭⠤⠤⠤⠤⠤⠒⠒⠉⠁ 3.21 ns
└ ┘untangling most common benchmarking spaghetti
mitata now allows you to skip all boilerplate code required for running same benchmark with different configurations.
import { bench, summary } from 'mitata';
bench('new URL($x)', () => {
const url = state.get('x');
yield () => new URL(url);
}).args(['https://example.com', 'https://example.com/foo?bar=baz']);
summary(() => {
// names have special $ keys that get replaced with argument value
bench('Array.from($size)', function* (state) {
const size = state.get('size');
yield () => Array.from({ length: size });
}).range('size', 1, 1024);
bench('new Array($size)', function* (state) {
const size = state.get('size');
yield () => new Array(size);
}).range('size', 1, 1024);
// summary
// new Array($len)
// 5.59…8.68x faster than Array.from($len)
});and more...
- automatic garbage collection
- accuracy down to picoseconds
- lite version as c++ single-header file
- clear warnings for optimized out benchmarks
- compatible with v8, spidermonkey, JavaScriptCore, Graal.js, QuickJS, and more