Skip to content

Commit

Permalink
Check benchmarks on CI + disable callgrind benchmarks on non-Linux (#752
Browse files Browse the repository at this point in the history
)

* Check benches on CI + disable callgrind benches on Windows

* Fix cfg syntax
  • Loading branch information
kazimuth authored Jan 25, 2024
1 parent 80d9372 commit 72f1550
Show file tree
Hide file tree
Showing 3 changed files with 639 additions and 615 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ jobs:
- name: Run cargo clippy
run: cargo clippy --all --tests --features odb_rocksdb,odb_sled -- -D warnings

- name: Check benchmarks
run: cd crates/bench && cargo check --benches

wasm_bindings:
name: Build and test wasm bindings
runs-on: spacetimedb-runner
Expand Down
5 changes: 4 additions & 1 deletion crates/bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,14 @@ tracing-subscriber.workspace = true
walkdir.workspace = true
itertools.workspace = true

[target.'cfg(target_os = "linux")'.dependencies]
# only try to build these on linux

# also:
# we've forked iai-callgrind to add custom entrypoint support.
# FIXME(jgilles): revert to depending on the crates version if we ever get that upstreamed.

# iai-callgrind = "0.7.2"
iai-callgrind = { git = "https://github.com/clockworklabs/iai-callgrind.git", branch = "main" }
iai-callgrind-runner = { git = "https://github.com/clockworklabs/iai-callgrind.git", branch = "main" }
iai-callgrind-macros = { git = "https://github.com/clockworklabs/iai-callgrind.git", branch = "main" }

Loading

1 comment on commit 72f1550

@github-actions
Copy link

@github-actions github-actions bot commented on 72f1550 Jan 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark results

Benchmark Report

Legend:

  • load: number of rows pre-loaded into the database
  • count: number of rows touched by the transaction
  • index types:
    • unique: a single index on the id column
    • non_unique: no indexes
    • multi_index: non-unique index on every column
  • schemas:
    • person(id: u32, name: String, age: u64)
    • location(id: u32, x: u64, y: u64)

All throughputs are single-threaded.

Empty transaction

db on disk new latency old latency new throughput old throughput
sqlite 💿 434.2±1.13ns 439.2±2.51ns - -
sqlite 🧠 432.2±1.67ns 432.8±1.77ns - -
stdb_module 💿 19.7±0.89µs 20.7±1.34µs - -
stdb_module 🧠 19.7±0.97µs 20.9±1.32µs - -
stdb_raw 💿 940.1±2.64ns 933.8±2.24ns - -
stdb_raw 🧠 895.0±5.48ns 886.0±5.73ns - -

Single-row insertions

db on disk schema index type load new latency old latency new throughput old throughput
sqlite 💿 location multi_index 0 14.8±1.11µs 14.4±0.15µs 66.0 Ktx/sec 67.6 Ktx/sec
sqlite 💿 location multi_index 1000 15.8±0.11µs 16.0±0.12µs 61.7 Ktx/sec 61.2 Ktx/sec
sqlite 💿 location non_unique 0 7.2±1.54µs 7.1±0.02µs 135.9 Ktx/sec 137.2 Ktx/sec
sqlite 💿 location non_unique 1000 7.0±0.04µs 7.1±0.06µs 139.9 Ktx/sec 138.4 Ktx/sec
sqlite 💿 location unique 0 7.1±0.66µs 7.1±0.04µs 137.5 Ktx/sec 138.4 Ktx/sec
sqlite 💿 location unique 1000 7.0±0.04µs 7.1±0.03µs 139.8 Ktx/sec 138.3 Ktx/sec
sqlite 💿 person multi_index 0 14.5±0.93µs 14.2±0.09µs 67.4 Ktx/sec 68.8 Ktx/sec
sqlite 💿 person multi_index 1000 16.1±0.12µs 16.2±0.12µs 60.6 Ktx/sec 60.2 Ktx/sec
sqlite 💿 person non_unique 0 7.1±0.03µs 7.3±0.36µs 136.8 Ktx/sec 134.1 Ktx/sec
sqlite 💿 person non_unique 1000 7.1±0.04µs 7.3±0.08µs 136.7 Ktx/sec 134.2 Ktx/sec
sqlite 💿 person unique 0 7.2±1.09µs 7.3±0.37µs 135.4 Ktx/sec 134.7 Ktx/sec
sqlite 💿 person unique 1000 7.2±0.06µs 7.3±0.05µs 135.8 Ktx/sec 133.5 Ktx/sec
sqlite 🧠 location multi_index 0 4.0±0.01µs 4.1±0.01µs 243.0 Ktx/sec 237.1 Ktx/sec
sqlite 🧠 location multi_index 1000 5.2±0.04µs 5.3±0.06µs 186.8 Ktx/sec 183.4 Ktx/sec
sqlite 🧠 location non_unique 0 1811.7±5.88ns 1876.1±9.20ns 539.0 Ktx/sec 520.5 Ktx/sec
sqlite 🧠 location non_unique 1000 1851.7±5.82ns 1937.5±16.68ns 527.4 Ktx/sec 504.0 Ktx/sec
sqlite 🧠 location unique 0 1787.4±5.95ns 1846.5±17.93ns 546.3 Ktx/sec 528.9 Ktx/sec
sqlite 🧠 location unique 1000 1908.6±5.88ns 1983.1±12.47ns 511.7 Ktx/sec 492.4 Ktx/sec
sqlite 🧠 person multi_index 0 3.6±0.02µs 3.7±0.02µs 269.2 Ktx/sec 266.3 Ktx/sec
sqlite 🧠 person multi_index 1000 5.4±0.04µs 5.5±0.05µs 179.8 Ktx/sec 177.2 Ktx/sec
sqlite 🧠 person non_unique 0 1896.6±7.65ns 1948.6±7.60ns 514.9 Ktx/sec 501.2 Ktx/sec
sqlite 🧠 person non_unique 1000 1952.3±8.51ns 2.0±0.01µs 500.2 Ktx/sec 477.3 Ktx/sec
sqlite 🧠 person unique 0 1882.0±3.98ns 1924.8±4.85ns 518.9 Ktx/sec 507.4 Ktx/sec
sqlite 🧠 person unique 1000 1994.1±19.84ns 2.1±0.01µs 489.7 Ktx/sec 474.8 Ktx/sec
stdb_module 💿 location multi_index 0 54.9±6.05µs 54.1±4.94µs 17.8 Ktx/sec 18.1 Ktx/sec
stdb_module 💿 location multi_index 1000 139.8±2.10µs 289.0±39.84µs 7.0 Ktx/sec 3.4 Ktx/sec
stdb_module 💿 location non_unique 0 50.4±4.11µs 48.2±5.31µs 19.4 Ktx/sec 20.2 Ktx/sec
stdb_module 💿 location non_unique 1000 267.6±2.69µs 191.7±70.26µs 3.6 Ktx/sec 5.1 Ktx/sec
stdb_module 💿 location unique 0 51.9±4.20µs 49.1±8.57µs 18.8 Ktx/sec 19.9 Ktx/sec
stdb_module 💿 location unique 1000 162.6±3.49µs 281.4±56.94µs 6.0 Ktx/sec 3.5 Ktx/sec
stdb_module 💿 person multi_index 0 72.7±20.60µs 68.4±5.02µs 13.4 Ktx/sec 14.3 Ktx/sec
stdb_module 💿 person multi_index 1000 330.4±5.04µs 405.5±16.62µs 3.0 Ktx/sec 2.4 Ktx/sec
stdb_module 💿 person non_unique 0 56.6±4.34µs 52.2±4.82µs 17.3 Ktx/sec 18.7 Ktx/sec
stdb_module 💿 person non_unique 1000 146.4±12.80µs 385.6±2.76µs 6.7 Ktx/sec 2.5 Ktx/sec
stdb_module 💿 person unique 0 60.2±5.41µs 60.4±4.87µs 16.2 Ktx/sec 16.2 Ktx/sec
stdb_module 💿 person unique 1000 129.5±11.29µs 303.3±137.03µs 7.5 Ktx/sec 3.2 Ktx/sec
stdb_module 🧠 location multi_index 0 41.6±3.90µs 41.9±4.36µs 23.5 Ktx/sec 23.3 Ktx/sec
stdb_module 🧠 location multi_index 1000 119.9±4.28µs 268.3±14.69µs 8.1 Ktx/sec 3.6 Ktx/sec
stdb_module 🧠 location non_unique 0 38.5±2.94µs 36.0±3.54µs 25.4 Ktx/sec 27.1 Ktx/sec
stdb_module 🧠 location non_unique 1000 269.3±3.94µs 133.0±3.93µs 3.6 Ktx/sec 7.3 Ktx/sec
stdb_module 🧠 location unique 0 40.5±3.44µs 38.3±5.08µs 24.1 Ktx/sec 25.5 Ktx/sec
stdb_module 🧠 location unique 1000 283.2±10.28µs 216.4±77.79µs 3.4 Ktx/sec 4.5 Ktx/sec
stdb_module 🧠 person multi_index 0 55.5±6.31µs 52.3±5.05µs 17.6 Ktx/sec 18.7 Ktx/sec
stdb_module 🧠 person multi_index 1000 352.1±19.56µs 383.0±7.77µs 2.8 Ktx/sec 2.6 Ktx/sec
stdb_module 🧠 person non_unique 0 36.9±2.82µs 36.2±3.95µs 26.5 Ktx/sec 27.0 Ktx/sec
stdb_module 🧠 person non_unique 1000 295.2±78.80µs 155.6±49.33µs 3.3 Ktx/sec 6.3 Ktx/sec
stdb_module 🧠 person unique 0 44.5±3.41µs 40.3±4.50µs 21.9 Ktx/sec 24.2 Ktx/sec
stdb_module 🧠 person unique 1000 318.5±141.15µs 147.7±4.33µs 3.1 Ktx/sec 6.6 Ktx/sec
stdb_raw 💿 location multi_index 0 7.2±0.04µs 7.0±0.01µs 136.3 Ktx/sec 139.5 Ktx/sec
stdb_raw 💿 location multi_index 1000 34.3±242.61µs 28.9±189.83µs 28.5 Ktx/sec 33.8 Ktx/sec
stdb_raw 💿 location non_unique 0 4.8±0.01µs 4.8±0.01µs 202.6 Ktx/sec 202.7 Ktx/sec
stdb_raw 💿 location non_unique 1000 6.5±0.14µs 16.9±103.48µs 151.2 Ktx/sec 57.9 Ktx/sec
stdb_raw 💿 location unique 0 6.1±0.03µs 6.0±0.02µs 159.2 Ktx/sec 162.1 Ktx/sec
stdb_raw 💿 location unique 1000 29.0±203.52µs 26.7±182.37µs 33.7 Ktx/sec 36.6 Ktx/sec
stdb_raw 💿 person multi_index 0 11.5±0.08µs 11.4±0.01µs 85.0 Ktx/sec 86.0 Ktx/sec
stdb_raw 💿 person multi_index 1000 14.9±0.17µs 15.0±0.20µs 65.3 Ktx/sec 65.1 Ktx/sec
stdb_raw 💿 person non_unique 0 6.0±0.37µs 5.9±0.01µs 162.6 Ktx/sec 164.8 Ktx/sec
stdb_raw 💿 person non_unique 1000 23.4±155.11µs 8.0±0.18µs 41.7 Ktx/sec 122.4 Ktx/sec
stdb_raw 💿 person unique 0 8.3±0.09µs 8.3±0.07µs 118.0 Ktx/sec 118.1 Ktx/sec
stdb_raw 💿 person unique 1000 11.1±0.32µs 11.2±0.15µs 87.7 Ktx/sec 87.4 Ktx/sec
stdb_raw 🧠 location multi_index 0 4.6±0.02µs 4.5±0.01µs 213.0 Ktx/sec 218.6 Ktx/sec
stdb_raw 🧠 location multi_index 1000 6.3±0.04µs 6.2±0.03µs 153.8 Ktx/sec 157.3 Ktx/sec
stdb_raw 🧠 location non_unique 0 2.3±0.00µs 2.3±0.01µs 416.6 Ktx/sec 422.7 Ktx/sec
stdb_raw 🧠 location non_unique 1000 3.1±0.03µs 3.1±0.01µs 320.2 Ktx/sec 317.8 Ktx/sec
stdb_raw 🧠 location unique 0 3.6±0.01µs 3.5±0.01µs 272.1 Ktx/sec 277.2 Ktx/sec
stdb_raw 🧠 location unique 1000 5.0±0.03µs 5.0±0.05µs 196.3 Ktx/sec 196.7 Ktx/sec
stdb_raw 🧠 person multi_index 0 8.3±0.01µs 8.1±0.01µs 118.1 Ktx/sec 120.2 Ktx/sec
stdb_raw 🧠 person multi_index 1000 10.5±0.06µs 10.4±0.08µs 93.4 Ktx/sec 94.0 Ktx/sec
stdb_raw 🧠 person non_unique 0 3.0±0.01µs 2.9±0.01µs 330.1 Ktx/sec 337.1 Ktx/sec
stdb_raw 🧠 person non_unique 1000 3.7±0.02µs 3.8±0.02µs 260.8 Ktx/sec 258.8 Ktx/sec
stdb_raw 🧠 person unique 0 5.2±0.02µs 5.1±0.01µs 187.3 Ktx/sec 190.2 Ktx/sec
stdb_raw 🧠 person unique 1000 6.7±0.04µs 6.7±0.05µs 145.0 Ktx/sec 146.3 Ktx/sec

Multi-row insertions

db on disk schema index type load count new latency old latency new throughput old throughput
sqlite 💿 location multi_index 0 100 128.3±3.04µs 130.2±2.54µs 7.6 Ktx/sec 7.5 Ktx/sec
sqlite 💿 location multi_index 1000 100 203.7±26.60µs 204.8±23.77µs 4.8 Ktx/sec 4.8 Ktx/sec
sqlite 💿 location non_unique 0 100 49.4±5.08µs 49.6±1.77µs 19.8 Ktx/sec 19.7 Ktx/sec
sqlite 💿 location non_unique 1000 100 51.2±0.31µs 54.4±16.04µs 19.1 Ktx/sec 17.9 Ktx/sec
sqlite 💿 location unique 0 100 49.3±0.47µs 51.4±1.71µs 19.8 Ktx/sec 19.0 Ktx/sec
sqlite 💿 location unique 1000 100 54.3±0.31µs 56.4±0.39µs 18.0 Ktx/sec 17.3 Ktx/sec
sqlite 💿 person multi_index 0 100 114.8±3.93µs 117.0±3.45µs 8.5 Ktx/sec 8.3 Ktx/sec
sqlite 💿 person multi_index 1000 100 230.4±1.75µs 228.2±0.58µs 4.2 Ktx/sec 4.3 Ktx/sec
sqlite 💿 person non_unique 0 100 46.4±1.68µs 47.5±2.35µs 21.0 Ktx/sec 20.6 Ktx/sec
sqlite 💿 person non_unique 1000 100 58.1±0.30µs 58.9±0.29µs 16.8 Ktx/sec 16.6 Ktx/sec
sqlite 💿 person unique 0 100 47.7±1.69µs 49.7±2.02µs 20.5 Ktx/sec 19.7 Ktx/sec
sqlite 💿 person unique 1000 100 55.2±15.79µs 55.3±0.49µs 17.7 Ktx/sec 17.7 Ktx/sec
sqlite 🧠 location multi_index 0 100 115.6±0.41µs 117.7±0.62µs 8.4 Ktx/sec 8.3 Ktx/sec
sqlite 🧠 location multi_index 1000 100 167.0±0.61µs 169.5±0.35µs 5.8 Ktx/sec 5.8 Ktx/sec
sqlite 🧠 location non_unique 0 100 40.7±0.45µs 43.9±0.74µs 24.0 Ktx/sec 22.2 Ktx/sec
sqlite 🧠 location non_unique 1000 100 42.5±0.41µs 44.4±0.39µs 23.0 Ktx/sec 22.0 Ktx/sec
sqlite 🧠 location unique 0 100 42.5±0.45µs 44.0±0.59µs 23.0 Ktx/sec 22.2 Ktx/sec
sqlite 🧠 location unique 1000 100 46.2±0.43µs 46.9±0.29µs 21.1 Ktx/sec 20.8 Ktx/sec
sqlite 🧠 person multi_index 0 100 103.1±0.44µs 103.8±0.55µs 9.5 Ktx/sec 9.4 Ktx/sec
sqlite 🧠 person multi_index 1000 100 186.9±0.35µs 188.2±0.51µs 5.2 Ktx/sec 5.2 Ktx/sec
sqlite 🧠 person non_unique 0 100 40.3±0.34µs 39.4±0.29µs 24.2 Ktx/sec 24.8 Ktx/sec
sqlite 🧠 person non_unique 1000 100 43.5±0.27µs 44.2±0.29µs 22.5 Ktx/sec 22.1 Ktx/sec
sqlite 🧠 person unique 0 100 40.7±0.31µs 42.1±0.23µs 24.0 Ktx/sec 23.2 Ktx/sec
sqlite 🧠 person unique 1000 100 45.5±0.34µs 46.4±0.66µs 21.5 Ktx/sec 21.0 Ktx/sec
stdb_module 💿 location multi_index 0 100 854.8±3.41µs 843.7±66.07µs 1169 tx/sec 1185 tx/sec
stdb_module 💿 location multi_index 1000 100 1069.8±1001.81µs 828.2±140.99µs 934 tx/sec 1207 tx/sec
stdb_module 💿 location non_unique 0 100 469.2±67.36µs 276.3±33.97µs 2.1 Ktx/sec 3.5 Ktx/sec
stdb_module 💿 location non_unique 1000 100 486.2±3.85µs 617.7±3.53µs 2.0 Ktx/sec 1618 tx/sec
stdb_module 💿 location unique 0 100 519.1±64.93µs 615.3±68.97µs 1926 tx/sec 1625 tx/sec
stdb_module 💿 location unique 1000 100 530.8±95.47µs 819.1±94.95µs 1883 tx/sec 1220 tx/sec
stdb_module 💿 person multi_index 0 100 1128.5±115.06µs 1048.1±2.29µs 886 tx/sec 954 tx/sec
stdb_module 💿 person multi_index 1000 100 1150.8±4.40µs 1137.7±11.32µs 868 tx/sec 878 tx/sec
stdb_module 💿 person non_unique 0 100 743.9±4.21µs 742.2±5.57µs 1344 tx/sec 1347 tx/sec
stdb_module 💿 person non_unique 1000 100 867.3±73.20µs 826.6±55.82µs 1153 tx/sec 1209 tx/sec
stdb_module 💿 person unique 0 100 760.8±43.10µs 712.1±11.18µs 1314 tx/sec 1404 tx/sec
stdb_module 💿 person unique 1000 100 978.9±85.26µs 965.2±57.97µs 1021 tx/sec 1036 tx/sec
stdb_module 🧠 location multi_index 0 100 516.5±1.39µs 529.2±7.64µs 1936 tx/sec 1889 tx/sec
stdb_module 🧠 location multi_index 1000 100 667.5±16.07µs 683.0±62.30µs 1498 tx/sec 1464 tx/sec
stdb_module 🧠 location non_unique 0 100 377.5±15.73µs 281.9±2.21µs 2.6 Ktx/sec 3.5 Ktx/sec
stdb_module 🧠 location non_unique 1000 100 528.0±41.78µs 544.5±11.57µs 1893 tx/sec 1836 tx/sec
stdb_module 🧠 location unique 0 100 406.2±4.09µs 454.2±65.97µs 2.4 Ktx/sec 2.2 Ktx/sec
stdb_module 🧠 location unique 1000 100 496.9±19.23µs 686.6±37.86µs 2012 tx/sec 1456 tx/sec
stdb_module 🧠 person multi_index 0 100 867.2±1.60µs 909.4±3.53µs 1153 tx/sec 1099 tx/sec
stdb_module 🧠 person multi_index 1000 100 1168.5±23.52µs 985.9±24.67µs 855 tx/sec 1014 tx/sec
stdb_module 🧠 person non_unique 0 100 404.9±29.73µs 523.8±6.88µs 2.4 Ktx/sec 1909 tx/sec
stdb_module 🧠 person non_unique 1000 100 579.7±2.92µs 599.5±67.61µs 1725 tx/sec 1668 tx/sec
stdb_module 🧠 person unique 0 100 636.4±41.10µs 621.6±33.68µs 1571 tx/sec 1608 tx/sec
stdb_module 🧠 person unique 1000 100 865.7±3.26µs 899.0±40.26µs 1155 tx/sec 1112 tx/sec
stdb_raw 💿 location multi_index 0 100 344.9±1.87µs 334.1±0.45µs 2.8 Ktx/sec 2.9 Ktx/sec
stdb_raw 💿 location multi_index 1000 100 397.2±301.25µs 385.5±273.40µs 2.5 Ktx/sec 2.5 Ktx/sec
stdb_raw 💿 location non_unique 0 100 121.1±5.13µs 120.4±0.24µs 8.1 Ktx/sec 8.1 Ktx/sec
stdb_raw 💿 location non_unique 1000 100 122.8±0.93µs 122.6±1.00µs 8.0 Ktx/sec 8.0 Ktx/sec
stdb_raw 💿 location unique 0 100 241.5±0.32µs 240.6±0.86µs 4.0 Ktx/sec 4.1 Ktx/sec
stdb_raw 💿 location unique 1000 100 273.3±146.46µs 264.0±57.24µs 3.6 Ktx/sec 3.7 Ktx/sec
stdb_raw 💿 person multi_index 0 100 711.9±5.02µs 702.0±2.38µs 1404 tx/sec 1424 tx/sec
stdb_raw 💿 person multi_index 1000 100 787.1±484.49µs 780.0±484.33µs 1270 tx/sec 1282 tx/sec
stdb_raw 💿 person non_unique 0 100 221.5±6.84µs 221.1±0.44µs 4.4 Ktx/sec 4.4 Ktx/sec
stdb_raw 💿 person non_unique 1000 100 256.5±232.96µs 229.5±50.00µs 3.8 Ktx/sec 4.3 Ktx/sec
stdb_raw 💿 person unique 0 100 428.2±0.74µs 425.3±0.64µs 2.3 Ktx/sec 2.3 Ktx/sec
stdb_raw 💿 person unique 1000 100 470.9±266.73µs 452.6±91.02µs 2.1 Ktx/sec 2.2 Ktx/sec
stdb_raw 🧠 location multi_index 0 100 296.6±0.36µs 291.0±0.32µs 3.3 Ktx/sec 3.4 Ktx/sec
stdb_raw 🧠 location multi_index 1000 100 319.2±0.37µs 314.5±0.31µs 3.1 Ktx/sec 3.1 Ktx/sec
stdb_raw 🧠 location non_unique 0 100 75.0±0.12µs 74.9±0.93µs 13.0 Ktx/sec 13.0 Ktx/sec
stdb_raw 🧠 location non_unique 1000 100 76.4±0.24µs 76.2±0.17µs 12.8 Ktx/sec 12.8 Ktx/sec
stdb_raw 🧠 location unique 0 100 195.6±0.26µs 194.5±0.21µs 5.0 Ktx/sec 5.0 Ktx/sec
stdb_raw 🧠 location unique 1000 100 213.0±0.33µs 212.9±0.48µs 4.6 Ktx/sec 4.6 Ktx/sec
stdb_raw 🧠 person multi_index 0 100 611.6±0.48µs 602.8±0.80µs 1635 tx/sec 1659 tx/sec
stdb_raw 🧠 person multi_index 1000 100 639.3±0.76µs 629.2±0.63µs 1564 tx/sec 1589 tx/sec
stdb_raw 🧠 person non_unique 0 100 126.2±0.13µs 126.9±0.14µs 7.7 Ktx/sec 7.7 Ktx/sec
stdb_raw 🧠 person non_unique 1000 100 128.4±0.21µs 129.2±0.33µs 7.6 Ktx/sec 7.6 Ktx/sec
stdb_raw 🧠 person unique 0 100 332.7±1.10µs 330.9±0.25µs 2.9 Ktx/sec 3.0 Ktx/sec
stdb_raw 🧠 person unique 1000 100 348.9±0.36µs 348.0±0.63µs 2.8 Ktx/sec 2.8 Ktx/sec

Full table iterate

db on disk schema index type new latency old latency new throughput old throughput
sqlite 💿 location unique 8.6±0.07µs 8.8±0.12µs 114.1 Ktx/sec 110.8 Ktx/sec
sqlite 💿 person unique 8.9±0.08µs 9.2±0.09µs 110.1 Ktx/sec 106.5 Ktx/sec
sqlite 🧠 location unique 7.2±0.09µs 7.7±0.13µs 134.8 Ktx/sec 126.3 Ktx/sec
sqlite 🧠 person unique 7.6±0.05µs 8.0±0.10µs 127.8 Ktx/sec 122.8 Ktx/sec
stdb_module 💿 location unique 49.8±8.92µs 55.7±6.70µs 19.6 Ktx/sec 17.5 Ktx/sec
stdb_module 💿 person unique 59.0±10.82µs 60.8±12.60µs 16.5 Ktx/sec 16.1 Ktx/sec
stdb_module 🧠 location unique 49.3±9.12µs 54.3±6.94µs 19.8 Ktx/sec 18.0 Ktx/sec
stdb_module 🧠 person unique 59.6±10.82µs 55.2±7.61µs 16.4 Ktx/sec 17.7 Ktx/sec
stdb_raw 💿 location unique 9.0±0.01µs 9.1±0.03µs 108.2 Ktx/sec 107.8 Ktx/sec
stdb_raw 💿 person unique 9.0±0.01µs 9.1±0.02µs 108.2 Ktx/sec 107.8 Ktx/sec
stdb_raw 🧠 location unique 9.0±0.02µs 9.0±0.10µs 108.8 Ktx/sec 108.3 Ktx/sec
stdb_raw 🧠 person unique 9.0±0.01µs 9.0±0.02µs 108.9 Ktx/sec 108.4 Ktx/sec

Find unique key

db on disk key type load new latency old latency new throughput old throughput
sqlite 💿 u32 1000 2.3±0.01µs 2.3±0.01µs 426.7 Ktx/sec 419.1 Ktx/sec
sqlite 🧠 u32 1000 1113.9±4.61ns 1145.1±4.24ns 876.7 Ktx/sec 852.8 Ktx/sec
stdb_module 💿 u32 1000 29.4±1.94µs 30.6±2.54µs 33.3 Ktx/sec 31.9 Ktx/sec
stdb_module 🧠 u32 1000 30.2±1.95µs 29.4±2.44µs 32.4 Ktx/sec 33.2 Ktx/sec
stdb_raw 💿 u32 1000 2.2±0.00µs 2.2±0.01µs 438.9 Ktx/sec 443.5 Ktx/sec
stdb_raw 🧠 u32 1000 2.2±0.00µs 2.1±0.01µs 452.7 Ktx/sec 458.6 Ktx/sec

Filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string indexed 1000 10 5.6±0.02µs 5.7±0.03µs 173.3 Ktx/sec 170.2 Ktx/sec
sqlite 💿 string non_indexed 1000 10 47.1±0.27µs 50.0±0.58µs 20.7 Ktx/sec 19.5 Ktx/sec
sqlite 💿 u64 indexed 1000 10 5.4±0.01µs 5.5±0.02µs 181.5 Ktx/sec 178.6 Ktx/sec
sqlite 💿 u64 non_indexed 1000 10 33.1±0.08µs 33.0±0.07µs 29.5 Ktx/sec 29.6 Ktx/sec
sqlite 🧠 string indexed 1000 10 4.2±0.05µs 4.3±0.02µs 231.3 Ktx/sec 226.1 Ktx/sec
sqlite 🧠 string non_indexed 1000 10 45.4±0.27µs 48.2±0.50µs 21.5 Ktx/sec 20.3 Ktx/sec
sqlite 🧠 u64 indexed 1000 10 4.0±0.02µs 4.1±0.01µs 246.3 Ktx/sec 238.8 Ktx/sec
sqlite 🧠 u64 non_indexed 1000 10 31.4±0.08µs 31.7±0.13µs 31.1 Ktx/sec 30.8 Ktx/sec
stdb_module 💿 string indexed 1000 10 38.7±2.95µs 38.6±3.51µs 25.2 Ktx/sec 25.3 Ktx/sec
stdb_module 💿 string non_indexed 1000 10 157.6±2.71µs 162.2±4.51µs 6.2 Ktx/sec 6.0 Ktx/sec
stdb_module 💿 u64 indexed 1000 10 36.3±3.01µs 34.8±2.95µs 26.9 Ktx/sec 28.1 Ktx/sec
stdb_module 💿 u64 non_indexed 1000 10 127.4±7.57µs 123.5±2.86µs 7.7 Ktx/sec 7.9 Ktx/sec
stdb_module 🧠 string indexed 1000 10 37.6±2.35µs 39.3±2.85µs 26.0 Ktx/sec 24.8 Ktx/sec
stdb_module 🧠 string non_indexed 1000 10 165.5±5.20µs 160.7±6.22µs 5.9 Ktx/sec 6.1 Ktx/sec
stdb_module 🧠 u64 indexed 1000 10 35.1±2.40µs 36.0±2.83µs 27.8 Ktx/sec 27.2 Ktx/sec
stdb_module 🧠 u64 non_indexed 1000 10 124.6±5.85µs 123.0±5.26µs 7.8 Ktx/sec 7.9 Ktx/sec
stdb_raw 💿 string indexed 1000 10 4.8±0.01µs 4.7±0.01µs 204.1 Ktx/sec 205.8 Ktx/sec
stdb_raw 💿 string non_indexed 1000 10 125.8±0.32µs 126.8±0.31µs 7.8 Ktx/sec 7.7 Ktx/sec
stdb_raw 💿 u64 indexed 1000 10 4.8±0.01µs 4.7±0.01µs 204.3 Ktx/sec 208.6 Ktx/sec
stdb_raw 💿 u64 non_indexed 1000 10 111.2±2.22µs 110.0±3.54µs 8.8 Ktx/sec 8.9 Ktx/sec
stdb_raw 🧠 string indexed 1000 10 4.7±0.05µs 4.7±0.01µs 207.3 Ktx/sec 209.0 Ktx/sec
stdb_raw 🧠 string non_indexed 1000 10 125.5±0.26µs 126.1±0.27µs 7.8 Ktx/sec 7.7 Ktx/sec
stdb_raw 🧠 u64 indexed 1000 10 4.7±0.02µs 4.6±0.01µs 207.3 Ktx/sec 212.1 Ktx/sec
stdb_raw 🧠 u64 non_indexed 1000 10 110.3±2.83µs 110.5±2.94µs 8.9 Ktx/sec 8.8 Ktx/sec

Serialize

schema format count new latency old latency new throughput old throughput
location bsatn 100 1620.6±20.11ns 1618.7±48.40ns 58.8 Mtx/sec 58.9 Mtx/sec
location json 100 3.1±0.03µs 3.4±0.07µs 31.3 Mtx/sec 28.3 Mtx/sec
location product_value 100 588.7±0.95ns 576.7±3.54ns 162.0 Mtx/sec 165.4 Mtx/sec
person bsatn 100 2.8±0.00µs 2.8±0.01µs 34.2 Mtx/sec 34.1 Mtx/sec
person json 100 5.0±0.02µs 5.1±0.12µs 19.1 Mtx/sec 18.6 Mtx/sec
person product_value 100 664.3±1.76ns 663.6±1.52ns 143.6 Mtx/sec 143.7 Mtx/sec

Module: invoke with large arguments

arg size new latency old latency new throughput old throughput
64KiB 62.0±9.11µs 69.7±10.42µs - -

Module: print bulk

line count new latency old latency new throughput old throughput
1 29.9±1.65µs 29.6±2.42µs - -
100 208.4±6.01µs 210.9±6.13µs - -
1000 1907.1±397.02µs 1959.1±370.35µs - -

Remaining benchmarks

name new latency old latency new throughput old throughput

Please sign in to comment.