Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify/deabstract module_host_actor and wasmer_module #417

Merged
merged 2 commits into from
Oct 27, 2023

Conversation

coolreader18
Copy link
Collaborator

Description of Changes

API and ABI

  • This is a breaking change to the module ABI
  • This is a breaking change to the module API
  • This is a breaking change to the ClientAPI
  • This is a breaking change to the SDK API

If the API is breaking, please state below what will break

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

@kim
Copy link
Contributor

kim commented Oct 12, 2023

This will conflict with #267

@kulakowski
Copy link
Contributor

Not to jump the gun, but I took a peek and think this is going to be helpful to people clicking around in and reading this code.

@kim
Copy link
Contributor

kim commented Oct 12, 2023

Not saying this is bad, but it will make @mamcx’s life harder to get #267 over the line. Also as a heads up: with the changes in #267 much of the init/update code becomes obsolete (for one because we can no longer update index changes). Please let me know when this becomes ready, I think I’m mostly the one familiar with that code.

@coolreader18
Copy link
Collaborator Author

More than happy to wait on #267 to be merged first.

@coolreader18
Copy link
Collaborator Author

What's the status with #267 re:being split into multiple parts? I can't tell whether the chunks that are relevant to wasmer_module are in master already or not.

@coolreader18
Copy link
Collaborator Author

benchmarks please

@github-actions
Copy link

Sorry, you don't have permission to run benchmarks.

@coolreader18
Copy link
Collaborator Author

lol. well

@coolreader18
Copy link
Collaborator Author

I guess that confirms that works @kurtismullins

@coolreader18
Copy link
Collaborator Author

benchmarks please

@github-actions
Copy link

Sorry, you don't have permission to run benchmarks.

@coolreader18
Copy link
Collaborator Author

benchmarks please

@mamcx
Copy link
Contributor

mamcx commented Oct 26, 2023

What's the status with #267 re:being split into multiple parts? I can't tell whether the chunks that are relevant to wasmer_module are in master already or not.

Is still waiting for approving, and it will take some time because was split in several other PRs...

@kim
Copy link
Contributor

kim commented Oct 26, 2023

Alright, then let’s merge this?

@kim
Copy link
Contributor

kim commented Oct 26, 2023

@coolreader18 Maybe you could add this from #386: https://github.com/clockworklabs/SpacetimeDB/pull/386/files#diff-840af797a96dc965f1b75269b1c343356ecb6a0b4e28b84c611a0db514e9d3faR113-R127 ?

I think it is still better meanwhile to create multi-column indexes than to ignore them.

@coolreader18
Copy link
Collaborator Author

Maybe as a separate PR based on this one 😬 ? unless I missed that when I was rebasing and it's in master already. Just cause it's kinda its own feature rather than a refactor like this.

@coolreader18
Copy link
Collaborator Author

benchmarks please

@github-actions
Copy link

github-actions bot commented Oct 26, 2023

Benchmark results

Benchmark Report

Legend:

  • load: number of rows pre-loaded into the database
  • count: number of rows touched by the transaction
  • index types:
    • unique: a single index on the id column
    • non_unique: no indexes
    • multi_index: non-unique index on every column
  • schemas:
    • person(id: u32, name: String, age: u64)
    • location(id: u32, x: u64, y: u64)

All throughputs are single-threaded.

Empty transaction

db on disk new latency old latency new throughput old throughput
sqlite 💿 - 426.7±1.84ns - -
sqlite 🧠 - 421.3±1.94ns - -
stdb_module 💿 15.9±0.32µs 16.4±0.34µs - -
stdb_module 🧠 16.4±0.60µs 16.6±0.49µs - -
stdb_raw 💿 205.0±0.59ns 193.6±1.32ns - -
stdb_raw 🧠 195.9±0.43ns 191.4±1.98ns - -

Single-row insertions

db on disk schema index type load new latency old latency new throughput old throughput
sqlite 💿 location multi_index 0 - 14.8±0.41µs - 66.1 Ktx/sec
sqlite 💿 location multi_index 1000 - 15.9±0.14µs - 61.5 Ktx/sec
sqlite 💿 location non_unique 0 - 7.2±0.10µs - 135.7 Ktx/sec
sqlite 💿 location non_unique 1000 - 7.0±0.02µs - 139.6 Ktx/sec
sqlite 💿 location unique 0 - 7.1±0.04µs - 137.2 Ktx/sec
sqlite 💿 location unique 1000 - 7.1±0.04µs - 138.2 Ktx/sec
sqlite 💿 person multi_index 0 - 14.4±0.04µs - 67.8 Ktx/sec
sqlite 💿 person multi_index 1000 - 16.2±0.16µs - 60.4 Ktx/sec
sqlite 💿 person non_unique 0 - 7.3±0.41µs - 134.1 Ktx/sec
sqlite 💿 person non_unique 1000 - 7.3±0.04µs - 133.7 Ktx/sec
sqlite 💿 person unique 0 - 7.3±0.46µs - 133.5 Ktx/sec
sqlite 💿 person unique 1000 - 7.3±0.05µs - 133.0 Ktx/sec
sqlite 🧠 location multi_index 0 - 4.0±0.01µs - 241.3 Ktx/sec
sqlite 🧠 location multi_index 1000 - 5.2±0.07µs - 186.3 Ktx/sec
sqlite 🧠 location non_unique 0 - 1873.6±15.20ns - 521.2 Ktx/sec
sqlite 🧠 location non_unique 1000 - 1929.1±28.26ns - 506.2 Ktx/sec
sqlite 🧠 location unique 0 - 1836.9±8.99ns - 531.6 Ktx/sec
sqlite 🧠 location unique 1000 - 1965.6±7.29ns - 496.8 Ktx/sec
sqlite 🧠 person multi_index 0 - 3.7±0.01µs - 265.1 Ktx/sec
sqlite 🧠 person multi_index 1000 - 5.5±0.06µs - 179.2 Ktx/sec
sqlite 🧠 person non_unique 0 - 1935.1±6.97ns - 504.7 Ktx/sec
sqlite 🧠 person non_unique 1000 - 1990.1±18.30ns - 490.7 Ktx/sec
sqlite 🧠 person unique 0 - 1934.1±7.51ns - 504.9 Ktx/sec
sqlite 🧠 person unique 1000 - 2.0±0.01µs - 485.7 Ktx/sec
stdb_module 💿 location multi_index 0 42.0±4.94µs 47.5±4.95µs 23.3 Ktx/sec 20.6 Ktx/sec
stdb_module 💿 location multi_index 1000 128.7±43.24µs 254.9±70.64µs 7.6 Ktx/sec 3.8 Ktx/sec
stdb_module 💿 location non_unique 0 41.6±4.66µs 42.4±6.02µs 23.5 Ktx/sec 23.0 Ktx/sec
stdb_module 💿 location non_unique 1000 259.1±53.22µs 139.5±34.32µs 3.8 Ktx/sec 7.0 Ktx/sec
stdb_module 💿 location unique 0 44.0±3.35µs 42.7±4.47µs 22.2 Ktx/sec 22.9 Ktx/sec
stdb_module 💿 location unique 1000 247.8±3.06µs 168.6±45.70µs 3.9 Ktx/sec 5.8 Ktx/sec
stdb_module 💿 person multi_index 0 57.9±8.07µs 60.9±4.86µs 16.9 Ktx/sec 16.0 Ktx/sec
stdb_module 💿 person multi_index 1000 248.3±97.12µs 322.7±13.45µs 3.9 Ktx/sec 3.0 Ktx/sec
stdb_module 💿 person non_unique 0 38.6±4.26µs 43.1±5.56µs 25.3 Ktx/sec 22.6 Ktx/sec
stdb_module 💿 person non_unique 1000 343.5±8.12µs 209.0±59.35µs 2.8 Ktx/sec 4.7 Ktx/sec
stdb_module 💿 person unique 0 48.4±4.70µs 49.7±4.90µs 20.2 Ktx/sec 19.7 Ktx/sec
stdb_module 💿 person unique 1000 264.1±73.17µs 164.5±2.95µs 3.7 Ktx/sec 5.9 Ktx/sec
stdb_module 🧠 location multi_index 0 30.0±2.66µs 30.4±1.85µs 32.6 Ktx/sec 32.2 Ktx/sec
stdb_module 🧠 location multi_index 1000 111.8±2.87µs 210.8±5.20µs 8.7 Ktx/sec 4.6 Ktx/sec
stdb_module 🧠 location non_unique 0 25.1±1.50µs 26.4±2.43µs 38.9 Ktx/sec 37.0 Ktx/sec
stdb_module 🧠 location non_unique 1000 113.1±3.19µs 116.5±4.10µs 8.6 Ktx/sec 8.4 Ktx/sec
stdb_module 🧠 location unique 0 30.2±2.67µs 29.6±2.19µs 32.4 Ktx/sec 33.0 Ktx/sec
stdb_module 🧠 location unique 1000 113.7±3.68µs 201.9±11.46µs 8.6 Ktx/sec 4.8 Ktx/sec
stdb_module 🧠 person multi_index 0 38.4±4.37µs 39.1±4.59µs 25.4 Ktx/sec 24.9 Ktx/sec
stdb_module 🧠 person multi_index 1000 317.2±7.03µs 235.2±16.62µs 3.1 Ktx/sec 4.2 Ktx/sec
stdb_module 🧠 person non_unique 0 27.4±2.28µs 27.0±1.61µs 35.6 Ktx/sec 36.2 Ktx/sec
stdb_module 🧠 person non_unique 1000 110.8±9.50µs 130.5±8.45µs 8.8 Ktx/sec 7.5 Ktx/sec
stdb_module 🧠 person unique 0 31.6±1.86µs 33.0±3.17µs 30.9 Ktx/sec 29.6 Ktx/sec
stdb_module 🧠 person unique 1000 185.0±3.76µs 155.3±17.34µs 5.3 Ktx/sec 6.3 Ktx/sec
stdb_raw 💿 location multi_index 0 6.5±0.02µs 6.5±0.05µs 149.2 Ktx/sec 149.5 Ktx/sec
stdb_raw 💿 location multi_index 1000 9.2±0.19µs 9.3±0.23µs 106.2 Ktx/sec 105.4 Ktx/sec
stdb_raw 💿 location non_unique 0 4.2±0.01µs 4.1±0.01µs 234.6 Ktx/sec 236.8 Ktx/sec
stdb_raw 💿 location non_unique 1000 5.6±0.13µs 5.5±0.13µs 173.3 Ktx/sec 176.0 Ktx/sec
stdb_raw 💿 location unique 0 5.5±0.06µs 5.5±0.11µs 178.0 Ktx/sec 177.6 Ktx/sec
stdb_raw 💿 location unique 1000 24.4±165.28µs 7.8±0.21µs 40.1 Ktx/sec 124.5 Ktx/sec
stdb_raw 💿 person multi_index 0 10.3±0.09µs 10.2±0.03µs 95.1 Ktx/sec 95.5 Ktx/sec
stdb_raw 💿 person multi_index 1000 41.5±281.45µs 49.1±355.90µs 23.5 Ktx/sec 19.9 Ktx/sec
stdb_raw 💿 person non_unique 0 4.7±0.01µs 4.7±0.01µs 206.5 Ktx/sec 207.5 Ktx/sec
stdb_raw 💿 person non_unique 1000 6.4±0.08µs 6.4±0.09µs 152.3 Ktx/sec 151.8 Ktx/sec
stdb_raw 💿 person unique 0 7.1±0.01µs 7.1±0.02µs 138.4 Ktx/sec 138.5 Ktx/sec
stdb_raw 💿 person unique 1000 30.8±211.55µs 9.7±0.20µs 31.7 Ktx/sec 101.1 Ktx/sec
stdb_raw 🧠 location multi_index 0 3.7±0.01µs 3.7±0.01µs 266.1 Ktx/sec 262.3 Ktx/sec
stdb_raw 🧠 location multi_index 1000 5.2±0.05µs 5.2±0.04µs 187.7 Ktx/sec 186.4 Ktx/sec
stdb_raw 🧠 location non_unique 0 1409.4±5.84ns 1411.3±2.68ns 692.9 Ktx/sec 692.0 Ktx/sec
stdb_raw 🧠 location non_unique 1000 1861.2±13.80ns 1862.8±13.44ns 524.7 Ktx/sec 524.3 Ktx/sec
stdb_raw 🧠 location unique 0 2.7±0.00µs 2.7±0.01µs 367.0 Ktx/sec 361.2 Ktx/sec
stdb_raw 🧠 location unique 1000 3.8±0.03µs 3.8±0.03µs 258.5 Ktx/sec 254.8 Ktx/sec
stdb_raw 🧠 person multi_index 0 7.3±0.05µs 7.3±0.01µs 133.4 Ktx/sec 134.1 Ktx/sec
stdb_raw 🧠 person multi_index 1000 9.4±0.21µs 9.4±0.09µs 104.1 Ktx/sec 104.4 Ktx/sec
stdb_raw 🧠 person non_unique 0 1945.5±20.15ns 1950.7±4.33ns 502.0 Ktx/sec 500.6 Ktx/sec
stdb_raw 🧠 person non_unique 1000 2.7±0.02µs 2.6±0.02µs 365.7 Ktx/sec 380.6 Ktx/sec
stdb_raw 🧠 person unique 0 4.2±0.01µs 4.2±0.01µs 233.1 Ktx/sec 234.0 Ktx/sec
stdb_raw 🧠 person unique 1000 5.6±0.14µs 5.5±0.02µs 173.7 Ktx/sec 176.0 Ktx/sec

Multi-row insertions

db on disk schema index type load count new latency old latency new throughput old throughput
sqlite 💿 location multi_index 0 100 - 130.3±4.03µs - 7.5 Ktx/sec
sqlite 💿 location multi_index 1000 100 - 202.8±2.27µs - 4.8 Ktx/sec
sqlite 💿 location non_unique 0 100 - 50.0±1.49µs - 19.5 Ktx/sec
sqlite 💿 location non_unique 1000 100 - 52.4±0.21µs - 18.6 Ktx/sec
sqlite 💿 location unique 0 100 - 51.1±1.27µs - 19.1 Ktx/sec
sqlite 💿 location unique 1000 100 - 56.7±0.29µs - 17.2 Ktx/sec
sqlite 💿 person multi_index 0 100 - 117.7±2.95µs - 8.3 Ktx/sec
sqlite 💿 person multi_index 1000 100 - 231.4±0.71µs - 4.2 Ktx/sec
sqlite 💿 person non_unique 0 100 - 47.7±0.89µs - 20.5 Ktx/sec
sqlite 💿 person non_unique 1000 100 - 59.7±0.39µs - 16.3 Ktx/sec
sqlite 💿 person unique 0 100 - 50.0±1.29µs - 19.5 Ktx/sec
sqlite 💿 person unique 1000 100 - 56.4±0.37µs - 17.3 Ktx/sec
sqlite 🧠 location multi_index 0 100 - 118.7±0.28µs - 8.2 Ktx/sec
sqlite 🧠 location multi_index 1000 100 - 171.6±0.54µs - 5.7 Ktx/sec
sqlite 🧠 location non_unique 0 100 - 44.0±0.55µs - 22.2 Ktx/sec
sqlite 🧠 location non_unique 1000 100 - 45.9±0.37µs - 21.3 Ktx/sec
sqlite 🧠 location unique 0 100 - 45.1±0.29µs - 21.6 Ktx/sec
sqlite 🧠 location unique 1000 100 - 49.5±0.32µs - 19.7 Ktx/sec
sqlite 🧠 person multi_index 0 100 - 106.8±0.31µs - 9.1 Ktx/sec
sqlite 🧠 person multi_index 1000 100 - 188.5±0.36µs - 5.2 Ktx/sec
sqlite 🧠 person non_unique 0 100 - 41.6±0.30µs - 23.5 Ktx/sec
sqlite 🧠 person non_unique 1000 100 - 45.3±0.40µs - 21.5 Ktx/sec
sqlite 🧠 person unique 0 100 - 44.1±0.47µs - 22.1 Ktx/sec
sqlite 🧠 person unique 1000 100 - 49.3±0.26µs - 19.8 Ktx/sec
stdb_module 💿 location multi_index 0 100 797.8±151.21µs 963.5±44.12µs 1253 tx/sec 1037 tx/sec
stdb_module 💿 location multi_index 1000 100 1051.4±173.14µs 983.1±167.70µs 951 tx/sec 1017 tx/sec
stdb_module 💿 location non_unique 0 100 520.9±46.46µs 374.8±96.93µs 1919 tx/sec 2.6 Ktx/sec
stdb_module 💿 location non_unique 1000 100 637.7±37.50µs 496.9±3.87µs 1568 tx/sec 2012 tx/sec
stdb_module 💿 location unique 0 100 486.4±77.52µs 596.0±62.36µs 2.0 Ktx/sec 1677 tx/sec
stdb_module 💿 location unique 1000 100 811.6±212.59µs 655.3±32.31µs 1232 tx/sec 1526 tx/sec
stdb_module 💿 person multi_index 0 100 978.1±2.04µs 1053.9±3.05µs 1022 tx/sec 948 tx/sec
stdb_module 💿 person multi_index 1000 100 1210.9±39.35µs 1252.2±4.82µs 825 tx/sec 798 tx/sec
stdb_module 💿 person non_unique 0 100 697.9±9.21µs 607.1±69.67µs 1432 tx/sec 1647 tx/sec
stdb_module 💿 person non_unique 1000 100 894.5±96.90µs 570.8±35.26µs 1117 tx/sec 1752 tx/sec
stdb_module 💿 person unique 0 100 707.4±80.09µs 686.5±2.23µs 1413 tx/sec 1456 tx/sec
stdb_module 💿 person unique 1000 100 1084.5±1576.91µs 820.2±131.84µs 922 tx/sec 1219 tx/sec
stdb_module 🧠 location multi_index 0 100 533.6±96.73µs 677.3±4.29µs 1874 tx/sec 1476 tx/sec
stdb_module 🧠 location multi_index 1000 100 639.1±57.18µs 629.5±9.70µs 1564 tx/sec 1588 tx/sec
stdb_module 🧠 location non_unique 0 100 224.2±40.26µs 292.2±25.20µs 4.4 Ktx/sec 3.3 Ktx/sec
stdb_module 🧠 location non_unique 1000 100 327.7±17.46µs 353.3±2.62µs 3.0 Ktx/sec 2.8 Ktx/sec
stdb_module 🧠 location unique 0 100 381.3±79.33µs 431.5±55.89µs 2.6 Ktx/sec 2.3 Ktx/sec
stdb_module 🧠 location unique 1000 100 555.3±9.35µs 548.7±89.46µs 1800 tx/sec 1822 tx/sec
stdb_module 🧠 person multi_index 0 100 874.7±70.53µs 783.6±12.73µs 1143 tx/sec 1276 tx/sec
stdb_module 🧠 person multi_index 1000 100 1100.3±46.15µs 856.6±50.16µs 908 tx/sec 1167 tx/sec
stdb_module 🧠 person non_unique 0 100 362.6±63.82µs 335.2±45.95µs 2.7 Ktx/sec 2.9 Ktx/sec
stdb_module 🧠 person non_unique 1000 100 704.2±14.32µs 481.3±5.01µs 1420 tx/sec 2.0 Ktx/sec
stdb_module 🧠 person unique 0 100 572.7±64.54µs 571.9±12.81µs 1745 tx/sec 1748 tx/sec
stdb_module 🧠 person unique 1000 100 807.1±49.37µs 639.5±2.17µs 1238 tx/sec 1563 tx/sec
stdb_raw 💿 location multi_index 0 100 392.0±46.18µs 382.8±2.64µs 2.5 Ktx/sec 2.6 Ktx/sec
stdb_raw 💿 location multi_index 1000 100 410.8±1.35µs 408.0±1.38µs 2.4 Ktx/sec 2.4 Ktx/sec
stdb_raw 💿 location non_unique 0 100 158.4±0.24µs 155.8±4.77µs 6.2 Ktx/sec 6.3 Ktx/sec
stdb_raw 💿 location non_unique 1000 100 160.6±1.04µs 170.2±124.87µs 6.1 Ktx/sec 5.7 Ktx/sec
stdb_raw 💿 location unique 0 100 283.5±0.35µs 281.2±0.36µs 3.4 Ktx/sec 3.5 Ktx/sec
stdb_raw 💿 location unique 1000 100 315.9±139.35µs 300.8±3.88µs 3.1 Ktx/sec 3.2 Ktx/sec
stdb_raw 💿 person multi_index 0 100 709.4±1.36µs 708.7±0.68µs 1409 tx/sec 1411 tx/sec
stdb_raw 💿 person multi_index 1000 100 738.2±3.23µs 828.0±902.25µs 1354 tx/sec 1207 tx/sec
stdb_raw 💿 person non_unique 0 100 215.0±0.37µs 214.3±0.19µs 4.5 Ktx/sec 4.6 Ktx/sec
stdb_raw 💿 person non_unique 1000 100 218.5±0.68µs 222.6±54.79µs 4.5 Ktx/sec 4.4 Ktx/sec
stdb_raw 💿 person unique 0 100 430.3±51.33µs 425.9±3.06µs 2.3 Ktx/sec 2.3 Ktx/sec
stdb_raw 💿 person unique 1000 100 444.0±0.87µs 443.8±0.87µs 2.2 Ktx/sec 2.2 Ktx/sec
stdb_raw 🧠 location multi_index 0 100 302.0±0.47µs 305.1±0.63µs 3.2 Ktx/sec 3.2 Ktx/sec
stdb_raw 🧠 location multi_index 1000 100 328.0±0.57µs 331.0±0.39µs 3.0 Ktx/sec 3.0 Ktx/sec
stdb_raw 🧠 location non_unique 0 100 75.5±0.46µs 74.6±0.10µs 12.9 Ktx/sec 13.1 Ktx/sec
stdb_raw 🧠 location non_unique 1000 100 76.3±0.21µs 76.0±0.14µs 12.8 Ktx/sec 12.9 Ktx/sec
stdb_raw 🧠 location unique 0 100 198.3±0.29µs 199.0±0.39µs 4.9 Ktx/sec 4.9 Ktx/sec
stdb_raw 🧠 location unique 1000 100 217.0±0.29µs 218.3±0.39µs 4.5 Ktx/sec 4.5 Ktx/sec
stdb_raw 🧠 person multi_index 0 100 621.1±0.68µs 619.8±0.96µs 1609 tx/sec 1613 tx/sec
stdb_raw 🧠 person multi_index 1000 100 649.8±0.83µs 649.7±1.94µs 1538 tx/sec 1539 tx/sec
stdb_raw 🧠 person non_unique 0 100 127.4±0.28µs 127.0±0.18µs 7.7 Ktx/sec 7.7 Ktx/sec
stdb_raw 🧠 person non_unique 1000 100 129.8±0.43µs 129.3±0.30µs 7.5 Ktx/sec 7.6 Ktx/sec
stdb_raw 🧠 person unique 0 100 336.9±0.44µs 337.1±0.34µs 2.9 Ktx/sec 2.9 Ktx/sec
stdb_raw 🧠 person unique 1000 100 354.9±0.40µs 355.9±0.39µs 2.8 Ktx/sec 2.7 Ktx/sec

Full table iterate

db on disk schema index type new latency old latency new throughput old throughput
sqlite 💿 location unique - 9.1±0.10µs - 107.9 Ktx/sec
sqlite 💿 person unique - 9.6±0.09µs - 101.7 Ktx/sec
sqlite 🧠 location unique - 7.8±0.13µs - 125.9 Ktx/sec
sqlite 🧠 person unique - 8.3±0.07µs - 118.0 Ktx/sec
stdb_module 💿 location unique 51.3±5.17µs 49.0±3.83µs 19.0 Ktx/sec 19.9 Ktx/sec
stdb_module 💿 person unique 59.6±10.67µs 51.5±10.51µs 16.4 Ktx/sec 19.0 Ktx/sec
stdb_module 🧠 location unique 51.2±6.23µs 47.5±3.09µs 19.1 Ktx/sec 20.6 Ktx/sec
stdb_module 🧠 person unique 64.1±8.75µs 62.9±7.83µs 15.2 Ktx/sec 15.5 Ktx/sec
stdb_raw 💿 location unique 9.1±0.01µs 9.7±0.23µs 107.4 Ktx/sec 100.8 Ktx/sec
stdb_raw 💿 person unique 9.1±0.02µs 9.6±0.01µs 107.5 Ktx/sec 101.5 Ktx/sec
stdb_raw 🧠 location unique 9.1±0.01µs 9.6±0.05µs 107.6 Ktx/sec 101.8 Ktx/sec
stdb_raw 🧠 person unique 9.1±0.01µs 9.6±0.02µs 107.5 Ktx/sec 101.5 Ktx/sec

Find unique key

db on disk key type load new latency old latency new throughput old throughput
sqlite 💿 u32 1000 - 2.4±0.01µs - 412.1 Ktx/sec
sqlite 🧠 u32 1000 - 1133.5±2.50ns - 861.6 Ktx/sec
stdb_module 💿 u32 1000 19.7±1.31µs 20.2±0.99µs 49.5 Ktx/sec 48.4 Ktx/sec
stdb_module 🧠 u32 1000 19.7±1.04µs 19.9±0.92µs 49.5 Ktx/sec 49.1 Ktx/sec
stdb_raw 💿 u32 1000 915.5±3.96ns 909.1±4.03ns 1066.7 Ktx/sec 1074.2 Ktx/sec
stdb_raw 🧠 u32 1000 904.1±2.24ns 904.8±2.26ns 1080.1 Ktx/sec 1079.3 Ktx/sec

Filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string indexed 1000 10 - 5.8±0.02µs - 169.7 Ktx/sec
sqlite 💿 string non_indexed 1000 10 - 50.1±0.37µs - 19.5 Ktx/sec
sqlite 💿 u64 indexed 1000 10 - 5.5±0.02µs - 176.3 Ktx/sec
sqlite 💿 u64 non_indexed 1000 10 - 32.7±0.03µs - 29.8 Ktx/sec
sqlite 🧠 string indexed 1000 10 - 4.3±0.02µs - 227.4 Ktx/sec
sqlite 🧠 string non_indexed 1000 10 - 49.4±0.59µs - 19.8 Ktx/sec
sqlite 🧠 u64 indexed 1000 10 - 4.1±0.02µs - 236.6 Ktx/sec
sqlite 🧠 u64 non_indexed 1000 10 - 31.5±0.04µs - 31.0 Ktx/sec
stdb_module 💿 string indexed 1000 10 30.4±3.42µs 30.4±2.53µs 32.1 Ktx/sec 32.1 Ktx/sec
stdb_module 💿 string non_indexed 1000 10 166.6±1.08µs 182.6±7.64µs 5.9 Ktx/sec 5.3 Ktx/sec
stdb_module 💿 u64 indexed 1000 10 24.7±1.91µs 25.0±1.83µs 39.6 Ktx/sec 39.1 Ktx/sec
stdb_module 💿 u64 non_indexed 1000 10 150.1±24.52µs 153.3±1.37µs 6.5 Ktx/sec 6.4 Ktx/sec
stdb_module 🧠 string indexed 1000 10 31.7±2.42µs 30.2±2.23µs 30.8 Ktx/sec 32.3 Ktx/sec
stdb_module 🧠 string non_indexed 1000 10 168.9±1.07µs 180.0±3.21µs 5.8 Ktx/sec 5.4 Ktx/sec
stdb_module 🧠 u64 indexed 1000 10 25.8±2.00µs 24.4±1.50µs 37.8 Ktx/sec 40.0 Ktx/sec
stdb_module 🧠 u64 non_indexed 1000 10 146.1±6.59µs 158.5±7.03µs 6.7 Ktx/sec 6.2 Ktx/sec
stdb_raw 💿 string indexed 1000 10 3.4±0.01µs 3.4±0.03µs 291.3 Ktx/sec 286.7 Ktx/sec
stdb_raw 💿 string non_indexed 1000 10 166.2±0.42µs 147.1±0.54µs 5.9 Ktx/sec 6.6 Ktx/sec
stdb_raw 💿 u64 indexed 1000 10 3.2±0.01µs 3.3±0.01µs 303.1 Ktx/sec 292.2 Ktx/sec
stdb_raw 💿 u64 non_indexed 1000 10 129.9±0.92µs 128.0±1.19µs 7.5 Ktx/sec 7.6 Ktx/sec
stdb_raw 🧠 string indexed 1000 10 3.4±0.01µs 3.4±0.01µs 290.1 Ktx/sec 288.0 Ktx/sec
stdb_raw 🧠 string non_indexed 1000 10 166.4±0.48µs 148.5±0.51µs 5.9 Ktx/sec 6.6 Ktx/sec
stdb_raw 🧠 u64 indexed 1000 10 3.2±0.06µs 3.3±0.01µs 303.0 Ktx/sec 294.2 Ktx/sec
stdb_raw 🧠 u64 non_indexed 1000 10 130.4±0.89µs 128.2±0.57µs 7.5 Ktx/sec 7.6 Ktx/sec

Serialize

schema format count new latency old latency new throughput old throughput
location bsatn 100 1851.3±35.49ns 1711.7±35.31ns 51.5 Mtx/sec 55.7 Mtx/sec
location json 100 3.5±0.05µs 3.2±0.02µs 27.3 Mtx/sec 30.2 Mtx/sec
location product_value 100 546.0±0.49ns 845.5±0.55ns 174.7 Mtx/sec 112.8 Mtx/sec
person bsatn 100 2.6±0.01µs 2.9±0.01µs 36.4 Mtx/sec 33.1 Mtx/sec
person json 100 5.2±0.07µs 4.8±0.04µs 18.4 Mtx/sec 19.7 Mtx/sec
person product_value 100 1012.3±0.75ns 1118.6±0.53ns 94.2 Mtx/sec 85.3 Mtx/sec

Module: invoke with large arguments

arg size new latency old latency new throughput old throughput
64KiB 72.6±3.25µs 75.2±4.77µs - -

Module: print bulk

line count new latency old latency new throughput old throughput
1 20.9±1.30µs 20.1±0.81µs - -
100 202.4±1.33µs 199.4±1.40µs - -
1000 1859.1±38.62µs 1818.4±7.22µs - -

Remaining benchmarks

name new latency old latency new throughput old throughput

@coolreader18
Copy link
Collaborator Author

cool it works :))))

Copy link
Contributor

@kulakowski kulakowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I think this is good and ready. It does a bit of unrelated cleanups but I like them all.

I'm not hitting approve just to make sure that we are all squared up on which PR lands before whichever other PR, but I'll hit it as as soon as someone acknowledges that the plan is ready.

@@ -370,11 +369,7 @@ impl RelationalDB {

/// Roll back transaction `tx` if `res` is `Err`, otherwise return it
/// alongside the `Ok` value.
#[tracing::instrument(skip_all)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd keep the instrument bit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only thing to fix before landing, then, I think.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's really unnecessary - it's a utility function that we have no need to track and only serves to clutter up logs/graphs. If you feel strongly about it I can add it back, but I really see no value in it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not that I do or don't see the value in it. It's that it is unrelated to the stated point of the PR.

@@ -292,6 +293,73 @@ pub struct TableDef {
pub(crate) table_access: StAccess,
}

impl TableDef {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this code being here. But I don't think there's a great place to put it when we are in this state of defining a pile of concrete types, and traits (that aren't really used, relationaldb directly makes a Locking or whatever). Curious how you feel about it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think there's better ways to go about this, but ideally that would come with a general reorganizing of all these types.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't love this code being here FWIW

Database(#[from] DBError),
}

pub fn update_database(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't scrutinize this code, am trusting that it's functionally unchanged.

@coolreader18 coolreader18 merged commit 628dac1 into master Oct 27, 2023
5 checks passed
@coolreader18 coolreader18 deleted the noa/deabstract branch October 27, 2023 20:53
@coolreader18
Copy link
Collaborator Author

Merged. @joshua-spacetime if you'd like help fixing #486 to work with this I'd be happy to help

@joshua-spacetime
Copy link
Collaborator

Thanks @coolreader18. The conflicts weren't too bad. I was able to get them all resolved. But feel free to take a look just in case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants