Skip to content

Commit

Permalink
nuke DataKey (#1093)
Browse files Browse the repository at this point in the history
  • Loading branch information
@Centril
Centril authored Apr 16, 2024
1 parent f423838 commit 5e78223
Show file tree
Hide file tree
Showing 3 changed files with 0 additions and 187 deletions.
1 change: 0 additions & 1 deletion crates/lib/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ pub use address::Address;
pub use identity::Identity;
pub use spacetimedb_sats::hash::{self, hash_bytes, Hash};
pub use spacetimedb_sats::relation;
pub use spacetimedb_sats::DataKey;
pub use spacetimedb_sats::{self as sats, bsatn, buffer, de, ser};
pub use type_def::*;
pub use type_value::{AlgebraicValue, ProductValue};
Expand Down
184 changes: 0 additions & 184 deletions crates/sats/src/data_key.rs
View file

This file was deleted.

2 changes: 0 additions & 2 deletions crates/sats/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ pub mod bsatn;
pub mod buffer;
pub mod builtin_type;
pub mod convert;
pub mod data_key;
pub mod db;
pub mod de;
pub mod hash;
Expand All @@ -33,7 +32,6 @@ pub use algebraic_value::{AlgebraicValue, F32, F64};
pub use array_type::ArrayType;
pub use array_value::ArrayValue;
pub use builtin_type::BuiltinType;
pub use data_key::{DataKey, ToDataKey};
pub use map_type::MapType;
pub use map_value::MapValue;
pub use product_type::ProductType;
Expand Down

2 comments on commit 5e78223

@github-actions
Copy link

@github-actions github-actions bot commented on 5e78223 Apr 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 417.6±1.80ns 416.2±1.57ns - -
sqlite 🧠 410.4±1.65ns 408.7±1.48ns - -
stdb_raw 💿 719.3±1.67ns 723.5±1.28ns - -
stdb_raw 🧠 695.8±3.87ns 698.1±1.15ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 512.9±0.85µs 510.3±1.15µs 1949 tx/sec 1959 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 138.3±0.59µs 129.8±0.53µs 7.1 Ktx/sec 7.5 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 420.9±0.92µs 416.7±0.44µs 2.3 Ktx/sec 2.3 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 130.2±9.20µs 121.3±0.85µs 7.5 Ktx/sec 8.1 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 445.8±0.46µs 443.2±0.54µs 2.2 Ktx/sec 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 123.5±0.78µs 117.7±1.47µs 7.9 Ktx/sec 8.3 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 367.2±0.77µs 363.9±0.71µs 2.7 Ktx/sec 2.7 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 109.4±0.96µs 103.0±0.63µs 8.9 Ktx/sec 9.5 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 518.2±13.36µs 623.9±16.40µs 1929 tx/sec 1602 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 504.3±31.58µs 486.3±27.98µs 1983 tx/sec 2.0 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 415.9±10.29µs 411.1±21.28µs 2.3 Ktx/sec 2.4 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 363.6±21.10µs 370.2±9.89µs 2.7 Ktx/sec 2.6 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 337.5±0.18µs 352.0±3.12µs 2.9 Ktx/sec 2.8 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 256.6±0.63µs 259.4±0.26µs 3.8 Ktx/sec 3.8 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 288.3±0.11µs 287.2±0.13µs 3.4 Ktx/sec 3.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 240.1±0.15µs 241.3±0.37µs 4.1 Ktx/sec 4.0 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 21.1±0.22µs 20.6±0.20µs 46.3 Ktx/sec 47.5 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 20.0±0.08µs 18.9±0.12µs 48.7 Ktx/sec 51.6 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 20.0±0.21µs 19.4±0.12µs 48.9 Ktx/sec 50.3 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 18.6±0.09µs 17.9±0.22µs 52.4 Ktx/sec 54.5 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 18.5±0.00µs 17.5±0.00µs 52.9 Ktx/sec 55.8 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 15.6±0.00µs 14.6±0.00µs 62.5 Ktx/sec 66.7 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 18.4±0.00µs 17.5±0.00µs 53.0 Ktx/sec 56.0 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 15.6±0.00µs 14.6±0.00µs 62.5 Ktx/sec 66.8 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 67.1±0.19µs 67.0±0.23µs 14.6 Ktx/sec 14.6 Ktx/sec
sqlite 💿 u64 index 2048 256 64.6±0.17µs 64.4±0.07µs 15.1 Ktx/sec 15.2 Ktx/sec
sqlite 🧠 string index 2048 256 65.2±0.16µs 64.9±0.20µs 15.0 Ktx/sec 15.1 Ktx/sec
sqlite 🧠 u64 index 2048 256 60.9±0.22µs 60.3±0.08µs 16.0 Ktx/sec 16.2 Ktx/sec
stdb_raw 💿 string index 2048 256 5.1±0.00µs 5.2±0.00µs 189.9 Ktx/sec 189.3 Ktx/sec
stdb_raw 💿 u64 index 2048 256 5.0±0.00µs 5.1±0.00µs 195.2 Ktx/sec 192.5 Ktx/sec
stdb_raw 🧠 string index 2048 256 5.1±0.00µs 5.1±0.00µs 190.8 Ktx/sec 192.7 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 5.0±0.00µs 5.0±0.00µs 196.2 Ktx/sec 193.8 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.3±0.01µs 3.3±0.00µs 28.9 Mtx/sec 28.7 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.3±0.01µs 3.3±0.00µs 28.8 Mtx/sec 28.8 Mtx/sec
u32_u64_str bsatn 100 2.4±0.09µs 2.4±0.00µs 39.6 Mtx/sec 39.7 Mtx/sec
u32_u64_str json 100 5.0±0.03µs 5.3±0.05µs 19.1 Mtx/sec 17.9 Mtx/sec
u32_u64_str product_value 100 1014.0±0.64ns 1014.9±0.43ns 94.1 Mtx/sec 94.0 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1373.4±7.81ns 1395.3±0.84ns 69.4 Mtx/sec 68.3 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.6±0.00µs 2.6±0.00µs 36.9 Mtx/sec 36.9 Mtx/sec
u32_u64_u64 bsatn 100 1662.1±42.22ns 1644.0±41.35ns 57.4 Mtx/sec 58.0 Mtx/sec
u32_u64_u64 json 100 3.2±0.03µs 3.4±0.03µs 29.5 Mtx/sec 27.8 Mtx/sec
u32_u64_u64 product_value 100 1011.8±0.46ns 1011.6±0.88ns 94.3 Mtx/sec 94.3 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 1111.1±3.30ns 1177.4±0.76ns 85.8 Mtx/sec 81.0 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.6±0.03µs 2.6±0.00µs 37.0 Mtx/sec 36.9 Mtx/sec
u64_u64_u32 bsatn 100 1667.0±35.73ns 1648.8±34.08ns 57.2 Mtx/sec 57.8 Mtx/sec
u64_u64_u32 json 100 3.3±0.14µs 3.5±0.05µs 29.1 Mtx/sec 27.2 Mtx/sec
u64_u64_u32 product_value 100 1011.4±2.54ns 1012.2±0.55ns 94.3 Mtx/sec 94.2 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 79.0±8.59µs 84.2±4.56µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 34.6±4.44µs 44.8±2.71µs - -
100 349.3±3.09µs 354.6±48.40µs - -
1000 2.8±0.45ms 2.0±0.13ms - -

remaining

name new latency old latency new throughput old throughput
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 46.9±0.16µs 46.3±0.31µs 20.8 Ktx/sec 21.1 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 40.7±0.08µs 41.9±0.26µs 24.0 Ktx/sec 23.3 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 39.3±0.09µs 39.2±0.24µs 24.8 Ktx/sec 24.9 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 35.1±0.09µs 35.6±0.20µs 27.9 Ktx/sec 27.5 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1329.2±4.35µs 1333.9±14.40µs 752 tx/sec 749 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 1085.7±14.98µs 1075.4±4.99µs 921 tx/sec 929 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 676.6±21.67µs 680.4±15.11µs 1477 tx/sec 1469 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 539.4±8.36µs 530.8±19.73µs 1854 tx/sec 1883 tx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 421.9±1.21µs 421.0±0.60µs 2.3 Ktx/sec 2.3 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 392.0±0.44µs 393.5±0.26µs 2.5 Ktx/sec 2.5 Ktx/sec

@github-actions
Copy link

@github-actions github-actions bot commented on 5e78223 Apr 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6480 6480 0.00% 7524 7516 0.11%
sqlite 5573 5558 0.27% 6031 5988 0.72%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 123027 123027 0.00% 123715 123631 0.07%
stdb_raw u32_u64_str no_index 64 128 1 u64 80699 80699 0.00% 81271 81175 0.12%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24476 24476 0.00% 24896 25044 -0.59%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25519 25517 0.01% 26199 26241 -0.16%
sqlite u32_u64_str no_index 64 128 1 u64 123011 123011 0.00% 124339 124459 -0.10%
sqlite u32_u64_str no_index 64 128 2 string 143670 143670 0.00% 145266 145302 -0.02%
sqlite u32_u64_str btree_each_column 64 128 1 u64 130328 130328 0.00% 132006 131926 0.06%
sqlite u32_u64_str btree_each_column 64 128 2 string 133533 133533 0.00% 135435 135379 0.04%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 802112 802078 0.00% 822080 823216 -0.14%
stdb_raw u32_u64_str btree_each_column 64 128 947142 946984 0.02% 980302 976626 0.38%
sqlite u32_u64_str unique_0 64 128 396122 396122 0.00% 416992 412922 0.99%
sqlite u32_u64_str btree_each_column 64 128 969195 969195 0.00% 1013599 1000915 1.27%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 176793 176793 0.00% 176835 176863 -0.02%
stdb_raw u32_u64_str unique_0 64 17361 17361 0.00% 17403 17431 -0.16%
sqlite u32_u64_str unique_0 1024 1044670 1044670 0.00% 1048004 1048040 -0.00%
sqlite u32_u64_str unique_0 64 74736 74751 -0.02% 75946 75909 0.05%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 48688 48688 0.00% 51340 51340 0.00%
64 bsatn 26691 26691 0.00% 28969 28901 0.24%
16 json 12434 12434 0.00% 14338 14338 0.00%
16 bsatn 8373 8373 0.00% 9699 9631 0.71%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 21811146 21809884 0.01% 22289068 22506584 -0.97%
stdb_raw u32_u64_str unique_0 64 128 1266115 1266620 -0.04% 1302127 1315842 -1.04%
sqlite u32_u64_str unique_0 1024 1024 1801877 1801877 0.00% 1811097 1810885 0.01%
sqlite u32_u64_str unique_0 64 128 128413 128413 0.00% 131257 131241 0.01%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6651 6651 0.00% 7649 7625 0.31%
sqlite 5600 5600 0.00% 6088 6056 0.53%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 123367 123367 0.00% 124251 124091 0.13%
stdb_raw u32_u64_str no_index 64 128 1 u64 81039 81039 0.00% 81727 81615 0.14%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24816 24816 0.00% 25288 25372 -0.33%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25857 25857 0.00% 26597 26605 -0.03%
sqlite u32_u64_str no_index 64 128 2 string 145591 145591 0.00% 147447 147483 -0.02%
sqlite u32_u64_str no_index 64 128 1 u64 124947 124932 0.01% 126639 126620 0.02%
sqlite u32_u64_str btree_each_column 64 128 1 u64 132424 132424 0.00% 134420 134288 0.10%
sqlite u32_u64_str btree_each_column 64 128 2 string 135583 135583 0.00% 137751 137683 0.05%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 753074 753546 -0.06% 770658 772930 -0.29%
stdb_raw u32_u64_str btree_each_column 64 128 896897 896411 0.05% 926869 924427 0.26%
sqlite u32_u64_str unique_0 64 128 413670 413670 0.00% 433970 429794 0.97%
sqlite u32_u64_str btree_each_column 64 128 1019774 1019770 0.00% 1061742 1049008 1.21%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 177133 177133 0.00% 177235 177195 0.02%
stdb_raw u32_u64_str unique_0 64 17701 17701 0.00% 17803 17763 0.23%
sqlite u32_u64_str unique_0 1024 1047738 1047738 0.00% 1051496 1051520 -0.00%
sqlite u32_u64_str unique_0 64 76508 76508 0.00% 77902 77854 0.06%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 48688 48688 0.00% 51340 51340 0.00%
64 bsatn 26691 26691 0.00% 28969 28901 0.24%
16 json 12434 12434 0.00% 14338 14338 0.00%
16 bsatn 8373 8373 0.00% 9699 9631 0.71%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20500642 20499735 0.00% 21046076 21232603 -0.88%
stdb_raw u32_u64_str unique_0 64 128 1224523 1224523 0.00% 1265799 1278723 -1.01%
sqlite u32_u64_str unique_0 1024 1024 1809667 1809667 0.00% 1818345 1818369 -0.00%
sqlite u32_u64_str unique_0 64 128 132555 132555 0.00% 135445 135553 -0.08%

Please sign in to comment.