Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: update the benchmark dependencies #429

Merged
merged 26 commits into from
Jan 17, 2025
Merged

fix: update the benchmark dependencies #429

merged 26 commits into from
Jan 17, 2025

Conversation

nazarhussain
Copy link
Contributor

Motivation

Benchmark in different runtimes.

Description

Use the benchmark utility that we forked, which have supported in different runtimes.

Steps to test or reproduce

Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner January 10, 2025 16:03
@nazarhussain nazarhussain changed the title Update the benchmark dependencies test: update the benchmark dependencies Jan 10, 2025
@nazarhussain nazarhussain changed the title test: update the benchmark dependencies chore: update the benchmark dependencies Jan 10, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 13, 2025 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9d47cf3 Previous: 2fd9c04 Ratio
digestTwoHashObjects 50023 times 48.432 ms/op 48.385 ms/op 1.00
digest2Bytes32 50023 times 55.268 ms/op 54.739 ms/op 1.01
digest 50023 times 54.071 ms/op 55.134 ms/op 0.98
input length 32 1.2550 us/op 1.2220 us/op 1.03
input length 64 1.3360 us/op 1.3350 us/op 1.00
input length 128 2.3550 us/op 2.2710 us/op 1.04
input length 256 3.4180 us/op 3.3710 us/op 1.01
input length 512 5.6720 us/op 5.5760 us/op 1.02
input length 1024 10.974 us/op 10.768 us/op 1.02
digest 1000000 times 866.31 ms/op 884.65 ms/op 0.98
hashObjectToByteArray 50023 times 1.2305 ms/op 1.2315 ms/op 1.00
byteArrayToHashObject 50023 times 1.5858 ms/op 1.8030 ms/op 0.88
digest64 200092 times 213.08 ms/op 213.67 ms/op 1.00
hash 200092 times using batchHash4UintArray64s 242.45 ms/op 240.28 ms/op 1.01
digest64HashObjects 200092 times 192.74 ms/op 194.67 ms/op 0.99
hash 200092 times using batchHash4HashObjectInputs 204.49 ms/op 207.74 ms/op 0.98
getGindicesAtDepth 3.4960 us/op 4.1450 us/op 0.84
iterateAtDepth 6.4220 us/op 7.4670 us/op 0.86
getGindexBits 382.00 ns/op 454.00 ns/op 0.84
gindexIterator 879.00 ns/op 1.0330 us/op 0.85
HashComputationLevel.push then loop 28.494 ms/op 27.802 ms/op 1.02
HashComputation[] push then loop 38.986 ms/op 49.368 ms/op 0.79
hash 2 Uint8Array 500000 times - hashtree 222.46 ms/op 216.43 ms/op 1.03
hashTwoObjects 500000 times - hashtree 220.77 ms/op 214.97 ms/op 1.03
executeHashComputations - hashtree 10.326 ms/op 9.8174 ms/op 1.05
hash 2 Uint8Array 500000 times - as-sha256 560.94 ms/op 566.39 ms/op 0.99
hashTwoObjects 500000 times - as-sha256 512.66 ms/op 512.46 ms/op 1.00
executeHashComputations - as-sha256 45.271 ms/op 49.092 ms/op 0.92
hash 2 Uint8Array 500000 times - noble 1.2722 s/op 1.2320 s/op 1.03
hashTwoObjects 500000 times - noble 1.7470 s/op 1.6458 s/op 1.06
executeHashComputations - noble 36.678 ms/op 40.524 ms/op 0.91
getHashComputations 2.4372 ms/op 3.2876 ms/op 0.74
executeHashComputations 10.661 ms/op 11.015 ms/op 0.97
get root 16.491 ms/op 17.228 ms/op 0.96
getNodeH() x7812.5 avg hindex 12.227 us/op 12.973 us/op 0.94
getNodeH() x7812.5 index 0 7.5610 us/op 7.4830 us/op 1.01
getNodeH() x7812.5 index 7 7.5030 us/op 7.4880 us/op 1.00
getNodeH() x7812.5 index 7 with key array 6.3860 us/op 6.2660 us/op 1.02
new LeafNode() x7812.5 298.52 us/op 288.84 us/op 1.03
getHashComputations 250000 nodes 15.209 ms/op 17.077 ms/op 0.89
batchHash 250000 nodes 89.662 ms/op 83.672 ms/op 1.07
get root 250000 nodes 122.07 ms/op 118.39 ms/op 1.03
getHashComputations 500000 nodes 31.542 ms/op 32.790 ms/op 0.96
batchHash 500000 nodes 162.61 ms/op 170.81 ms/op 0.95
get root 500000 nodes 240.42 ms/op 237.45 ms/op 1.01
getHashComputations 1000000 nodes 69.182 ms/op 68.401 ms/op 1.01
batchHash 1000000 nodes 378.31 ms/op 314.97 ms/op 1.20
get root 1000000 nodes 486.50 ms/op 474.60 ms/op 1.03
multiproof - depth 15, 1 requested leaves 9.1980 us/op 8.0360 us/op 1.14
tree offset multiproof - depth 15, 1 requested leaves 20.241 us/op 17.505 us/op 1.16
compact multiproof - depth 15, 1 requested leaves 3.3970 us/op 2.9850 us/op 1.14
multiproof - depth 15, 2 requested leaves 11.621 us/op 11.659 us/op 1.00
tree offset multiproof - depth 15, 2 requested leaves 21.847 us/op 21.064 us/op 1.04
compact multiproof - depth 15, 2 requested leaves 3.4850 us/op 3.0240 us/op 1.15
multiproof - depth 15, 3 requested leaves 17.933 us/op 16.464 us/op 1.09
tree offset multiproof - depth 15, 3 requested leaves 30.491 us/op 27.416 us/op 1.11
compact multiproof - depth 15, 3 requested leaves 4.8260 us/op 3.6760 us/op 1.31
multiproof - depth 15, 4 requested leaves 23.399 us/op 21.480 us/op 1.09
tree offset multiproof - depth 15, 4 requested leaves 37.597 us/op 33.923 us/op 1.11
compact multiproof - depth 15, 4 requested leaves 6.0390 us/op 4.2500 us/op 1.42
packedRootsBytesToLeafNodes bytes 4000 offset 0 5.7660 us/op 5.5760 us/op 1.03
packedRootsBytesToLeafNodes bytes 4000 offset 1 5.5500 us/op 5.5020 us/op 1.01
packedRootsBytesToLeafNodes bytes 4000 offset 2 5.5360 us/op 5.5960 us/op 0.99
packedRootsBytesToLeafNodes bytes 4000 offset 3 5.5140 us/op 5.5090 us/op 1.00
subtreeFillToContents depth 40 count 250000 56.301 ms/op 49.667 ms/op 1.13
setRoot - gindexBitstring 20.961 ms/op 20.717 ms/op 1.01
setRoot - gindex 23.451 ms/op 21.769 ms/op 1.08
getRoot - gindexBitstring 2.5429 ms/op 2.6428 ms/op 0.96
getRoot - gindex 3.0694 ms/op 3.4452 ms/op 0.89
getHashObject then setHashObject 20.994 ms/op 22.668 ms/op 0.93
setNodeWithFn 19.475 ms/op 19.949 ms/op 0.98
getNodeAtDepth depth 0 x100000 279.95 us/op 284.47 us/op 0.98
setNodeAtDepth depth 0 x100000 2.3324 ms/op 2.5826 ms/op 0.90
getNodesAtDepth depth 0 x100000 312.30 us/op 312.60 us/op 1.00
setNodesAtDepth depth 0 x100000 750.24 us/op 770.25 us/op 0.97
getNodeAtDepth depth 1 x100000 342.20 us/op 345.54 us/op 0.99
setNodeAtDepth depth 1 x100000 7.6426 ms/op 8.3900 ms/op 0.91
getNodesAtDepth depth 1 x100000 437.63 us/op 436.17 us/op 1.00
setNodesAtDepth depth 1 x100000 7.7524 ms/op 8.0911 ms/op 0.96
getNodeAtDepth depth 2 x100000 757.14 us/op 742.41 us/op 1.02
setNodeAtDepth depth 2 x100000 15.090 ms/op 15.448 ms/op 0.98
getNodesAtDepth depth 2 x100000 18.200 ms/op 19.301 ms/op 0.94
setNodesAtDepth depth 2 x100000 22.067 ms/op 22.443 ms/op 0.98
tree.getNodesAtDepth - gindexes 8.6334 ms/op 9.0838 ms/op 0.95
tree.getNodesAtDepth - push all nodes 1.9315 ms/op 2.2499 ms/op 0.86
tree.getNodesAtDepth - navigation 311.38 us/op 312.00 us/op 1.00
tree.setNodesAtDepth - indexes 631.80 us/op 702.50 us/op 0.90
set at depth 8 780.00 ns/op 673.00 ns/op 1.16
set at depth 16 1.0830 us/op 1.2090 us/op 0.90
set at depth 32 1.8350 us/op 1.9830 us/op 0.93
iterateNodesAtDepth 8 256 13.685 us/op 14.029 us/op 0.98
getNodesAtDepth 8 256 3.7250 us/op 3.9860 us/op 0.93
iterateNodesAtDepth 16 65536 4.5705 ms/op 4.9523 ms/op 0.92
getNodesAtDepth 16 65536 1.6274 ms/op 2.1318 ms/op 0.76
iterateNodesAtDepth 32 250000 15.940 ms/op 17.971 ms/op 0.89
getNodesAtDepth 32 250000 4.6476 ms/op 5.0235 ms/op 0.93
iterateNodesAtDepth 40 250000 16.855 ms/op 16.973 ms/op 0.99
getNodesAtDepth 40 250000 4.7594 ms/op 5.2017 ms/op 0.91
250000 validators root getter 122.31 ms/op 119.60 ms/op 1.02
250000 validators batchHash() 93.422 ms/op 96.226 ms/op 0.97
250000 validators hashComputations 14.595 ms/op 13.681 ms/op 1.07
bitlist bytes to struct (120,90) 1.0270 us/op 1.0320 us/op 1.00
bitlist bytes to tree (120,90) 3.8600 us/op 2.9040 us/op 1.33
bitlist bytes to struct (2048,2048) 1.4310 us/op 1.1460 us/op 1.25
bitlist bytes to tree (2048,2048) 4.7220 us/op 4.6090 us/op 1.02
ByteListType - deserialize 12.286 ms/op 8.2940 ms/op 1.48
BasicListType - deserialize 19.032 ms/op 17.814 ms/op 1.07
ByteListType - serialize 8.4324 ms/op 8.2229 ms/op 1.03
BasicListType - serialize 14.724 ms/op 10.528 ms/op 1.40
BasicListType - tree_convertToStruct 30.528 ms/op 30.733 ms/op 0.99
List[uint8, 68719476736] len 300000 ViewDU.getAll() + iterate 7.1900 ms/op 5.0479 ms/op 1.42
List[uint8, 68719476736] len 300000 ViewDU.get(i) 4.2258 ms/op 4.4003 ms/op 0.96
Array.push len 300000 empty Array - number 7.8421 ms/op 7.7576 ms/op 1.01
Array.set len 300000 from new Array - number 2.6225 ms/op 2.3929 ms/op 1.10
Array.set len 300000 - number 7.0611 ms/op 6.8603 ms/op 1.03
Uint8Array.set len 300000 493.34 us/op 490.48 us/op 1.01
Uint32Array.set len 300000 573.44 us/op 554.60 us/op 1.03
Container({a: uint8, b: uint8}) getViewDU x300000 28.558 ms/op 25.535 ms/op 1.12
ContainerNodeStruct({a: uint8, b: uint8}) getViewDU x300000 11.661 ms/op 10.557 ms/op 1.10
List(Container) len 300000 ViewDU.getAllReadonly() + iterate 226.22 ms/op 222.67 ms/op 1.02
List(Container) len 300000 ViewDU.getAllReadonlyValues() + iterate 288.43 ms/op 288.04 ms/op 1.00
List(Container) len 300000 ViewDU.get(i) 8.7320 ms/op 8.4440 ms/op 1.03
List(Container) len 300000 ViewDU.getReadonly(i) 8.3568 ms/op 7.2905 ms/op 1.15
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonly() + iterate 39.180 ms/op 34.764 ms/op 1.13
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonlyValues() + iterate 6.0488 ms/op 6.9714 ms/op 0.87
List(ContainerNodeStruct) len 300000 ViewDU.get(i) 10.814 ms/op 6.7554 ms/op 1.60
List(ContainerNodeStruct) len 300000 ViewDU.getReadonly(i) 8.5564 ms/op 6.6821 ms/op 1.28
Array.push len 300000 empty Array - object 7.6656 ms/op 8.2629 ms/op 0.93
Array.set len 300000 from new Array - object 2.1500 ms/op 2.0682 ms/op 1.04
Array.set len 300000 - object 6.4500 ms/op 6.8760 ms/op 0.94
cachePermanentRootStruct no cache 5.6950 us/op 5.9280 us/op 0.96
cachePermanentRootStruct with cache 228.00 ns/op 245.00 ns/op 0.93
epochParticipation len 250000 rws 7813 2.2525 ms/op 2.3026 ms/op 0.98
BeaconState ViewDU hashTreeRoot() vc=200000 118.06 ms/op 119.71 ms/op 0.99
BeaconState ViewDU recursive hash - commit step vc=200000 5.2331 ms/op 5.0692 ms/op 1.03
BeaconState ViewDU validator tree creation vc=10000 41.072 ms/op 42.276 ms/op 0.97
BeaconState ViewDU batchHashTreeRoot vc=200000 110.20 ms/op 109.21 ms/op 1.01
BeaconState ViewDU hashTreeRoot - commit step vc=200000 94.586 ms/op 96.290 ms/op 0.98
BeaconState ViewDU hashTreeRoot - hash step vc=200000 19.476 ms/op 18.397 ms/op 1.06
deserialize Attestation - tree 3.8870 us/op 3.9770 us/op 0.98
deserialize Attestation - struct 2.0280 us/op 2.1880 us/op 0.93
deserialize SignedAggregateAndProof - tree 5.3740 us/op 5.3310 us/op 1.01
deserialize SignedAggregateAndProof - struct 3.3510 us/op 3.4600 us/op 0.97
deserialize SyncCommitteeMessage - tree 1.5250 us/op 1.5160 us/op 1.01
deserialize SyncCommitteeMessage - struct 1.2290 us/op 1.2600 us/op 0.98
deserialize SignedContributionAndProof - tree 3.4650 us/op 3.3480 us/op 1.03
deserialize SignedContributionAndProof - struct 3.1330 us/op 2.7630 us/op 1.13
deserialize SignedBeaconBlock - tree 342.46 us/op 320.10 us/op 1.07
deserialize SignedBeaconBlock - struct 152.86 us/op 146.11 us/op 1.05
BeaconState vc 300000 - deserialize tree 690.56 ms/op 676.70 ms/op 1.02
BeaconState vc 300000 - serialize tree 161.83 ms/op 176.66 ms/op 0.92
BeaconState.historicalRoots vc 300000 - deserialize tree 926.00 ns/op 963.00 ns/op 0.96
BeaconState.historicalRoots vc 300000 - serialize tree 799.00 ns/op 797.00 ns/op 1.00
BeaconState.validators vc 300000 - deserialize tree 644.24 ms/op 645.65 ms/op 1.00
BeaconState.validators vc 300000 - serialize tree 118.37 ms/op 108.09 ms/op 1.10
BeaconState.balances vc 300000 - deserialize tree 27.275 ms/op 31.118 ms/op 0.88
BeaconState.balances vc 300000 - serialize tree 4.9730 ms/op 5.4166 ms/op 0.92
BeaconState.previousEpochParticipation vc 300000 - deserialize tree 1.0772 ms/op 1.1177 ms/op 0.96
BeaconState.previousEpochParticipation vc 300000 - serialize tree 333.52 us/op 329.73 us/op 1.01
BeaconState.currentEpochParticipation vc 300000 - deserialize tree 1.0746 ms/op 1.1398 ms/op 0.94
BeaconState.currentEpochParticipation vc 300000 - serialize tree 331.30 us/op 329.92 us/op 1.00
BeaconState.inactivityScores vc 300000 - deserialize tree 26.872 ms/op 28.841 ms/op 0.93
BeaconState.inactivityScores vc 300000 - serialize tree 5.3007 ms/op 6.4320 ms/op 0.82
hashTreeRoot Attestation - struct 19.045 us/op 22.568 us/op 0.84
hashTreeRoot Attestation - tree 9.7150 us/op 9.7040 us/op 1.00
hashTreeRoot SignedAggregateAndProof - struct 23.949 us/op 34.438 us/op 0.70
hashTreeRoot SignedAggregateAndProof - tree 14.714 us/op 14.744 us/op 1.00
hashTreeRoot SyncCommitteeMessage - struct 6.0740 us/op 9.1540 us/op 0.66
hashTreeRoot SyncCommitteeMessage - tree 4.0270 us/op 3.6090 us/op 1.12
hashTreeRoot SignedContributionAndProof - struct 14.473 us/op 24.373 us/op 0.59
hashTreeRoot SignedContributionAndProof - tree 9.9280 us/op 10.036 us/op 0.99
hashTreeRoot SignedBeaconBlock - struct 1.4302 ms/op 1.7085 ms/op 0.84
hashTreeRoot SignedBeaconBlock - tree 872.53 us/op 874.80 us/op 1.00
hashTreeRoot Validator - struct 9.0220 us/op 10.415 us/op 0.87
hashTreeRoot Validator - tree 7.9400 us/op 8.8040 us/op 0.90
BeaconState vc 300000 - hashTreeRoot tree 2.3689 s/op 2.5106 s/op 0.94
BeaconState vc 300000 - batchHashTreeRoot tree 4.3709 s/op 4.5668 s/op 0.96
BeaconState.historicalRoots vc 300000 - hashTreeRoot tree 1.1980 us/op 1.1800 us/op 1.02
BeaconState.validators vc 300000 - hashTreeRoot tree 2.6630 s/op 2.6840 s/op 0.99
BeaconState.balances vc 300000 - hashTreeRoot tree 37.222 ms/op 37.232 ms/op 1.00
BeaconState.previousEpochParticipation vc 300000 - hashTreeRoot tree 5.0096 ms/op 4.6431 ms/op 1.08
BeaconState.currentEpochParticipation vc 300000 - hashTreeRoot tree 5.0096 ms/op 4.5680 ms/op 1.10
BeaconState.inactivityScores vc 300000 - hashTreeRoot tree 43.891 ms/op 39.894 ms/op 1.10
hash64 x18 9.3010 us/op 9.8100 us/op 0.95
hashTwoObjects x18 8.6340 us/op 8.9900 us/op 0.96
hash64 x1740 864.29 us/op 889.16 us/op 0.97
hashTwoObjects x1740 794.49 us/op 817.95 us/op 0.97
hash64 x2700000 1.3306 s/op 1.3707 s/op 0.97
hashTwoObjects x2700000 1.2403 s/op 1.2561 s/op 0.99
get_exitEpoch - ContainerType 306.00 ns/op 470.00 ns/op 0.65
get_exitEpoch - ContainerNodeStructType 298.00 ns/op 531.00 ns/op 0.56
set_exitEpoch - ContainerType 318.00 ns/op 743.00 ns/op 0.43
set_exitEpoch - ContainerNodeStructType 313.00 ns/op 867.00 ns/op 0.36
get_pubkey - ContainerType 1.2570 us/op 2.1440 us/op 0.59
get_pubkey - ContainerNodeStructType 293.00 ns/op 628.00 ns/op 0.47
hashTreeRoot - ContainerType 539.00 ns/op 911.00 ns/op 0.59
hashTreeRoot - ContainerNodeStructType 478.00 ns/op 822.00 ns/op 0.58
createProof - ContainerType 5.0560 us/op 7.2270 us/op 0.70
createProof - ContainerNodeStructType 25.448 us/op 38.455 us/op 0.66
serialize - ContainerType 2.0150 us/op 3.0960 us/op 0.65
serialize - ContainerNodeStructType 1.6870 us/op 2.3860 us/op 0.71
set_exitEpoch_and_hashTreeRoot - ContainerType 3.0930 us/op 5.1660 us/op 0.60
set_exitEpoch_and_hashTreeRoot - ContainerNodeStructType 8.4070 us/op 11.845 us/op 0.71
Array - for of 5.6080 us/op 6.0820 us/op 0.92
Array - for(;;) 5.6170 us/op 6.0990 us/op 0.92
basicListValue.readonlyValuesArray() 5.7151 ms/op 8.5034 ms/op 0.67
basicListValue.readonlyValuesArray() + loop all 6.0575 ms/op 9.2078 ms/op 0.66
compositeListValue.readonlyValuesArray() 32.104 ms/op 31.312 ms/op 1.03
compositeListValue.readonlyValuesArray() + loop all 31.898 ms/op 32.164 ms/op 0.99
Number64UintType - get balances list 5.1828 ms/op 4.9334 ms/op 1.05
Number64UintType - set balances list 10.165 ms/op 10.233 ms/op 0.99
Number64UintType - get and increase 10 then set 46.657 ms/op 45.730 ms/op 1.02
Number64UintType - increase 10 using applyDelta 18.827 ms/op 19.214 ms/op 0.98
Number64UintType - increase 10 using applyDeltaInBatch 18.497 ms/op 18.532 ms/op 1.00
tree_newTreeFromUint64Deltas 21.885 ms/op 23.803 ms/op 0.92
unsafeUint8ArrayToTree 41.762 ms/op 43.554 ms/op 0.96
bitLength(50) 277.00 ns/op 283.00 ns/op 0.98
bitLengthStr(50) 243.00 ns/op 260.00 ns/op 0.93
bitLength(8000) 271.00 ns/op 280.00 ns/op 0.97
bitLengthStr(8000) 290.00 ns/op 301.00 ns/op 0.96
bitLength(250000) 263.00 ns/op 276.00 ns/op 0.95
bitLengthStr(250000) 345.00 ns/op 340.00 ns/op 1.01
floor - Math.floor (53) 1.2444 ns/op 1.2463 ns/op 1.00
floor - << 0 (53) 1.2446 ns/op 1.2449 ns/op 1.00
floor - Math.floor (512) 1.2435 ns/op 1.2485 ns/op 1.00
floor - << 0 (512) 1.2435 ns/op 1.2445 ns/op 1.00
fnIf(0) 1.5538 ns/op 1.5550 ns/op 1.00
fnSwitch(0) 2.1766 ns/op 2.2084 ns/op 0.99
fnObj(0) 1.5591 ns/op 1.5592 ns/op 1.00
fnArr(0) 1.5802 ns/op 1.5564 ns/op 1.02
fnIf(4) 2.1759 ns/op 2.2012 ns/op 0.99
fnSwitch(4) 2.1767 ns/op 2.1851 ns/op 1.00
fnObj(4) 1.5549 ns/op 1.5553 ns/op 1.00
fnArr(4) 1.5558 ns/op 1.5566 ns/op 1.00
fnIf(9) 3.1079 ns/op 3.1105 ns/op 1.00
fnSwitch(9) 2.1826 ns/op 2.1789 ns/op 1.00
fnObj(9) 1.5566 ns/op 1.5566 ns/op 1.00
fnArr(9) 1.5562 ns/op 1.5585 ns/op 1.00
Container {a,b,vec} - as struct x100000 124.51 us/op 124.70 us/op 1.00
Container {a,b,vec} - as tree x100000 871.82 us/op 528.88 us/op 1.65
Container {a,vec,b} - as struct x100000 156.25 us/op 156.29 us/op 1.00
Container {a,vec,b} - as tree x100000 498.02 us/op 497.88 us/op 1.00
get 2 props x1000000 - rawObject 311.62 us/op 311.55 us/op 1.00
get 2 props x1000000 - proxy 73.595 ms/op 73.231 ms/op 1.00
get 2 props x1000000 - customObj 310.84 us/op 311.63 us/op 1.00
Simple object binary -> struct 821.00 ns/op 666.00 ns/op 1.23
Simple object binary -> tree_backed 2.3160 us/op 1.8000 us/op 1.29
Simple object struct -> tree_backed 3.1500 us/op 2.5550 us/op 1.23
Simple object tree_backed -> struct 2.1780 us/op 1.8870 us/op 1.15
Simple object struct -> binary 1.0200 us/op 1.1100 us/op 0.92
Simple object tree_backed -> binary 1.7310 us/op 1.5810 us/op 1.09
aggregationBits binary -> struct 703.00 ns/op 556.00 ns/op 1.26
aggregationBits binary -> tree_backed 2.7840 us/op 2.3520 us/op 1.18
aggregationBits struct -> tree_backed 3.1730 us/op 2.8250 us/op 1.12
aggregationBits tree_backed -> struct 1.3210 us/op 1.1760 us/op 1.12
aggregationBits struct -> binary 869.00 ns/op 895.00 ns/op 0.97
aggregationBits tree_backed -> binary 1.2050 us/op 1.0710 us/op 1.13
List(uint8) 100000 binary -> struct 1.6995 ms/op 1.7193 ms/op 0.99
List(uint8) 100000 binary -> tree_backed 340.43 us/op 378.88 us/op 0.90
List(uint8) 100000 struct -> tree_backed 1.4644 ms/op 1.4716 ms/op 1.00
List(uint8) 100000 tree_backed -> struct 1.1412 ms/op 1.1255 ms/op 1.01
List(uint8) 100000 struct -> binary 1.1321 ms/op 1.1255 ms/op 1.01
List(uint8) 100000 tree_backed -> binary 109.44 us/op 133.70 us/op 0.82
List(uint64Number) 100000 binary -> struct 1.5037 ms/op 1.4898 ms/op 1.01
List(uint64Number) 100000 binary -> tree_backed 4.8328 ms/op 5.0622 ms/op 0.95
List(uint64Number) 100000 struct -> tree_backed 6.8299 ms/op 7.2356 ms/op 0.94
List(uint64Number) 100000 tree_backed -> struct 2.7538 ms/op 2.9213 ms/op 0.94
List(uint64Number) 100000 struct -> binary 1.7865 ms/op 1.8754 ms/op 0.95
List(uint64Number) 100000 tree_backed -> binary 1.0432 ms/op 1.9946 ms/op 0.52
List(Uint64Bigint) 100000 binary -> struct 4.0728 ms/op 4.3077 ms/op 0.95
List(Uint64Bigint) 100000 binary -> tree_backed 5.6162 ms/op 5.2529 ms/op 1.07
List(Uint64Bigint) 100000 struct -> tree_backed 7.6165 ms/op 8.0646 ms/op 0.94
List(Uint64Bigint) 100000 tree_backed -> struct 5.9576 ms/op 5.6323 ms/op 1.06
List(Uint64Bigint) 100000 struct -> binary 2.1018 ms/op 2.0885 ms/op 1.01
List(Uint64Bigint) 100000 tree_backed -> binary 1.8562 ms/op 1.8135 ms/op 1.02
Vector(Root) 100000 binary -> struct 42.518 ms/op 41.100 ms/op 1.03
Vector(Root) 100000 binary -> tree_backed 45.164 ms/op 44.078 ms/op 1.02
Vector(Root) 100000 struct -> tree_backed 57.422 ms/op 55.892 ms/op 1.03
Vector(Root) 100000 tree_backed -> struct 57.326 ms/op 56.983 ms/op 1.01
Vector(Root) 100000 struct -> binary 3.9038 ms/op 2.7534 ms/op 1.42
Vector(Root) 100000 tree_backed -> binary 7.8065 ms/op 8.0801 ms/op 0.97
List(Validator) 100000 binary -> struct 107.86 ms/op 113.34 ms/op 0.95
List(Validator) 100000 binary -> tree_backed 381.67 ms/op 390.64 ms/op 0.98
List(Validator) 100000 struct -> tree_backed 405.05 ms/op 417.76 ms/op 0.97
List(Validator) 100000 tree_backed -> struct 221.55 ms/op 226.69 ms/op 0.98
List(Validator) 100000 struct -> binary 29.143 ms/op 29.274 ms/op 1.00
List(Validator) 100000 tree_backed -> binary 111.38 ms/op 108.34 ms/op 1.03
List(Validator-NS) 100000 binary -> struct 120.97 ms/op 135.28 ms/op 0.89
List(Validator-NS) 100000 binary -> tree_backed 170.78 ms/op 184.51 ms/op 0.93
List(Validator-NS) 100000 struct -> tree_backed 215.31 ms/op 235.91 ms/op 0.91
List(Validator-NS) 100000 tree_backed -> struct 172.45 ms/op 192.62 ms/op 0.90
List(Validator-NS) 100000 struct -> binary 28.925 ms/op 29.747 ms/op 0.97
List(Validator-NS) 100000 tree_backed -> binary 35.665 ms/op 35.712 ms/op 1.00
get epochStatuses - MutableVector 99.292 us/op 101.62 us/op 0.98
get epochStatuses - ViewDU 204.98 us/op 203.65 us/op 1.01
set epochStatuses - ListTreeView 2.2476 ms/op 2.2079 ms/op 1.02
set epochStatuses - ListTreeView - set() 439.95 us/op 431.41 us/op 1.02
set epochStatuses - ListTreeView - commit() 798.92 us/op 804.31 us/op 0.99
bitstring 518.73 ns/op 520.44 ns/op 1.00
bit mask 13.575 ns/op 13.407 ns/op 1.01
struct - increase slot to 1000000 933.61 us/op 936.53 us/op 1.00
UintNumberType - increase slot to 1000000 26.774 ms/op 26.798 ms/op 1.00
UintBigintType - increase slot to 1000000 197.85 ms/op 214.36 ms/op 0.92
UintBigint8 x 100000 tree_deserialize 5.4253 ms/op 5.8918 ms/op 0.92
UintBigint8 x 100000 tree_serialize 1.0994 ms/op 1.1095 ms/op 0.99
UintBigint16 x 100000 tree_deserialize 5.5547 ms/op 6.7823 ms/op 0.82
UintBigint16 x 100000 tree_serialize 1.4518 ms/op 1.7868 ms/op 0.81
UintBigint32 x 100000 tree_deserialize 6.5090 ms/op 7.0351 ms/op 0.93
UintBigint32 x 100000 tree_serialize 2.1800 ms/op 2.3450 ms/op 0.93
UintBigint64 x 100000 tree_deserialize 6.9033 ms/op 8.0089 ms/op 0.86
UintBigint64 x 100000 tree_serialize 2.7886 ms/op 3.3089 ms/op 0.84
UintBigint8 x 100000 value_deserialize 435.37 us/op 436.55 us/op 1.00
UintBigint8 x 100000 value_serialize 811.00 us/op 944.72 us/op 0.86
UintBigint16 x 100000 value_deserialize 467.40 us/op 466.98 us/op 1.00
UintBigint16 x 100000 value_serialize 921.68 us/op 949.22 us/op 0.97
UintBigint32 x 100000 value_deserialize 497.91 us/op 497.75 us/op 1.00
UintBigint32 x 100000 value_serialize 979.08 us/op 1.0340 ms/op 0.95
UintBigint64 x 100000 value_deserialize 566.37 us/op 561.86 us/op 1.01
UintBigint64 x 100000 value_serialize 1.1073 ms/op 1.4075 ms/op 0.79
UintBigint8 x 100000 deserialize 3.6885 ms/op 4.0786 ms/op 0.90
UintBigint8 x 100000 serialize 1.5640 ms/op 1.6495 ms/op 0.95
UintBigint16 x 100000 deserialize 3.7278 ms/op 4.2306 ms/op 0.88
UintBigint16 x 100000 serialize 1.5809 ms/op 1.5280 ms/op 1.03
UintBigint32 x 100000 deserialize 3.7624 ms/op 4.1933 ms/op 0.90
UintBigint32 x 100000 serialize 3.0197 ms/op 3.1683 ms/op 0.95
UintBigint64 x 100000 deserialize 4.2314 ms/op 4.8024 ms/op 0.88
UintBigint64 x 100000 serialize 1.6573 ms/op 1.6952 ms/op 0.98
UintBigint128 x 100000 deserialize 6.0831 ms/op 6.8769 ms/op 0.88
UintBigint128 x 100000 serialize 16.352 ms/op 15.576 ms/op 1.05
UintBigint256 x 100000 deserialize 9.9283 ms/op 9.2019 ms/op 1.08
UintBigint256 x 100000 serialize 46.533 ms/op 45.317 ms/op 1.03
Slice from Uint8Array x25000 1.6066 ms/op 1.3730 ms/op 1.17
Slice from ArrayBuffer x25000 16.391 ms/op 16.661 ms/op 0.98
Slice from ArrayBuffer x25000 + new Uint8Array 18.177 ms/op 18.610 ms/op 0.98
Copy Uint8Array 100000 iterate 2.7517 ms/op 2.6541 ms/op 1.04
Copy Uint8Array 100000 slice 115.24 us/op 120.03 us/op 0.96
Copy Uint8Array 100000 Uint8Array.prototype.slice.call 113.22 us/op 119.67 us/op 0.95
Copy Buffer 100000 Uint8Array.prototype.slice.call 112.51 us/op 124.86 us/op 0.90
Copy Uint8Array 100000 slice + set 228.05 us/op 251.50 us/op 0.91
Copy Uint8Array 100000 subarray + set 110.91 us/op 122.68 us/op 0.90
Copy Uint8Array 100000 slice arrayBuffer 108.03 us/op 124.22 us/op 0.87
Uint64 deserialize 100000 - iterate Uint8Array 2.0942 ms/op 2.2252 ms/op 0.94
Uint64 deserialize 100000 - by Uint32A 2.0842 ms/op 2.1293 ms/op 0.98
Uint64 deserialize 100000 - by DataView.getUint32 x2 2.1064 ms/op 2.1917 ms/op 0.96
Uint64 deserialize 100000 - by DataView.getBigUint64 5.2852 ms/op 5.4629 ms/op 0.97
Uint64 deserialize 100000 - by byte 40.864 ms/op 41.007 ms/op 1.00

by benchmarkbot/action

matthewkeil
matthewkeil reviewed Jan 17, 2025
View reviewed changes
Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!! 🎸

Just the couple questions

.benchrc.yaml Show resolved Hide resolved
package.json Show resolved Hide resolved
matthewkeil
matthewkeil approved these changes Jan 17, 2025
View reviewed changes
Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work figuring out that memory leak!!! You 🪨 thoroughly!!!

@matthewkeil matthewkeil enabled auto-merge (squash) January 17, 2025 15:09
@matthewkeil matthewkeil changed the title chore: update the benchmark dependencies fix: update the benchmark dependencies Jan 17, 2025
@matthewkeil matthewkeil merged commit 3d931df into master Jan 17, 2025
9 checks passed
@matthewkeil matthewkeil deleted the nh/benchmark-update branch January 17, 2025 15:13
@matthewkeil matthewkeil restored the nh/benchmark-update branch January 17, 2025 15:14
@nazarhussain nazarhussain deleted the nh/benchmark-update branch January 17, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewkeil matthewkeil matthewkeil approved these changes

Assignees
No one assigned
Labels
as-sha256 persistent-merkle-tree persistent-ts ssz
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nazarhussain @matthewkeil