Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: single state tree at start up #7056

Merged
merged 5 commits into from
Sep 10, 2024
Merged

fix: single state tree at start up #7056

merged 5 commits into from
Sep 10, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Aug 28, 2024 •
edited
Loading

Motivation

  • we deserialize states 2 times to 2 separate tree: one for db state and one for ws state, and each hashTreeRoot() call takes ~24s of blocking time at startup
  • when persisting anchor state, we serialize it again while we've just downloaded it. Also with db state, we should not persist it to db again

Description

  • after we got db state, use loadState() api to load ws state
  • only persist anchor state if it's ws state, and do not serialize again
  • rename initStateFromAnchorState to checkAndPersistAnchorState and no return value, this is to reflect its implementation
  • minor thing: add formatBytes() util to log how many bytes we downloaded/loaded. Later on we should use the same api to track state diff PR feat: use binary diff to persist finalized states #7005

Closes #7027

Test

Aug-28 07:44:40.777[]                 ^[[32minfo^[[39m: Lodestar network=holesky, version=v1.21.0/te/single_state_tree_at_start_up/ba3cd96, commit=ba3cd96ac968f5598bbc998c7a3b5ebf650a91e3
Aug-28 07:44:40.962[]                 ^[[32minfo^[[39m: Connected to LevelDB database
Aug-28 07:44:41.732[]              ^[[36mverbose^[[39m: Found the last archived state slot=2410272, size=241.63 MB
Aug-28 07:45:04.668[]                 ^[[33mwarn^[[39m: Forced syncing from checkpoint even though db state at slot 2410272 is within weak subjectivity period
Aug-28 07:45:04.668[]                 ^[[33mwarn^[[39m: Please consider removing --forceCheckpointSync flag unless absolutely necessary
Aug-28 07:45:04.669[]                 ^[[32minfo^[[39m: Fetching checkpoint state checkpointSyncUrl=https://beaconstate-holesky.chainsafe.io

Aug-28 07:45:09.532[]                 ^[[32minfo^[[39m: Download completed stateId=finalized, size=241.53 MB, slot=2410656
Aug-28 07:45:10.858[]                 ^[[32minfo^[[39m: Initializing beacon from a valid checkpoint state slot=2410656, epoch=75333, stateRoot=0x68615af64eed72e6381584063955956a9cbe74a6d8e37c95a10dd0989493f05b, isWithinWeakSubjectivityPeriod=true

after "Download completed" log, the beacon node can continue almost immediately instead of blocking for ~24s as currently #7027 (comment)

@codecov Codecov
Copy link

codecov bot commented Aug 28, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 26.36364% with 81 lines in your changes missing coverage. Please review.

Project coverage is 49.24%. Comparing base (4e22884) to head (067f620).
Report is 5 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7056      +/-   ##
============================================
- Coverage     49.26%   49.24%   -0.02%     
============================================
  Files           592      593       +1     
  Lines         39365    39447      +82     
  Branches       2251     2248       -3     
============================================
+ Hits          19393    19427      +34     
- Misses        19931    19979      +48     
  Partials         41       41

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 28, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4c54bd3 Previous: b05c93e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9256 ms/op 1.8922 ms/op 1.02
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 39.989 us/op 50.311 us/op 0.79
BLS verify - blst 880.94 us/op 910.11 us/op 0.97
BLS verifyMultipleSignatures 3 - blst 1.3164 ms/op 1.2178 ms/op 1.08
BLS verifyMultipleSignatures 8 - blst 2.0327 ms/op 1.9058 ms/op 1.07
BLS verifyMultipleSignatures 32 - blst 4.4245 ms/op 5.4504 ms/op 0.81
BLS verifyMultipleSignatures 64 - blst 8.3506 ms/op 10.784 ms/op 0.77
BLS verifyMultipleSignatures 128 - blst 16.087 ms/op 17.466 ms/op 0.92
BLS deserializing 10000 signatures 609.98 ms/op 682.83 ms/op 0.89
BLS deserializing 100000 signatures 6.0346 s/op 6.8248 s/op 0.88
BLS verifyMultipleSignatures - same message - 3 - blst 966.47 us/op 976.91 us/op 0.99
BLS verifyMultipleSignatures - same message - 8 - blst 1.0888 ms/op 1.0508 ms/op 1.04
BLS verifyMultipleSignatures - same message - 32 - blst 1.7091 ms/op 1.7194 ms/op 0.99
BLS verifyMultipleSignatures - same message - 64 - blst 2.5895 ms/op 2.6214 ms/op 0.99
BLS verifyMultipleSignatures - same message - 128 - blst 4.2082 ms/op 4.3843 ms/op 0.96
BLS aggregatePubkeys 32 - blst 17.901 us/op 20.089 us/op 0.89
BLS aggregatePubkeys 128 - blst 63.355 us/op 70.620 us/op 0.90
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 46.950 ms/op 66.242 ms/op 0.71
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 44.831 ms/op 65.782 ms/op 0.68
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 29.434 ms/op 40.036 ms/op 0.74
getSlashingsAndExits - default max 69.171 us/op 95.929 us/op 0.72
getSlashingsAndExits - 2k 247.40 us/op 325.96 us/op 0.76
proposeBlockBody type=full, size=empty 4.7973 ms/op 5.8315 ms/op 0.82
isKnown best case - 1 super set check 463.00 ns/op 337.00 ns/op 1.37
isKnown normal case - 2 super set checks 439.00 ns/op 323.00 ns/op 1.36
isKnown worse case - 16 super set checks 441.00 ns/op 308.00 ns/op 1.43
InMemoryCheckpointStateCache - add get delete 2.6000 us/op 3.1530 us/op 0.82
updateUnfinalizedPubkeys - updating 10 pubkeys 610.17 us/op 619.81 us/op 0.98
updateUnfinalizedPubkeys - updating 100 pubkeys 3.2116 ms/op 3.1527 ms/op 1.02
updateUnfinalizedPubkeys - updating 1000 pubkeys 37.924 ms/op 55.751 ms/op 0.68
validate api signedAggregateAndProof - struct 1.9618 ms/op 1.6223 ms/op 1.21
validate gossip signedAggregateAndProof - struct 1.9193 ms/op 2.5773 ms/op 0.74
validate gossip attestation - vc 640000 950.48 us/op 1.1353 ms/op 0.84
batch validate gossip attestation - vc 640000 - chunk 32 118.89 us/op 138.26 us/op 0.86
batch validate gossip attestation - vc 640000 - chunk 64 101.12 us/op 114.88 us/op 0.88
batch validate gossip attestation - vc 640000 - chunk 128 95.343 us/op 107.09 us/op 0.89
batch validate gossip attestation - vc 640000 - chunk 256 92.485 us/op 106.31 us/op 0.87
pickEth1Vote - no votes 779.66 us/op 1.2374 ms/op 0.63
pickEth1Vote - max votes 5.6848 ms/op 8.7804 ms/op 0.65
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.037 ms/op 13.294 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.437 ms/op 19.563 ms/op 1.04
pickEth1Vote - Eth1Data fastSerialize value x2048 329.71 us/op 488.64 us/op 0.67
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.4173 ms/op 2.7160 ms/op 1.26
bytes32 toHexString 550.00 ns/op 475.00 ns/op 1.16
bytes32 Buffer.toString(hex) 404.00 ns/op 264.00 ns/op 1.53
bytes32 Buffer.toString(hex) from Uint8Array 510.00 ns/op 390.00 ns/op 1.31
bytes32 Buffer.toString(hex) + 0x 414.00 ns/op 270.00 ns/op 1.53
Object access 1 prop 0.30300 ns/op 0.16000 ns/op 1.89
Map access 1 prop 0.30700 ns/op 0.15200 ns/op 2.02
Object get x1000 4.7570 ns/op 6.0560 ns/op 0.79
Map get x1000 5.4470 ns/op 6.7270 ns/op 0.81
Object set x1000 25.507 ns/op 35.636 ns/op 0.72
Map set x1000 18.618 ns/op 24.800 ns/op 0.75
Return object 10000 times 0.28060 ns/op 0.29920 ns/op 0.94
Throw Error 10000 times 2.5062 us/op 3.4705 us/op 0.72
toHex 97.949 ns/op 149.65 ns/op 0.65
Buffer.from 93.174 ns/op 139.04 ns/op 0.67
shared Buffer 63.578 ns/op 89.769 ns/op 0.71
fastMsgIdFn sha256 / 200 bytes 1.8340 us/op 2.3200 us/op 0.79
fastMsgIdFn h32 xxhash / 200 bytes 383.00 ns/op 245.00 ns/op 1.56
fastMsgIdFn h64 xxhash / 200 bytes 426.00 ns/op 280.00 ns/op 1.52
fastMsgIdFn sha256 / 1000 bytes 5.4520 us/op 7.8060 us/op 0.70
fastMsgIdFn h32 xxhash / 1000 bytes 504.00 ns/op 383.00 ns/op 1.32
fastMsgIdFn h64 xxhash / 1000 bytes 490.00 ns/op 364.00 ns/op 1.35
fastMsgIdFn sha256 / 10000 bytes 45.949 us/op 66.111 us/op 0.70
fastMsgIdFn h32 xxhash / 10000 bytes 1.8000 us/op 1.9200 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.2510 us/op 1.2590 us/op 0.99
send data - 1000 256B messages 9.9826 ms/op 13.248 ms/op 0.75
send data - 1000 512B messages 12.924 ms/op 17.720 ms/op 0.73
send data - 1000 1024B messages 20.494 ms/op 26.656 ms/op 0.77
send data - 1000 1200B messages 23.252 ms/op 23.076 ms/op 1.01
send data - 1000 2048B messages 29.634 ms/op 32.507 ms/op 0.91
send data - 1000 4096B messages 26.461 ms/op 32.327 ms/op 0.82
send data - 1000 16384B messages 66.011 ms/op 75.337 ms/op 0.88
send data - 1000 65536B messages 255.78 ms/op 223.62 ms/op 1.14
enrSubnets - fastDeserialize 64 bits 1.0470 us/op 1.0880 us/op 0.96
enrSubnets - ssz BitVector 64 bits 497.00 ns/op 402.00 ns/op 1.24
enrSubnets - fastDeserialize 4 bits 313.00 ns/op 195.00 ns/op 1.61
enrSubnets - ssz BitVector 4 bits 505.00 ns/op 447.00 ns/op 1.13
prioritizePeers score -10:0 att 32-0.1 sync 2-0 125.06 us/op 175.39 us/op 0.71
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 151.40 us/op 195.94 us/op 0.77
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 270.75 us/op 391.73 us/op 0.69
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 547.44 us/op 481.67 us/op 1.14
prioritizePeers score 0:0 att 64-1 sync 4-1 553.63 us/op 739.55 us/op 0.75
array of 16000 items push then shift 1.2003 us/op 1.6761 us/op 0.72
LinkedList of 16000 items push then shift 7.0510 ns/op 7.8510 ns/op 0.90
array of 16000 items push then pop 81.621 ns/op 140.00 ns/op 0.58
LinkedList of 16000 items push then pop 6.0050 ns/op 7.8060 ns/op 0.77
array of 24000 items push then shift 1.7466 us/op 2.5742 us/op 0.68
LinkedList of 24000 items push then shift 6.6790 ns/op 8.4590 ns/op 0.79
array of 24000 items push then pop 105.90 ns/op 188.23 ns/op 0.56
LinkedList of 24000 items push then pop 5.9450 ns/op 7.6230 ns/op 0.78
intersect bitArray bitLen 8 5.0990 ns/op 6.6780 ns/op 0.76
intersect array and set length 8 36.729 ns/op 54.364 ns/op 0.68
intersect bitArray bitLen 128 25.304 ns/op 31.345 ns/op 0.81
intersect array and set length 128 544.87 ns/op 929.26 ns/op 0.59
bitArray.getTrueBitIndexes() bitLen 128 1.8810 us/op 2.3200 us/op 0.81
bitArray.getTrueBitIndexes() bitLen 248 3.4190 us/op 4.3940 us/op 0.78
bitArray.getTrueBitIndexes() bitLen 512 6.2800 us/op 9.7670 us/op 0.64
Buffer.concat 32 items 1.0130 us/op 1.0730 us/op 0.94
Uint8Array.set 32 items 1.4970 us/op 1.6300 us/op 0.92
Buffer.copy 1.6620 us/op 2.1820 us/op 0.76
Uint8Array.set - with subarray 2.5330 us/op 3.1510 us/op 0.80
Uint8Array.set - without subarray 2.0820 us/op 1.6310 us/op 1.28
getUint32 - dataview 392.00 ns/op 316.00 ns/op 1.24
getUint32 - manual 323.00 ns/op 255.00 ns/op 1.27
Set add up to 64 items then delete first 1.6928 us/op 2.7536 us/op 0.61
OrderedSet add up to 64 items then delete first 2.6382 us/op 4.0716 us/op 0.65
Set add up to 64 items then delete last 1.9193 us/op 2.9983 us/op 0.64
OrderedSet add up to 64 items then delete last 2.9135 us/op 4.7860 us/op 0.61
Set add up to 64 items then delete middle 1.9220 us/op 2.6288 us/op 0.73
OrderedSet add up to 64 items then delete middle 4.2518 us/op 6.3458 us/op 0.67
Set add up to 128 items then delete first 3.7807 us/op 5.9284 us/op 0.64
OrderedSet add up to 128 items then delete first 5.9909 us/op 10.197 us/op 0.59
Set add up to 128 items then delete last 3.6620 us/op 5.8826 us/op 0.62
OrderedSet add up to 128 items then delete last 5.6344 us/op 8.8673 us/op 0.64
Set add up to 128 items then delete middle 3.6687 us/op 5.8234 us/op 0.63
OrderedSet add up to 128 items then delete middle 10.895 us/op 15.080 us/op 0.72
Set add up to 256 items then delete first 7.4726 us/op 10.754 us/op 0.69
OrderedSet add up to 256 items then delete first 12.039 us/op 16.999 us/op 0.71
Set add up to 256 items then delete last 7.2311 us/op 11.025 us/op 0.66
OrderedSet add up to 256 items then delete last 11.350 us/op 16.788 us/op 0.68
Set add up to 256 items then delete middle 7.2131 us/op 10.133 us/op 0.71
OrderedSet add up to 256 items then delete middle 32.977 us/op 43.009 us/op 0.77
transfer serialized Status (84 B) 1.5020 us/op 1.4900 us/op 1.01
copy serialized Status (84 B) 1.3040 us/op 1.2340 us/op 1.06
transfer serialized SignedVoluntaryExit (112 B) 1.6620 us/op 1.5790 us/op 1.05
copy serialized SignedVoluntaryExit (112 B) 1.4240 us/op 1.2630 us/op 1.13
transfer serialized ProposerSlashing (416 B) 2.4040 us/op 2.0960 us/op 1.15
copy serialized ProposerSlashing (416 B) 2.5020 us/op 1.8120 us/op 1.38
transfer serialized Attestation (485 B) 2.4820 us/op 1.9030 us/op 1.30
copy serialized Attestation (485 B) 2.5130 us/op 1.8250 us/op 1.38
transfer serialized AttesterSlashing (33232 B) 2.9200 us/op 2.5030 us/op 1.17
copy serialized AttesterSlashing (33232 B) 10.885 us/op 5.4330 us/op 2.00
transfer serialized Small SignedBeaconBlock (128000 B) 3.7150 us/op 3.0420 us/op 1.22
copy serialized Small SignedBeaconBlock (128000 B) 14.894 us/op 15.933 us/op 0.93
transfer serialized Avg SignedBeaconBlock (200000 B) 3.8640 us/op 3.4720 us/op 1.11
copy serialized Avg SignedBeaconBlock (200000 B) 20.374 us/op 21.993 us/op 0.93
transfer serialized BlobsSidecar (524380 B) 3.0970 us/op 3.2270 us/op 0.96
copy serialized BlobsSidecar (524380 B) 77.025 us/op 97.114 us/op 0.79
transfer serialized Big SignedBeaconBlock (1000000 B) 3.0270 us/op 3.2000 us/op 0.95
copy serialized Big SignedBeaconBlock (1000000 B) 248.05 us/op 258.56 us/op 0.96
pass gossip attestations to forkchoice per slot 2.4365 ms/op 2.8533 ms/op 0.85
forkChoice updateHead vc 100000 bc 64 eq 0 445.24 us/op 497.46 us/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 0 2.6932 ms/op 3.9550 ms/op 0.68
forkChoice updateHead vc 1000000 bc 64 eq 0 4.4293 ms/op 5.2531 ms/op 0.84
forkChoice updateHead vc 600000 bc 320 eq 0 2.3533 ms/op 3.0745 ms/op 0.77
forkChoice updateHead vc 600000 bc 1200 eq 0 2.5767 ms/op 3.2760 ms/op 0.79
forkChoice updateHead vc 600000 bc 7200 eq 0 2.8175 ms/op 3.7638 ms/op 0.75
forkChoice updateHead vc 600000 bc 64 eq 1000 9.1395 ms/op 10.896 ms/op 0.84
forkChoice updateHead vc 600000 bc 64 eq 10000 9.0798 ms/op 10.949 ms/op 0.83
forkChoice updateHead vc 600000 bc 64 eq 300000 13.898 ms/op 14.463 ms/op 0.96
computeDeltas 500000 validators 300 proto nodes 2.8788 ms/op 3.7476 ms/op 0.77
computeDeltas 500000 validators 1200 proto nodes 2.8974 ms/op 3.7564 ms/op 0.77
computeDeltas 500000 validators 7200 proto nodes 2.9662 ms/op 3.8664 ms/op 0.77
computeDeltas 750000 validators 300 proto nodes 4.4846 ms/op 5.4558 ms/op 0.82
computeDeltas 750000 validators 1200 proto nodes 4.3739 ms/op 5.4714 ms/op 0.80
computeDeltas 750000 validators 7200 proto nodes 4.2493 ms/op 5.3540 ms/op 0.79
computeDeltas 1400000 validators 300 proto nodes 8.2088 ms/op 10.441 ms/op 0.79
computeDeltas 1400000 validators 1200 proto nodes 8.2526 ms/op 10.633 ms/op 0.78
computeDeltas 1400000 validators 7200 proto nodes 8.1490 ms/op 10.172 ms/op 0.80
computeDeltas 2100000 validators 300 proto nodes 12.177 ms/op 15.100 ms/op 0.81
computeDeltas 2100000 validators 1200 proto nodes 12.273 ms/op 15.241 ms/op 0.81
computeDeltas 2100000 validators 7200 proto nodes 12.370 ms/op 15.623 ms/op 0.79
altair processAttestation - 250000 vs - 7PWei normalcase 1.4308 ms/op 2.1149 ms/op 0.68
altair processAttestation - 250000 vs - 7PWei worstcase 2.1569 ms/op 3.5275 ms/op 0.61
altair processAttestation - setStatus - 1/6 committees join 63.038 us/op 103.77 us/op 0.61
altair processAttestation - setStatus - 1/3 committees join 127.01 us/op 175.70 us/op 0.72
altair processAttestation - setStatus - 1/2 committees join 206.33 us/op 245.06 us/op 0.84
altair processAttestation - setStatus - 2/3 committees join 248.52 us/op 325.33 us/op 0.76
altair processAttestation - setStatus - 4/5 committees join 382.94 us/op 479.87 us/op 0.80
altair processAttestation - setStatus - 100% committees join 462.56 us/op 573.26 us/op 0.81
altair processBlock - 250000 vs - 7PWei normalcase 3.5167 ms/op 4.2532 ms/op 0.83
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.025 ms/op 27.265 ms/op 0.92
altair processBlock - 250000 vs - 7PWei worstcase 36.595 ms/op 41.032 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase hashState 82.753 ms/op 93.696 ms/op 0.88
phase0 processBlock - 250000 vs - 7PWei normalcase 1.3536 ms/op 2.6724 ms/op 0.51
phase0 processBlock - 250000 vs - 7PWei worstcase 20.398 ms/op 25.272 ms/op 0.81
altair processEth1Data - 250000 vs - 7PWei normalcase 244.62 us/op 375.11 us/op 0.65
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.8540 us/op 6.8920 us/op 0.70
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 28.827 us/op 37.023 us/op 0.78
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 7.9540 us/op 9.7680 us/op 0.81
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.7830 us/op 6.2710 us/op 0.92
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 128.22 us/op 158.84 us/op 0.81
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 841.83 us/op 1.3477 ms/op 0.62
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.1167 ms/op 1.8789 ms/op 0.59
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.2394 ms/op 1.5759 ms/op 0.79
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.8644 ms/op 3.5937 ms/op 0.80
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.1839 ms/op 2.4625 ms/op 0.48
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.0951 ms/op 3.4929 ms/op 0.89
Tree 40 250000 create 185.45 ms/op 217.23 ms/op 0.85
Tree 40 250000 get(125000) 115.09 ns/op 153.43 ns/op 0.75
Tree 40 250000 set(125000) 568.92 ns/op 700.02 ns/op 0.81
Tree 40 250000 toArray() 10.283 ms/op 20.130 ms/op 0.51
Tree 40 250000 iterate all - toArray() + loop 10.156 ms/op 20.919 ms/op 0.49
Tree 40 250000 iterate all - get(i) 42.092 ms/op 57.789 ms/op 0.73
Array 250000 create 2.3388 ms/op 3.2383 ms/op 0.72
Array 250000 clone - spread 1.3427 ms/op 1.4802 ms/op 0.91
Array 250000 get(125000) 0.58000 ns/op 0.42100 ns/op 1.38
Array 250000 set(125000) 0.59600 ns/op 0.43100 ns/op 1.38
Array 250000 iterate all - loop 77.174 us/op 82.040 us/op 0.94
phase0 afterProcessEpoch - 250000 vs - 7PWei 76.865 ms/op 94.741 ms/op 0.81
Array.fill - length 1000000 2.6866 ms/op 3.8215 ms/op 0.70
Array push - length 1000000 9.5923 ms/op 21.785 ms/op 0.44
Array.get 0.26815 ns/op 0.29227 ns/op 0.92
Uint8Array.get 0.34485 ns/op 0.44766 ns/op 0.77
phase0 beforeProcessEpoch - 250000 vs - 7PWei 13.798 ms/op 17.651 ms/op 0.78
altair processEpoch - mainnet_e81889 261.51 ms/op 359.23 ms/op 0.73
mainnet_e81889 - altair beforeProcessEpoch 17.670 ms/op 18.440 ms/op 0.96
mainnet_e81889 - altair processJustificationAndFinalization 9.9030 us/op 17.955 us/op 0.55
mainnet_e81889 - altair processInactivityUpdates 4.2797 ms/op 5.2782 ms/op 0.81
mainnet_e81889 - altair processRewardsAndPenalties 57.329 ms/op 59.699 ms/op 0.96
mainnet_e81889 - altair processRegistryUpdates 1.0870 us/op 3.0090 us/op 0.36
mainnet_e81889 - altair processSlashings 433.00 ns/op 605.00 ns/op 0.72
mainnet_e81889 - altair processEth1DataReset 408.00 ns/op 379.00 ns/op 1.08
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6660 ms/op 2.6697 ms/op 0.62
mainnet_e81889 - altair processSlashingsReset 1.4770 us/op 2.5420 us/op 0.58
mainnet_e81889 - altair processRandaoMixesReset 2.8330 us/op 3.1160 us/op 0.91
mainnet_e81889 - altair processHistoricalRootsUpdate 727.00 ns/op 832.00 ns/op 0.87
mainnet_e81889 - altair processParticipationFlagUpdates 1.6400 us/op 3.0030 us/op 0.55
mainnet_e81889 - altair processSyncCommitteeUpdates 507.00 ns/op 477.00 ns/op 1.06
mainnet_e81889 - altair afterProcessEpoch 80.908 ms/op 99.297 ms/op 0.81
capella processEpoch - mainnet_e217614 968.16 ms/op 1.2617 s/op 0.77
mainnet_e217614 - capella beforeProcessEpoch 63.312 ms/op 73.985 ms/op 0.86
mainnet_e217614 - capella processJustificationAndFinalization 12.517 us/op 23.602 us/op 0.53
mainnet_e217614 - capella processInactivityUpdates 13.570 ms/op 16.281 ms/op 0.83
mainnet_e217614 - capella processRewardsAndPenalties 248.55 ms/op 280.78 ms/op 0.89
mainnet_e217614 - capella processRegistryUpdates 12.262 us/op 20.223 us/op 0.61
mainnet_e217614 - capella processSlashings 827.00 ns/op 658.00 ns/op 1.26
mainnet_e217614 - capella processEth1DataReset 702.00 ns/op 603.00 ns/op 1.16
mainnet_e217614 - capella processEffectiveBalanceUpdates 12.015 ms/op 12.762 ms/op 0.94
mainnet_e217614 - capella processSlashingsReset 1.6360 us/op 3.1980 us/op 0.51
mainnet_e217614 - capella processRandaoMixesReset 2.8460 us/op 4.3990 us/op 0.65
mainnet_e217614 - capella processHistoricalRootsUpdate 742.00 ns/op 479.00 ns/op 1.55
mainnet_e217614 - capella processParticipationFlagUpdates 4.4890 us/op 3.9390 us/op 1.14
mainnet_e217614 - capella afterProcessEpoch 190.74 ms/op 238.58 ms/op 0.80
phase0 processEpoch - mainnet_e58758 398.65 ms/op 436.80 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 72.736 ms/op 83.891 ms/op 0.87
mainnet_e58758 - phase0 processJustificationAndFinalization 12.676 us/op 22.790 us/op 0.56
mainnet_e58758 - phase0 processRewardsAndPenalties 34.917 ms/op 36.336 ms/op 0.96
mainnet_e58758 - phase0 processRegistryUpdates 6.4610 us/op 12.488 us/op 0.52
mainnet_e58758 - phase0 processSlashings 725.00 ns/op 490.00 ns/op 1.48
mainnet_e58758 - phase0 processEth1DataReset 713.00 ns/op 548.00 ns/op 1.30
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 897.67 us/op 2.9223 ms/op 0.31
mainnet_e58758 - phase0 processSlashingsReset 1.8270 us/op 4.1130 us/op 0.44
mainnet_e58758 - phase0 processRandaoMixesReset 2.2890 us/op 5.7890 us/op 0.40
mainnet_e58758 - phase0 processHistoricalRootsUpdate 402.00 ns/op 469.00 ns/op 0.86
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.5170 us/op 4.3250 us/op 0.58
mainnet_e58758 - phase0 afterProcessEpoch 66.104 ms/op 82.402 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0424 ms/op 1.3079 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4897 ms/op 2.1866 ms/op 0.68
altair processInactivityUpdates - 250000 normalcase 15.292 ms/op 21.012 ms/op 0.73
altair processInactivityUpdates - 250000 worstcase 15.393 ms/op 21.391 ms/op 0.72
phase0 processRegistryUpdates - 250000 normalcase 5.2850 us/op 6.1370 us/op 0.86
phase0 processRegistryUpdates - 250000 badcase_full_deposits 287.61 us/op 341.00 us/op 0.84
phase0 processRegistryUpdates - 250000 worstcase 0.5 99.679 ms/op 122.16 ms/op 0.82
altair processRewardsAndPenalties - 250000 normalcase 33.257 ms/op 48.660 ms/op 0.68
altair processRewardsAndPenalties - 250000 worstcase 43.368 ms/op 56.556 ms/op 0.77
phase0 getAttestationDeltas - 250000 normalcase 6.3679 ms/op 7.1894 ms/op 0.89
phase0 getAttestationDeltas - 250000 worstcase 6.4733 ms/op 7.0507 ms/op 0.92
phase0 processSlashings - 250000 worstcase 80.724 us/op 120.83 us/op 0.67
altair processSyncCommitteeUpdates - 250000 95.742 ms/op 121.16 ms/op 0.79
BeaconState.hashTreeRoot - No change 463.00 ns/op 258.00 ns/op 1.79
BeaconState.hashTreeRoot - 1 full validator 142.13 us/op 147.99 us/op 0.96
BeaconState.hashTreeRoot - 32 full validator 1.4748 ms/op 1.6052 ms/op 0.92
BeaconState.hashTreeRoot - 512 full validator 12.370 ms/op 13.486 ms/op 0.92
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 116.68 us/op 142.53 us/op 0.82
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.9968 ms/op 2.1058 ms/op 0.95
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 27.292 ms/op 29.176 ms/op 0.94
BeaconState.hashTreeRoot - 1 balances 132.76 us/op 133.49 us/op 0.99
BeaconState.hashTreeRoot - 32 balances 1.2835 ms/op 1.1740 ms/op 1.09
BeaconState.hashTreeRoot - 512 balances 10.967 ms/op 11.658 ms/op 0.94
BeaconState.hashTreeRoot - 250000 balances 167.79 ms/op 190.76 ms/op 0.88
aggregationBits - 2048 els - zipIndexesInBitList 20.041 us/op 23.370 us/op 0.86
byteArrayEquals 32 45.449 ns/op 51.519 ns/op 0.88
Buffer.compare 32 14.551 ns/op 16.419 ns/op 0.89
byteArrayEquals 1024 1.1884 us/op 1.5260 us/op 0.78
Buffer.compare 1024 22.642 ns/op 23.599 ns/op 0.96
byteArrayEquals 16384 18.924 us/op 24.250 us/op 0.78
Buffer.compare 16384 165.34 ns/op 190.85 ns/op 0.87
byteArrayEquals 123687377 150.85 ms/op 183.35 ms/op 0.82
Buffer.compare 123687377 4.0879 ms/op 6.1803 ms/op 0.66
byteArrayEquals 32 - diff last byte 47.001 ns/op 51.871 ns/op 0.91
Buffer.compare 32 - diff last byte 16.389 ns/op 16.811 ns/op 0.97
byteArrayEquals 1024 - diff last byte 1.2587 us/op 1.5570 us/op 0.81
Buffer.compare 1024 - diff last byte 23.514 ns/op 25.262 ns/op 0.93
byteArrayEquals 16384 - diff last byte 19.965 us/op 24.810 us/op 0.80
Buffer.compare 16384 - diff last byte 171.82 ns/op 195.81 ns/op 0.88
byteArrayEquals 123687377 - diff last byte 150.17 ms/op 185.48 ms/op 0.81
Buffer.compare 123687377 - diff last byte 4.7654 ms/op 6.1979 ms/op 0.77
byteArrayEquals 32 - random bytes 4.8810 ns/op 5.0080 ns/op 0.97
Buffer.compare 32 - random bytes 16.614 ns/op 16.699 ns/op 0.99
byteArrayEquals 1024 - random bytes 4.8890 ns/op 5.0240 ns/op 0.97
Buffer.compare 1024 - random bytes 16.282 ns/op 16.664 ns/op 0.98
byteArrayEquals 16384 - random bytes 4.9550 ns/op 5.0120 ns/op 0.99
Buffer.compare 16384 - random bytes 16.091 ns/op 17.000 ns/op 0.95
byteArrayEquals 123687377 - random bytes 7.7300 ns/op 6.3100 ns/op 1.23
Buffer.compare 123687377 - random bytes 19.520 ns/op 17.950 ns/op 1.09
regular array get 100000 times 30.784 us/op 41.309 us/op 0.75
wrappedArray get 100000 times 30.710 us/op 32.121 us/op 0.96
arrayWithProxy get 100000 times 10.473 ms/op 14.838 ms/op 0.71
ssz.Root.equals 45.221 ns/op 44.671 ns/op 1.01
byteArrayEquals 39.464 ns/op 43.937 ns/op 0.90
Buffer.compare 9.7350 ns/op 10.101 ns/op 0.96
shuffle list - 16384 els 5.5030 ms/op 5.9155 ms/op 0.93
shuffle list - 250000 els 78.123 ms/op 87.288 ms/op 0.90
processSlot - 1 slots 11.789 us/op 14.754 us/op 0.80
processSlot - 32 slots 3.2312 ms/op 3.4247 ms/op 0.94
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 38.282 ms/op 42.149 ms/op 0.91
getCommitteeAssignments - req 1 vs - 250000 vc 1.7953 ms/op 2.1085 ms/op 0.85
getCommitteeAssignments - req 100 vs - 250000 vc 3.5254 ms/op 4.0632 ms/op 0.87
getCommitteeAssignments - req 1000 vs - 250000 vc 3.8272 ms/op 4.4299 ms/op 0.86
findModifiedValidators - 10000 modified validators 237.15 ms/op 248.60 ms/op 0.95
findModifiedValidators - 1000 modified validators 141.04 ms/op 173.35 ms/op 0.81
findModifiedValidators - 100 modified validators 140.97 ms/op 157.35 ms/op 0.90
findModifiedValidators - 10 modified validators 132.24 ms/op 163.10 ms/op 0.81
findModifiedValidators - 1 modified validators 136.33 ms/op 157.23 ms/op 0.87
findModifiedValidators - no difference 125.89 ms/op 161.34 ms/op 0.78
compare ViewDUs 2.9940 s/op 3.1548 s/op 0.95
compare each validator Uint8Array 1.2044 s/op 1.8921 s/op 0.64
compare ViewDU to Uint8Array 848.11 ms/op 959.79 ms/op 0.88
migrate state 1000000 validators, 24 modified, 0 new 572.07 ms/op 622.81 ms/op 0.92
migrate state 1000000 validators, 1700 modified, 1000 new 855.06 ms/op 876.93 ms/op 0.98
migrate state 1000000 validators, 3400 modified, 2000 new 945.92 ms/op 1.1290 s/op 0.84
migrate state 1500000 validators, 24 modified, 0 new 623.69 ms/op 634.19 ms/op 0.98
migrate state 1500000 validators, 1700 modified, 1000 new 808.27 ms/op 897.46 ms/op 0.90
migrate state 1500000 validators, 3400 modified, 2000 new 984.90 ms/op 1.1186 s/op 0.88
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.7600 ns/op 4.5300 ns/op 1.49
state getBlockRootAtSlot - 250000 vs - 7PWei 718.39 ns/op 686.92 ns/op 1.05
computeProposers - vc 250000 6.3136 ms/op 8.2807 ms/op 0.76
computeEpochShuffling - vc 250000 81.664 ms/op 92.225 ms/op 0.89
getNextSyncCommittee - vc 250000 105.01 ms/op 129.70 ms/op 0.81
computeSigningRoot for AttestationData 20.492 us/op 20.795 us/op 0.99
hash AttestationData serialized data then Buffer.toString(base64) 1.2595 us/op 1.5428 us/op 0.82
toHexString serialized data 812.16 ns/op 944.59 ns/op 0.86
Buffer.toString(base64) 175.57 ns/op 185.19 ns/op 0.95
nodejs block root to RootHex using toHex 126.07 ns/op 146.32 ns/op 0.86
nodejs block root to RootHex using toRootHex 78.881 ns/op 88.568 ns/op 0.89
browser block root to RootHex using the deprecated toHexString 229.48 ns/op 227.77 ns/op 1.01
browser block root to RootHex using toHex 191.87 ns/op 184.91 ns/op 1.04
browser block root to RootHex using toRootHex 155.97 ns/op 171.18 ns/op 0.91

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review August 28, 2024 09:50
@twoeths twoeths requested a review from a team as a code owner August 28, 2024 09:50
@twoeths twoeths force-pushed the te/single_state_tree_at_start_up branch from ba3cd96 to bec9bfc Compare August 29, 2024 01:47
@philknows philknows added this to the v1.22.0 milestone Sep 3, 2024
@twoeths twoeths force-pushed the te/single_state_tree_at_start_up branch from bec9bfc to 067f620 Compare September 6, 2024 01:34
@twoeths twoeths merged commit 6c1e335 into unstable Sep 10, 2024
20 checks passed
@twoeths twoeths deleted the te/single_state_tree_at_start_up branch September 10, 2024 08:11
philknows pushed a commit that referenced this pull request Sep 11, 2024
@twoeths @philknows
fix: single state tree at start up (#7056)
fa67e62 
* feat: use db state to load ws state

* feat: log state size

* fix: rename initStateFromAnchorState to checkAndPersistAnchorState

* fix: only persist anchor state if it's cp state

* fix: avoid redundant anchor state serialization
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.22.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.22.0
Development

Successfully merging this pull request may close these issues.

Use single state tree at startup
3 participants
@twoeths @wemeetagain @philknows