Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: fix flaky sim tests #6163

Merged
merged 9 commits into from
Dec 7, 2023
Merged

test: fix flaky sim tests #6163

merged 9 commits into from
Dec 7, 2023

Conversation

nazarhussain
Copy link
Contributor

Motivation

Make CI more stable.

Description

Fix the failing sim tests by adding grace slots for unknown block sync.

Steps to test or reproduce

Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner December 6, 2023 15:38
@nazarhussain nazarhussain self-assigned this Dec 6, 2023
nflaig
nflaig reviewed Dec 6, 2023
View reviewed changes
packages/beacon-node/src/sync/sync.ts Outdated Show resolved Hide resolved
packages/cli/test/sim/multi_fork.test.ts Outdated Show resolved Hide resolved
packages/cli/src/options/beaconNodeOptions/sync.ts Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 6, 2023

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 6e4547f Previous: f8d3ab0 Ratio
getSlashingsAndExits - default max 416.24 us/op 111.34 us/op 3.74
Full benchmark results
Benchmark suite Current: 6e4547f Previous: f8d3ab0 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 774.66 us/op 990.27 us/op 0.78
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 123.25 us/op 77.431 us/op 1.59
BLS verify - blst-native 1.3125 ms/op 1.0980 ms/op 1.20
BLS verifyMultipleSignatures 3 - blst-native 2.7844 ms/op 2.3539 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 6.1439 ms/op 5.1750 ms/op 1.19
BLS verifyMultipleSignatures 32 - blst-native 22.349 ms/op 19.109 ms/op 1.17
BLS verifyMultipleSignatures 64 - blst-native 44.105 ms/op 37.859 ms/op 1.16
BLS verifyMultipleSignatures 128 - blst-native 87.734 ms/op 74.712 ms/op 1.17
BLS deserializing 10000 signatures 936.51 ms/op 832.24 ms/op 1.13
BLS deserializing 100000 signatures 9.1546 s/op 8.1250 s/op 1.13
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.5202 ms/op 1.0863 ms/op 1.40
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5742 ms/op 1.4128 ms/op 1.11
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4894 ms/op 1.9306 ms/op 1.29
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.9199 ms/op 3.7723 ms/op 1.04
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.0942 ms/op 4.8114 ms/op 1.27
BLS aggregatePubkeys 32 - blst-native 27.780 us/op 21.665 us/op 1.28
BLS aggregatePubkeys 128 - blst-native 106.03 us/op 84.314 us/op 1.26
getAttestationsForBlock 95.030 ms/op 37.316 ms/op 2.55
getSlashingsAndExits - default max 416.24 us/op 111.34 us/op 3.74
getSlashingsAndExits - 2k 379.17 us/op 334.42 us/op 1.13
proposeBlockBody type=full, size=empty 6.1400 ms/op 4.1067 ms/op 1.50
isKnown best case - 1 super set check 383.00 ns/op 324.00 ns/op 1.18
isKnown normal case - 2 super set checks 458.00 ns/op 321.00 ns/op 1.43
isKnown worse case - 16 super set checks 405.00 ns/op 319.00 ns/op 1.27
CheckpointStateCache - add get delete 5.8810 us/op 3.8880 us/op 1.51
validate api signedAggregateAndProof - struct 2.7710 ms/op 2.3566 ms/op 1.18
validate gossip signedAggregateAndProof - struct 2.7907 ms/op 2.3843 ms/op 1.17
validate gossip attestation - vc 640000 1.3468 ms/op 1.1335 ms/op 1.19
batch validate gossip attestation - vc 640000 - chunk 32 164.92 us/op 139.13 us/op 1.19
batch validate gossip attestation - vc 640000 - chunk 64 147.62 us/op 127.15 us/op 1.16
batch validate gossip attestation - vc 640000 - chunk 128 144.56 us/op 119.42 us/op 1.21
batch validate gossip attestation - vc 640000 - chunk 256 138.75 us/op 115.70 us/op 1.20
pickEth1Vote - no votes 1.3068 ms/op 991.97 us/op 1.32
pickEth1Vote - max votes 11.557 ms/op 8.6975 ms/op 1.33
pickEth1Vote - Eth1Data hashTreeRoot value x2048 19.392 ms/op 17.199 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 36.011 ms/op 19.390 ms/op 1.86
pickEth1Vote - Eth1Data fastSerialize value x2048 653.81 us/op 379.25 us/op 1.72
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.8630 ms/op 5.4900 ms/op 1.61
bytes32 toHexString 675.00 ns/op 459.00 ns/op 1.47
bytes32 Buffer.toString(hex) 304.00 ns/op 341.00 ns/op 0.89
bytes32 Buffer.toString(hex) from Uint8Array 504.00 ns/op 424.00 ns/op 1.19
bytes32 Buffer.toString(hex) + 0x 305.00 ns/op 329.00 ns/op 0.93
Object access 1 prop 0.19800 ns/op 0.19400 ns/op 1.02
Map access 1 prop 0.15100 ns/op 0.19500 ns/op 0.77
Object get x1000 7.0970 ns/op 5.4840 ns/op 1.29
Map get x1000 0.83700 ns/op 0.70200 ns/op 1.19
Object set x1000 66.719 ns/op 25.744 ns/op 2.59
Map set x1000 46.496 ns/op 17.200 ns/op 2.70
Return object 10000 times 0.25880 ns/op 0.22500 ns/op 1.15
Throw Error 10000 times 4.0728 us/op 2.7343 us/op 1.49
fastMsgIdFn sha256 / 200 bytes 3.3530 us/op 2.0090 us/op 1.67
fastMsgIdFn h32 xxhash / 200 bytes 336.00 ns/op 330.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 200 bytes 380.00 ns/op 367.00 ns/op 1.04
fastMsgIdFn sha256 / 1000 bytes 11.587 us/op 6.1660 us/op 1.88
fastMsgIdFn h32 xxhash / 1000 bytes 470.00 ns/op 404.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 1000 bytes 435.00 ns/op 411.00 ns/op 1.06
fastMsgIdFn sha256 / 10000 bytes 103.11 us/op 52.402 us/op 1.97
fastMsgIdFn h32 xxhash / 10000 bytes 2.0100 us/op 1.8070 us/op 1.11
fastMsgIdFn h64 xxhash / 10000 bytes 1.4150 us/op 1.2810 us/op 1.10
send data - 1000 256B messages 22.013 ms/op 12.848 ms/op 1.71
send data - 1000 512B messages 28.933 ms/op 17.976 ms/op 1.61
send data - 1000 1024B messages 41.554 ms/op 22.755 ms/op 1.83
send data - 1000 1200B messages 40.956 ms/op 29.619 ms/op 1.38
send data - 1000 2048B messages 45.736 ms/op 34.349 ms/op 1.33
send data - 1000 4096B messages 54.320 ms/op 31.940 ms/op 1.70
send data - 1000 16384B messages 118.28 ms/op 103.66 ms/op 1.14
send data - 1000 65536B messages 476.21 ms/op 370.08 ms/op 1.29
enrSubnets - fastDeserialize 64 bits 1.5390 us/op 1.0320 us/op 1.49
enrSubnets - ssz BitVector 64 bits 528.00 ns/op 528.00 ns/op 1.00
enrSubnets - fastDeserialize 4 bits 228.00 ns/op 245.00 ns/op 0.93
enrSubnets - ssz BitVector 4 bits 511.00 ns/op 422.00 ns/op 1.21
prioritizePeers score -10:0 att 32-0.1 sync 2-0 122.29 us/op 65.592 us/op 1.86
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 148.55 us/op 78.263 us/op 1.90
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 196.76 us/op 115.28 us/op 1.71
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 336.06 us/op 182.54 us/op 1.84
prioritizePeers score 0:0 att 64-1 sync 4-1 390.20 us/op 201.10 us/op 1.94
array of 16000 items push then shift 1.6710 us/op 1.2874 us/op 1.30
LinkedList of 16000 items push then shift 9.2430 ns/op 5.9560 ns/op 1.55
array of 16000 items push then pop 110.93 ns/op 106.78 ns/op 1.04
LinkedList of 16000 items push then pop 9.0180 ns/op 5.9150 ns/op 1.52
array of 24000 items push then shift 2.4707 us/op 1.8422 us/op 1.34
LinkedList of 24000 items push then shift 9.4710 ns/op 6.0910 ns/op 1.55
array of 24000 items push then pop 148.21 ns/op 130.71 ns/op 1.13
LinkedList of 24000 items push then pop 9.6370 ns/op 6.1540 ns/op 1.57
intersect bitArray bitLen 8 6.8570 ns/op 5.1680 ns/op 1.33
intersect array and set length 8 79.684 ns/op 53.200 ns/op 1.50
intersect bitArray bitLen 128 35.842 ns/op 27.672 ns/op 1.30
intersect array and set length 128 936.52 ns/op 679.93 ns/op 1.38
bitArray.getTrueBitIndexes() bitLen 128 1.5790 us/op 1.1530 us/op 1.37
bitArray.getTrueBitIndexes() bitLen 248 2.8950 us/op 2.2610 us/op 1.28
bitArray.getTrueBitIndexes() bitLen 512 5.8850 us/op 4.8440 us/op 1.21
Buffer.concat 32 items 966.00 ns/op 923.00 ns/op 1.05
Uint8Array.set 32 items 2.2750 us/op 1.9070 us/op 1.19
Set add up to 64 items then delete first 4.4613 us/op 1.9853 us/op 2.25
OrderedSet add up to 64 items then delete first 5.9768 us/op 2.8205 us/op 2.12
Set add up to 64 items then delete last 5.5986 us/op 1.9869 us/op 2.82
OrderedSet add up to 64 items then delete last 7.1111 us/op 3.3322 us/op 2.13
Set add up to 64 items then delete middle 5.3176 us/op 1.9811 us/op 2.68
OrderedSet add up to 64 items then delete middle 8.3607 us/op 4.3721 us/op 1.91
Set add up to 128 items then delete first 10.215 us/op 4.0473 us/op 2.52
OrderedSet add up to 128 items then delete first 13.300 us/op 6.2263 us/op 2.14
Set add up to 128 items then delete last 10.891 us/op 3.9393 us/op 2.76
OrderedSet add up to 128 items then delete last 13.787 us/op 5.9193 us/op 2.33
Set add up to 128 items then delete middle 10.268 us/op 3.6730 us/op 2.80
OrderedSet add up to 128 items then delete middle 19.697 us/op 11.468 us/op 1.72
Set add up to 256 items then delete first 21.504 us/op 8.3567 us/op 2.57
OrderedSet add up to 256 items then delete first 30.710 us/op 12.260 us/op 2.50
Set add up to 256 items then delete last 21.585 us/op 7.3550 us/op 2.93
OrderedSet add up to 256 items then delete last 27.535 us/op 12.179 us/op 2.26
Set add up to 256 items then delete middle 20.494 us/op 7.2950 us/op 2.81
OrderedSet add up to 256 items then delete middle 49.707 us/op 32.351 us/op 1.54
transfer serialized Status (84 B) 1.7560 us/op 1.6960 us/op 1.04
copy serialized Status (84 B) 1.5810 us/op 1.4410 us/op 1.10
transfer serialized SignedVoluntaryExit (112 B) 1.9300 us/op 1.5580 us/op 1.24
copy serialized SignedVoluntaryExit (112 B) 1.6360 us/op 1.4030 us/op 1.17
transfer serialized ProposerSlashing (416 B) 2.4770 us/op 2.1860 us/op 1.13
copy serialized ProposerSlashing (416 B) 2.4420 us/op 2.2460 us/op 1.09
transfer serialized Attestation (485 B) 2.4810 us/op 2.1990 us/op 1.13
copy serialized Attestation (485 B) 2.4160 us/op 2.0860 us/op 1.16
transfer serialized AttesterSlashing (33232 B) 2.6720 us/op 1.9020 us/op 1.40
copy serialized AttesterSlashing (33232 B) 8.8160 us/op 4.6060 us/op 1.91
transfer serialized Small SignedBeaconBlock (128000 B) 3.0290 us/op 2.5440 us/op 1.19
copy serialized Small SignedBeaconBlock (128000 B) 19.586 us/op 9.5800 us/op 2.04
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6450 us/op 2.2020 us/op 1.66
copy serialized Avg SignedBeaconBlock (200000 B) 31.891 us/op 22.919 us/op 1.39
transfer serialized BlobsSidecar (524380 B) 3.2630 us/op 3.0830 us/op 1.06
copy serialized BlobsSidecar (524380 B) 141.21 us/op 76.663 us/op 1.84
transfer serialized Big SignedBeaconBlock (1000000 B) 4.1440 us/op 2.8970 us/op 1.43
copy serialized Big SignedBeaconBlock (1000000 B) 221.76 us/op 168.19 us/op 1.32
pass gossip attestations to forkchoice per slot 4.6795 ms/op 2.9731 ms/op 1.57
forkChoice updateHead vc 100000 bc 64 eq 0 745.34 us/op 482.38 us/op 1.55
forkChoice updateHead vc 600000 bc 64 eq 0 5.7408 ms/op 3.8260 ms/op 1.50
forkChoice updateHead vc 1000000 bc 64 eq 0 7.8224 ms/op 4.6642 ms/op 1.68
forkChoice updateHead vc 600000 bc 320 eq 0 4.3525 ms/op 2.8592 ms/op 1.52
forkChoice updateHead vc 600000 bc 1200 eq 0 4.7048 ms/op 3.0914 ms/op 1.52
forkChoice updateHead vc 600000 bc 7200 eq 0 5.4932 ms/op 3.7224 ms/op 1.48
forkChoice updateHead vc 600000 bc 64 eq 1000 11.364 ms/op 10.276 ms/op 1.11
forkChoice updateHead vc 600000 bc 64 eq 10000 12.115 ms/op 10.011 ms/op 1.21
forkChoice updateHead vc 600000 bc 64 eq 300000 26.401 ms/op 14.607 ms/op 1.81
computeDeltas 500000 validators 300 proto nodes 7.2557 ms/op 3.2500 ms/op 2.23
computeDeltas 500000 validators 1200 proto nodes 6.8433 ms/op 3.0829 ms/op 2.22
computeDeltas 500000 validators 7200 proto nodes 6.7030 ms/op 3.2258 ms/op 2.08
computeDeltas 750000 validators 300 proto nodes 9.8902 ms/op 4.8535 ms/op 2.04
computeDeltas 750000 validators 1200 proto nodes 10.145 ms/op 4.9219 ms/op 2.06
computeDeltas 750000 validators 7200 proto nodes 10.016 ms/op 4.6823 ms/op 2.14
computeDeltas 1400000 validators 300 proto nodes 19.568 ms/op 9.3481 ms/op 2.09
computeDeltas 1400000 validators 1200 proto nodes 20.351 ms/op 9.1057 ms/op 2.23
computeDeltas 1400000 validators 7200 proto nodes 20.132 ms/op 8.7210 ms/op 2.31
computeDeltas 2100000 validators 300 proto nodes 31.076 ms/op 13.957 ms/op 2.23
computeDeltas 2100000 validators 1200 proto nodes 30.435 ms/op 13.699 ms/op 2.22
computeDeltas 2100000 validators 7200 proto nodes 30.559 ms/op 14.078 ms/op 2.17
computeProposerBoostScoreFromBalances 500000 validators 3.9457 ms/op 3.2951 ms/op 1.20
computeProposerBoostScoreFromBalances 750000 validators 3.8193 ms/op 3.3428 ms/op 1.14
computeProposerBoostScoreFromBalances 1400000 validators 3.7984 ms/op 3.8623 ms/op 0.98
computeProposerBoostScoreFromBalances 2100000 validators 3.7705 ms/op 3.7005 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 3.7429 ms/op 3.0162 ms/op 1.24
altair processAttestation - 250000 vs - 7PWei worstcase 4.5501 ms/op 2.6168 ms/op 1.74
altair processAttestation - setStatus - 1/6 committees join 163.50 us/op 77.270 us/op 2.12
altair processAttestation - setStatus - 1/3 committees join 297.36 us/op 150.13 us/op 1.98
altair processAttestation - setStatus - 1/2 committees join 388.63 us/op 234.20 us/op 1.66
altair processAttestation - setStatus - 2/3 committees join 504.41 us/op 264.10 us/op 1.91
altair processAttestation - setStatus - 4/5 committees join 698.18 us/op 425.99 us/op 1.64
altair processAttestation - setStatus - 100% committees join 839.99 us/op 464.56 us/op 1.81
altair processBlock - 250000 vs - 7PWei normalcase 10.686 ms/op 9.8948 ms/op 1.08
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.330 ms/op 26.732 ms/op 1.40
altair processBlock - 250000 vs - 7PWei worstcase 40.978 ms/op 30.771 ms/op 1.33
altair processBlock - 250000 vs - 7PWei worstcase hashState 103.48 ms/op 78.043 ms/op 1.33
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1595 ms/op 3.0785 ms/op 1.03
phase0 processBlock - 250000 vs - 7PWei worstcase 33.561 ms/op 28.083 ms/op 1.20
altair processEth1Data - 250000 vs - 7PWei normalcase 733.41 us/op 576.18 us/op 1.27
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 17.558 us/op 9.9570 us/op 1.76
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 63.749 us/op 53.105 us/op 1.20
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 24.850 us/op 15.054 us/op 1.65
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 17.457 us/op 10.535 us/op 1.66
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 211.72 us/op 120.39 us/op 1.76
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 2.0259 ms/op 809.85 us/op 2.50
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.0231 ms/op 1.2314 ms/op 1.64
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.9257 ms/op 1.3026 ms/op 1.48
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.9781 ms/op 2.3124 ms/op 1.72
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.7944 ms/op 1.4588 ms/op 1.92
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.4910 ms/op 3.3705 ms/op 1.63
Tree 40 250000 create 421.95 ms/op 246.31 ms/op 1.71
Tree 40 250000 get(125000) 209.34 ns/op 102.08 ns/op 2.05
Tree 40 250000 set(125000) 1.5576 us/op 786.37 ns/op 1.98
Tree 40 250000 toArray() 25.102 ms/op 16.241 ms/op 1.55
Tree 40 250000 iterate all - toArray() + loop 24.038 ms/op 21.057 ms/op 1.14
Tree 40 250000 iterate all - get(i) 79.828 ms/op 42.775 ms/op 1.87
MutableVector 250000 create 19.017 ms/op 12.499 ms/op 1.52
MutableVector 250000 get(125000) 6.7380 ns/op 5.6040 ns/op 1.20
MutableVector 250000 set(125000) 315.12 ns/op 216.68 ns/op 1.45
MutableVector 250000 toArray() 4.0702 ms/op 2.6842 ms/op 1.52
MutableVector 250000 iterate all - toArray() + loop 4.2912 ms/op 3.3447 ms/op 1.28
MutableVector 250000 iterate all - get(i) 1.5284 ms/op 1.3334 ms/op 1.15
Array 250000 create 4.1393 ms/op 3.8844 ms/op 1.07
Array 250000 clone - spread 1.2462 ms/op 1.1231 ms/op 1.11
Array 250000 get(125000) 1.0790 ns/op 1.0060 ns/op 1.07
Array 250000 set(125000) 4.5950 ns/op 1.2190 ns/op 3.77
Array 250000 iterate all - loop 173.29 us/op 157.22 us/op 1.10
effectiveBalanceIncrements clone Uint8Array 300000 38.575 us/op 12.680 us/op 3.04
effectiveBalanceIncrements clone MutableVector 300000 374.00 ns/op 373.00 ns/op 1.00
effectiveBalanceIncrements rw all Uint8Array 300000 200.81 us/op 183.63 us/op 1.09
effectiveBalanceIncrements rw all MutableVector 300000 100.83 ms/op 61.517 ms/op 1.64
phase0 afterProcessEpoch - 250000 vs - 7PWei 119.81 ms/op 75.384 ms/op 1.59
phase0 beforeProcessEpoch - 250000 vs - 7PWei 62.008 ms/op 46.951 ms/op 1.32
altair processEpoch - mainnet_e81889 530.88 ms/op 391.82 ms/op 1.35
mainnet_e81889 - altair beforeProcessEpoch 90.410 ms/op 69.203 ms/op 1.31
mainnet_e81889 - altair processJustificationAndFinalization 22.387 us/op 16.123 us/op 1.39
mainnet_e81889 - altair processInactivityUpdates 7.7915 ms/op 5.1859 ms/op 1.50
mainnet_e81889 - altair processRewardsAndPenalties 71.942 ms/op 51.928 ms/op 1.39
mainnet_e81889 - altair processRegistryUpdates 3.7400 us/op 3.8180 us/op 0.98
mainnet_e81889 - altair processSlashings 648.00 ns/op 650.00 ns/op 1.00
mainnet_e81889 - altair processEth1DataReset 948.00 ns/op 973.00 ns/op 0.97
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.7351 ms/op 1.5984 ms/op 1.09
mainnet_e81889 - altair processSlashingsReset 4.3570 us/op 4.9130 us/op 0.89
mainnet_e81889 - altair processRandaoMixesReset 5.0250 us/op 8.7350 us/op 0.58
mainnet_e81889 - altair processHistoricalRootsUpdate 862.00 ns/op 1.3740 us/op 0.63
mainnet_e81889 - altair processParticipationFlagUpdates 1.7790 us/op 4.5730 us/op 0.39
mainnet_e81889 - altair processSyncCommitteeUpdates 723.00 ns/op 1.4660 us/op 0.49
mainnet_e81889 - altair afterProcessEpoch 123.77 ms/op 87.640 ms/op 1.41
capella processEpoch - mainnet_e217614 2.9643 s/op 2.1614 s/op 1.37
mainnet_e217614 - capella beforeProcessEpoch 678.10 ms/op 493.99 ms/op 1.37
mainnet_e217614 - capella processJustificationAndFinalization 30.399 us/op 14.860 us/op 2.05
mainnet_e217614 - capella processInactivityUpdates 36.167 ms/op 23.602 ms/op 1.53
mainnet_e217614 - capella processRewardsAndPenalties 520.06 ms/op 440.77 ms/op 1.18
mainnet_e217614 - capella processRegistryUpdates 38.889 us/op 34.351 us/op 1.13
mainnet_e217614 - capella processSlashings 936.00 ns/op 1.1020 us/op 0.85
mainnet_e217614 - capella processEth1DataReset 649.00 ns/op 993.00 ns/op 0.65
mainnet_e217614 - capella processEffectiveBalanceUpdates 7.9306 ms/op 4.1363 ms/op 1.92
mainnet_e217614 - capella processSlashingsReset 6.2100 us/op 3.1380 us/op 1.98
mainnet_e217614 - capella processRandaoMixesReset 6.3120 us/op 5.6200 us/op 1.12
mainnet_e217614 - capella processHistoricalRootsUpdate 647.00 ns/op 936.00 ns/op 0.69
mainnet_e217614 - capella processParticipationFlagUpdates 2.1770 us/op 2.3730 us/op 0.92
mainnet_e217614 - capella afterProcessEpoch 334.19 ms/op 227.04 ms/op 1.47
phase0 processEpoch - mainnet_e58758 481.37 ms/op 441.08 ms/op 1.09
mainnet_e58758 - phase0 beforeProcessEpoch 139.68 ms/op 137.80 ms/op 1.01
mainnet_e58758 - phase0 processJustificationAndFinalization 17.167 us/op 18.932 us/op 0.91
mainnet_e58758 - phase0 processRewardsAndPenalties 62.793 ms/op 56.213 ms/op 1.12
mainnet_e58758 - phase0 processRegistryUpdates 14.587 us/op 13.012 us/op 1.12
mainnet_e58758 - phase0 processSlashings 682.00 ns/op 898.00 ns/op 0.76
mainnet_e58758 - phase0 processEth1DataReset 906.00 ns/op 996.00 ns/op 0.91
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.5138 ms/op 1.0087 ms/op 1.50
mainnet_e58758 - phase0 processSlashingsReset 5.8190 us/op 4.0270 us/op 1.44
mainnet_e58758 - phase0 processRandaoMixesReset 8.9230 us/op 7.2240 us/op 1.24
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4320 us/op 1.1750 us/op 1.22
mainnet_e58758 - phase0 processParticipationRecordUpdates 9.1310 us/op 5.7060 us/op 1.60
mainnet_e58758 - phase0 afterProcessEpoch 106.86 ms/op 68.617 ms/op 1.56
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.7514 ms/op 1.0612 ms/op 1.65
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9191 ms/op 1.2506 ms/op 1.53
altair processInactivityUpdates - 250000 normalcase 33.717 ms/op 25.021 ms/op 1.35
altair processInactivityUpdates - 250000 worstcase 29.551 ms/op 25.922 ms/op 1.14
phase0 processRegistryUpdates - 250000 normalcase 14.513 us/op 12.224 us/op 1.19
phase0 processRegistryUpdates - 250000 badcase_full_deposits 451.76 us/op 440.09 us/op 1.03
phase0 processRegistryUpdates - 250000 worstcase 0.5 161.02 ms/op 127.60 ms/op 1.26
altair processRewardsAndPenalties - 250000 normalcase 75.614 ms/op 63.116 ms/op 1.20
altair processRewardsAndPenalties - 250000 worstcase 65.002 ms/op 56.359 ms/op 1.15
phase0 getAttestationDeltas - 250000 normalcase 12.697 ms/op 7.6533 ms/op 1.66
phase0 getAttestationDeltas - 250000 worstcase 12.597 ms/op 5.2947 ms/op 2.38
phase0 processSlashings - 250000 worstcase 129.40 us/op 85.527 us/op 1.51
altair processSyncCommitteeUpdates - 250000 170.61 ms/op 125.29 ms/op 1.36
BeaconState.hashTreeRoot - No change 260.00 ns/op 374.00 ns/op 0.70
BeaconState.hashTreeRoot - 1 full validator 143.19 us/op 171.14 us/op 0.84
BeaconState.hashTreeRoot - 32 full validator 1.6405 ms/op 1.0850 ms/op 1.51
BeaconState.hashTreeRoot - 512 full validator 20.073 ms/op 13.757 ms/op 1.46
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 236.64 us/op 166.25 us/op 1.42
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 3.3766 ms/op 2.0386 ms/op 1.66
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 37.928 ms/op 25.438 ms/op 1.49
BeaconState.hashTreeRoot - 1 balances 160.73 us/op 155.80 us/op 1.03
BeaconState.hashTreeRoot - 32 balances 1.4177 ms/op 1.0906 ms/op 1.30
BeaconState.hashTreeRoot - 512 balances 13.509 ms/op 10.347 ms/op 1.31
BeaconState.hashTreeRoot - 250000 balances 233.97 ms/op 197.90 ms/op 1.18
aggregationBits - 2048 els - zipIndexesInBitList 25.723 us/op 16.484 us/op 1.56
byteArrayEquals 32 75.855 ns/op 62.734 ns/op 1.21
Buffer.compare 32 56.057 ns/op 35.799 ns/op 1.57
byteArrayEquals 1024 2.0819 us/op 1.7265 us/op 1.21
Buffer.compare 1024 71.794 ns/op 42.794 ns/op 1.68
byteArrayEquals 16384 33.054 us/op 27.252 us/op 1.21
Buffer.compare 16384 276.15 ns/op 204.07 ns/op 1.35
byteArrayEquals 123687377 245.97 ms/op 212.39 ms/op 1.16
Buffer.compare 123687377 7.1597 ms/op 5.8759 ms/op 1.22
byteArrayEquals 32 - diff last byte 74.388 ns/op 59.488 ns/op 1.25
Buffer.compare 32 - diff last byte 58.829 ns/op 35.962 ns/op 1.64
byteArrayEquals 1024 - diff last byte 2.0304 us/op 1.6582 us/op 1.22
Buffer.compare 1024 - diff last byte 71.806 ns/op 43.098 ns/op 1.67
byteArrayEquals 16384 - diff last byte 32.393 us/op 26.634 us/op 1.22
Buffer.compare 16384 - diff last byte 260.33 ns/op 216.24 ns/op 1.20
byteArrayEquals 123687377 - diff last byte 245.58 ms/op 212.73 ms/op 1.15
Buffer.compare 123687377 - diff last byte 6.9980 ms/op 6.8564 ms/op 1.02
byteArrayEquals 32 - random bytes 9.0550 ns/op 4.9680 ns/op 1.82
Buffer.compare 32 - random bytes 64.621 ns/op 41.468 ns/op 1.56
byteArrayEquals 1024 - random bytes 6.1990 ns/op 5.0820 ns/op 1.22
Buffer.compare 1024 - random bytes 63.348 ns/op 40.540 ns/op 1.56
byteArrayEquals 16384 - random bytes 5.4560 ns/op 4.9630 ns/op 1.10
Buffer.compare 16384 - random bytes 60.122 ns/op 41.602 ns/op 1.45
byteArrayEquals 123687377 - random bytes 8.9100 ns/op 8.4200 ns/op 1.06
Buffer.compare 123687377 - random bytes 63.750 ns/op 46.280 ns/op 1.38
regular array get 100000 times 47.222 us/op 43.793 us/op 1.08
wrappedArray get 100000 times 45.506 us/op 43.401 us/op 1.05
arrayWithProxy get 100000 times 14.597 ms/op 9.6935 ms/op 1.51
ssz.Root.equals 53.169 ns/op 55.385 ns/op 0.96
byteArrayEquals 53.125 ns/op 55.257 ns/op 0.96
Buffer.compare 11.068 ns/op 10.583 ns/op 1.05
shuffle list - 16384 els 7.1912 ms/op 4.6993 ms/op 1.53
shuffle list - 250000 els 103.78 ms/op 68.436 ms/op 1.52
processSlot - 1 slots 17.416 us/op 17.749 us/op 0.98
processSlot - 32 slots 4.0325 ms/op 3.6004 ms/op 1.12
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 63.692 ms/op 49.852 ms/op 1.28
getCommitteeAssignments - req 1 vs - 250000 vc 2.5734 ms/op 2.2081 ms/op 1.17
getCommitteeAssignments - req 100 vs - 250000 vc 3.8555 ms/op 3.3590 ms/op 1.15
getCommitteeAssignments - req 1000 vs - 250000 vc 4.2327 ms/op 3.6741 ms/op 1.15
findModifiedValidators - 10000 modified validators 624.63 ms/op 439.31 ms/op 1.42
findModifiedValidators - 1000 modified validators 461.51 ms/op 304.03 ms/op 1.52
findModifiedValidators - 100 modified validators 464.49 ms/op 323.55 ms/op 1.44
findModifiedValidators - 10 modified validators 458.85 ms/op 300.57 ms/op 1.53
findModifiedValidators - 1 modified validators 458.91 ms/op 311.82 ms/op 1.47
findModifiedValidators - no difference 400.42 ms/op 312.08 ms/op 1.28
compare ViewDUs 5.1790 s/op 3.9075 s/op 1.33
compare each validator Uint8Array 1.8830 s/op 1.5525 s/op 1.21
compare ViewDU to Uint8Array 1.4788 s/op 1.1262 s/op 1.31
migrate state 1000000 validators, 24 modified, 0 new 954.97 ms/op 769.11 ms/op 1.24
migrate state 1000000 validators, 1700 modified, 1000 new 1.1346 s/op 983.27 ms/op 1.15
migrate state 1000000 validators, 3400 modified, 2000 new 1.3715 s/op 1.3217 s/op 1.04
migrate state 1500000 validators, 24 modified, 0 new 791.38 ms/op 726.33 ms/op 1.09
migrate state 1500000 validators, 1700 modified, 1000 new 1.0663 s/op 910.93 ms/op 1.17
migrate state 1500000 validators, 3400 modified, 2000 new 1.3231 s/op 1.2965 s/op 1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.2300 ns/op 5.0800 ns/op 0.83
state getBlockRootAtSlot - 250000 vs - 7PWei 765.37 ns/op 980.00 ns/op 0.78
computeProposers - vc 250000 9.2360 ms/op 7.3237 ms/op 1.26
computeEpochShuffling - vc 250000 101.67 ms/op 72.183 ms/op 1.41
getNextSyncCommittee - vc 250000 151.82 ms/op 148.65 ms/op 1.02
computeSigningRoot for AttestationData 22.862 us/op 23.650 us/op 0.97
hash AttestationData serialized data then Buffer.toString(base64) 2.2578 us/op 1.2884 us/op 1.75
toHexString serialized data 1.0589 us/op 825.25 ns/op 1.28
Buffer.toString(base64) 213.32 ns/op 153.54 ns/op 1.39

by benchmarkbot/action

@nazarhussain nazarhussain requested a review from nflaig December 7, 2023 10:04
@nazarhussain nazarhussain requested a review from nflaig December 7, 2023 11:01
nflaig
nflaig approved these changes Dec 7, 2023
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks for removing all unrelated changes

@nazarhussain nazarhussain merged commit 6154029 into unstable Dec 7, 2023
14 of 15 checks passed
@nazarhussain nazarhussain deleted the nh/fix-sim-tests branch December 7, 2023 13:32
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.13.0 🎉

@nflaig nflaig mentioned this pull request Jan 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nazarhussain @wemeetagain @nflaig