Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump spec tests to v1.1.10 #3819

Merged
merged 1 commit into from
Mar 2, 2022
Merged

Bump spec tests to v1.1.10 #3819

merged 1 commit into from
Mar 2, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Mar 2, 2022

Motivation

Description

Closes #issue_number

Steps to test or reproduce

@codecov
Copy link

codecov bot commented Mar 2, 2022

Codecov Report

Merging #3819 (a9321d8) into master (bbe818d) will not change coverage.
The diff coverage is n/a.

❗ Current head a9321d8 differs from pull request most recent head 668c5cf. Consider uploading reports for the commit 668c5cf to get more accurate results

@@           Coverage Diff           @@
##           master    #3819   +/-   ##
=======================================
  Coverage   36.43%   36.43%           
=======================================
  Files         325      325           
  Lines        8984     8984           
  Branches     1402     1402           
=======================================
  Hits         3273     3273           
  Misses       5568     5568           
  Partials      143      143           

@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a9321d8 Previous: bbe818d Ratio
BeaconState.hashTreeRoot - No change 680.00 ns/op 725.00 ns/op 0.94
BeaconState.hashTreeRoot - 1 full validator 99.068 us/op 117.60 us/op 0.84
BeaconState.hashTreeRoot - 32 full validator 1.4522 ms/op 1.6688 ms/op 0.87
BeaconState.hashTreeRoot - 512 full validator 19.715 ms/op 21.287 ms/op 0.93
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 100.22 us/op 112.67 us/op 0.89
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.9865 ms/op 2.1887 ms/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 23.127 ms/op 25.301 ms/op 0.91
BeaconState.hashTreeRoot - 1 balances 68.842 us/op 82.620 us/op 0.83
BeaconState.hashTreeRoot - 32 balances 559.94 us/op 719.41 us/op 0.78
BeaconState.hashTreeRoot - 512 balances 5.9329 ms/op 6.7091 ms/op 0.88
BeaconState.hashTreeRoot - 250000 balances 108.16 ms/op 128.22 ms/op 0.84
processSlot - 1 slots 50.451 us/op 66.036 us/op 0.76
processSlot - 32 slots 2.5823 ms/op 3.0951 ms/op 0.83
getCommitteeAssignments - req 1 vs - 250000 vc 6.4940 ms/op 5.9152 ms/op 1.10
getCommitteeAssignments - req 100 vs - 250000 vc 9.0527 ms/op 8.2074 ms/op 1.10
getCommitteeAssignments - req 1000 vs - 250000 vc 9.6282 ms/op 8.8276 ms/op 1.09
computeProposers - vc 250000 25.809 ms/op 27.839 ms/op 0.93
computeEpochShuffling - vc 250000 225.53 ms/op 222.15 ms/op 1.02
getNextSyncCommittee - vc 250000 421.53 ms/op 453.24 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei normalcase 35.143 ms/op 45.258 ms/op 0.78
altair processAttestation - 250000 vs - 7PWei worstcase 41.916 ms/op 41.864 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 14.847 ms/op 16.961 ms/op 0.88
altair processAttestation - setStatus - 1/3 committees join 30.650 ms/op 33.721 ms/op 0.91
altair processAttestation - setStatus - 1/2 committees join 45.073 ms/op 49.721 ms/op 0.91
altair processAttestation - setStatus - 2/3 committees join 59.514 ms/op 67.156 ms/op 0.89
altair processAttestation - setStatus - 4/5 committees join 71.259 ms/op 81.753 ms/op 0.87
altair processAttestation - setStatus - 100% committees join 92.238 ms/op 99.224 ms/op 0.93
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.591 ms/op 17.294 ms/op 0.90
altair processAttestation - updateEpochParticipants - 1/3 committees join 32.015 ms/op 36.558 ms/op 0.88
altair processAttestation - updateEpochParticipants - 1/2 committees join 25.412 ms/op 24.698 ms/op 1.03
altair processAttestation - updateEpochParticipants - 2/3 committees join 29.628 ms/op 30.116 ms/op 0.98
altair processAttestation - updateEpochParticipants - 4/5 committees join 27.038 ms/op 26.547 ms/op 1.02
altair processAttestation - updateEpochParticipants - 100% committees join 28.423 ms/op 27.707 ms/op 1.03
altair processAttestation - updateAllStatus 23.714 ms/op 22.614 ms/op 1.05
altair processBlock - 250000 vs - 7PWei normalcase 39.922 ms/op 46.043 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase 124.12 ms/op 138.56 ms/op 0.90
altair processEpoch - mainnet_e81889 930.81 ms/op 961.40 ms/op 0.97
mainnet_e81889 - altair beforeProcessEpoch 371.58 ms/op 356.76 ms/op 1.04
mainnet_e81889 - altair processJustificationAndFinalization 69.491 us/op 128.07 us/op 0.54
mainnet_e81889 - altair processInactivityUpdates 21.406 ms/op 20.289 ms/op 1.06
mainnet_e81889 - altair processRewardsAndPenalties 141.93 ms/op 170.53 ms/op 0.83
mainnet_e81889 - altair processRegistryUpdates 9.5760 us/op 19.766 us/op 0.48
mainnet_e81889 - altair processSlashings 3.7470 us/op 5.6490 us/op 0.66
mainnet_e81889 - altair processEth1DataReset 3.2610 us/op 6.1400 us/op 0.53
mainnet_e81889 - altair processEffectiveBalanceUpdates 7.5178 ms/op 7.4127 ms/op 1.01
mainnet_e81889 - altair processSlashingsReset 19.176 us/op 33.919 us/op 0.57
mainnet_e81889 - altair processRandaoMixesReset 20.854 us/op 41.906 us/op 0.50
mainnet_e81889 - altair processHistoricalRootsUpdate 4.2870 us/op 7.9970 us/op 0.54
mainnet_e81889 - altair processParticipationFlagUpdates 88.720 ms/op 74.229 ms/op 1.20
mainnet_e81889 - altair processSyncCommitteeUpdates 3.1590 us/op 5.2700 us/op 0.60
mainnet_e81889 - altair afterProcessEpoch 264.07 ms/op 257.64 ms/op 1.02
altair processInactivityUpdates - 250000 normalcase 85.664 ms/op 102.40 ms/op 0.84
altair processInactivityUpdates - 250000 worstcase 85.252 ms/op 90.684 ms/op 0.94
altair processParticipationFlagUpdates - 250000 anycase 72.017 ms/op 69.937 ms/op 1.03
altair processRewardsAndPenalties - 250000 normalcase 98.348 ms/op 106.76 ms/op 0.92
altair processRewardsAndPenalties - 250000 worstcase 121.44 ms/op 97.887 ms/op 1.24
altair processSyncCommitteeUpdates - 250000 432.79 ms/op 467.25 ms/op 0.93
Tree 40 250000 create 820.87 ms/op 932.64 ms/op 0.88
Tree 40 250000 get(125000) 391.46 ns/op 355.87 ns/op 1.10
Tree 40 250000 set(125000) 2.6435 us/op 3.1404 us/op 0.84
Tree 40 250000 toArray() 47.466 ms/op 48.566 ms/op 0.98
Tree 40 250000 iterate all - toArray() + loop 47.358 ms/op 48.617 ms/op 0.97
Tree 40 250000 iterate all - get(i) 142.71 ms/op 140.40 ms/op 1.02
MutableVector 250000 create 26.809 ms/op 23.404 ms/op 1.15
MutableVector 250000 get(125000) 14.980 ns/op 16.331 ns/op 0.92
MutableVector 250000 set(125000) 761.45 ns/op 902.75 ns/op 0.84
MutableVector 250000 toArray() 10.598 ms/op 10.165 ms/op 1.04
MutableVector 250000 iterate all - toArray() + loop 10.790 ms/op 10.591 ms/op 1.02
MutableVector 250000 iterate all - get(i) 4.5317 ms/op 5.4814 ms/op 0.83
Array 250000 create 6.9820 ms/op 7.0906 ms/op 0.98
Array 250000 clone - spread 3.1594 ms/op 2.7931 ms/op 1.13
Array 250000 get(125000) 1.4840 ns/op 1.4110 ns/op 1.05
Array 250000 set(125000) 1.5240 ns/op 1.3400 ns/op 1.14
Array 250000 iterate all - loop 201.47 us/op 153.06 us/op 1.32
effectiveBalanceIncrements clone Uint8Array 300000 91.032 us/op 309.27 us/op 0.29
effectiveBalanceIncrements clone MutableVector 300000 941.00 ns/op 695.00 ns/op 1.35
effectiveBalanceIncrements rw all Uint8Array 300000 362.43 us/op 205.81 us/op 1.76
effectiveBalanceIncrements rw all MutableVector 300000 259.67 ms/op 263.19 ms/op 0.99
aggregationBits - 2048 els - readonlyValues 227.97 us/op 220.04 us/op 1.04
aggregationBits - 2048 els - zipIndexesInBitList 47.380 us/op 43.830 us/op 1.08
regular array get 100000 times 80.930 us/op 62.793 us/op 1.29
wrappedArray get 100000 times 80.949 us/op 61.736 us/op 1.31
arrayWithProxy get 100000 times 34.363 ms/op 47.848 ms/op 0.72
ssz.Root.equals 1.3530 us/op 1.3480 us/op 1.00
ssz.Root.equals with valueOf() 1.5980 us/op 1.6160 us/op 0.99
byteArrayEquals with valueOf() 1.6210 us/op 1.5860 us/op 1.02
phase0 processBlock - 250000 vs - 7PWei normalcase 9.8012 ms/op 10.814 ms/op 0.91
phase0 processBlock - 250000 vs - 7PWei worstcase 88.917 ms/op 105.59 ms/op 0.84
phase0 afterProcessEpoch - 250000 vs - 7PWei 269.31 ms/op 244.51 ms/op 1.10
phase0 beforeProcessEpoch - 250000 vs - 7PWei 641.92 ms/op 776.73 ms/op 0.83
phase0 processEpoch - mainnet_e58758 909.90 ms/op 1.0409 s/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 502.09 ms/op 610.08 ms/op 0.82
mainnet_e58758 - phase0 processJustificationAndFinalization 66.144 us/op 108.19 us/op 0.61
mainnet_e58758 - phase0 processRewardsAndPenalties 112.57 ms/op 95.898 ms/op 1.17
mainnet_e58758 - phase0 processRegistryUpdates 51.261 us/op 87.194 us/op 0.59
mainnet_e58758 - phase0 processSlashings 3.1560 us/op 6.2600 us/op 0.50
mainnet_e58758 - phase0 processEth1DataReset 3.1870 us/op 5.1830 us/op 0.61
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 6.1750 ms/op 6.0786 ms/op 1.02
mainnet_e58758 - phase0 processSlashingsReset 14.516 us/op 30.235 us/op 0.48
mainnet_e58758 - phase0 processRandaoMixesReset 18.901 us/op 34.597 us/op 0.55
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.5520 us/op 8.3210 us/op 0.55
mainnet_e58758 - phase0 processParticipationRecordUpdates 14.006 us/op 25.084 us/op 0.56
mainnet_e58758 - phase0 afterProcessEpoch 214.85 ms/op 214.20 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 7.1988 ms/op 7.2898 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 7.5535 ms/op 7.3696 ms/op 1.02
phase0 processRegistryUpdates - 250000 normalcase 51.447 us/op 91.975 us/op 0.56
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.4812 ms/op 4.9013 ms/op 0.71
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.0300 s/op 2.6714 s/op 0.76
phase0 getAttestationDeltas - 250000 normalcase 15.502 ms/op 15.251 ms/op 1.02
phase0 getAttestationDeltas - 250000 worstcase 15.698 ms/op 15.461 ms/op 1.02
phase0 processSlashings - 250000 worstcase 47.278 ms/op 49.593 ms/op 0.95
shuffle list - 16384 els 15.678 ms/op 15.770 ms/op 0.99
shuffle list - 250000 els 222.63 ms/op 219.48 ms/op 1.01
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 571.46 us/op 955.08 us/op 0.60
pass gossip attestations to forkchoice per slot 16.922 ms/op 20.918 ms/op 0.81
computeDeltas 3.7427 ms/op 3.8037 ms/op 0.98
computeProposerBoostScoreFromBalances 603.94 us/op 514.94 us/op 1.17
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.8598 ms/op 2.2513 ms/op 1.27
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 979.04 us/op 883.73 us/op 1.11
BLS verify - blst-native 2.2314 ms/op 2.7802 ms/op 0.80
BLS verifyMultipleSignatures 3 - blst-native 4.5851 ms/op 5.6939 ms/op 0.81
BLS verifyMultipleSignatures 8 - blst-native 9.8903 ms/op 12.402 ms/op 0.80
BLS verifyMultipleSignatures 32 - blst-native 35.882 ms/op 44.711 ms/op 0.80
BLS aggregatePubkeys 32 - blst-native 47.725 us/op 62.597 us/op 0.76
BLS aggregatePubkeys 128 - blst-native 185.04 us/op 240.02 us/op 0.77
getAttestationsForBlock 76.304 ms/op 76.998 ms/op 0.99
CheckpointStateCache - add get delete 22.650 us/op 25.314 us/op 0.89
validate gossip signedAggregateAndProof - struct 5.4777 ms/op 6.8585 ms/op 0.80
validate gossip signedAggregateAndProof - treeBacked 5.3041 ms/op 6.7537 ms/op 0.79
validate gossip attestation - struct 2.4939 ms/op 3.1348 ms/op 0.80
validate gossip attestation - treeBacked 2.5222 ms/op 3.2536 ms/op 0.78
pickEth1Vote - no votes 12.000 ms/op 10.989 ms/op 1.09
pickEth1Vote - max votes 62.239 ms/op 63.753 ms/op 0.98
pickEth1Vote - Eth1Data hashTreeRoot value x2048 29.899 ms/op 33.762 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 11.444 ms/op 12.595 ms/op 0.91
pickEth1Vote - Eth1Data fastSerialize value x2048 6.8194 ms/op 6.6272 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize tree x2048 27.061 ms/op 30.783 ms/op 0.88
bytes32 toHexString 2.1540 us/op 2.2310 us/op 0.97
bytes32 Buffer.toString(hex) 820.00 ns/op 913.00 ns/op 0.90
bytes32 Buffer.toString(hex) from Uint8Array 1.1120 us/op 1.2740 us/op 0.87
bytes32 Buffer.toString(hex) + 0x 834.00 ns/op 887.00 ns/op 0.94
Object access 1 prop 0.40900 ns/op 0.43900 ns/op 0.93
Map access 1 prop 0.34200 ns/op 0.36100 ns/op 0.95
Object get x1000 20.620 ns/op 17.095 ns/op 1.21
Map get x1000 1.2940 ns/op 1.1000 ns/op 1.18
Object set x1000 128.84 ns/op 117.86 ns/op 1.09
Map set x1000 80.552 ns/op 88.187 ns/op 0.91
Return object 10000 times 0.44970 ns/op 0.43820 ns/op 1.03
Throw Error 10000 times 7.0289 us/op 7.1793 us/op 0.98
enrSubnets - fastDeserialize 64 bits 1.5430 us/op 1.6550 us/op 0.93
enrSubnets - ssz BitVector 64 bits 19.603 us/op 20.109 us/op 0.97
enrSubnets - fastDeserialize 4 bits 546.00 ns/op 575.00 ns/op 0.95
enrSubnets - ssz BitVector 4 bits 3.4630 us/op 3.7510 us/op 0.92
RateTracker 1000000 limit, 1 obj count per request 214.88 ns/op 217.42 ns/op 0.99
RateTracker 1000000 limit, 2 obj count per request 161.70 ns/op 160.39 ns/op 1.01
RateTracker 1000000 limit, 4 obj count per request 134.31 ns/op 131.61 ns/op 1.02
RateTracker 1000000 limit, 8 obj count per request 120.74 ns/op 119.80 ns/op 1.01
RateTracker with prune 5.0350 us/op 5.5090 us/op 0.91
array of 16000 items push then shift 3.7634 us/op 5.3688 us/op 0.70
LinkedList of 16000 items push then shift 20.808 ns/op 20.273 ns/op 1.03
array of 16000 items push then pop 243.27 ns/op 245.81 ns/op 0.99
LinkedList of 16000 items push then pop 18.320 ns/op 20.947 ns/op 0.87
array of 24000 items push then shift 5.4701 us/op 8.0212 us/op 0.68
LinkedList of 24000 items push then shift 23.843 ns/op 20.005 ns/op 1.19
array of 24000 items push then pop 253.80 ns/op 255.31 ns/op 0.99
LinkedList of 24000 items push then pop 20.068 ns/op 18.973 ns/op 1.06

by benchmarkbot/action

@dapplion dapplion merged commit cf5b293 into master Mar 2, 2022
@dapplion dapplion deleted the dapplion/specs-v1.1.10 branch March 2, 2022 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants