Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't manipualte spec tests casing #3715

Merged
merged 5 commits into from
Feb 7, 2022
Merged

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Feb 7, 2022

Motivation

Current master manipulates spec tests casing, which makes it harder to reason about when comparing with the spec tests docs. While re-mapping types casing is justified (user facing functionality, consistent with JS space), I don't believe remapping this spec test keys is worth it.

Description

  • Don't manipualte spec tests casing
  • Declare correct types of numbers and convert to number typesafe

@dapplion dapplion force-pushed the dapplion/spec-tests-casing branch from 74ef583 to 8331c66 Compare February 7, 2022 09:28
@codecov
Copy link

codecov bot commented Feb 7, 2022

Codecov Report

Merging #3715 (942b725) into master (14f5284) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3715   +/-   ##
=======================================
  Coverage   37.16%   37.16%           
=======================================
  Files         321      321           
  Lines        8712     8712           
  Branches     1350     1350           
=======================================
  Hits         3238     3238           
  Misses       5332     5332           
  Partials      142      142           

@github-actions
Copy link
Contributor

github-actions bot commented Feb 7, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a90f4d5 Previous: 14f5284 Ratio
BeaconState.hashTreeRoot - No change 609.00 ns/op 616.00 ns/op 0.99
BeaconState.hashTreeRoot - 1 full validator 153.85 us/op 148.01 us/op 1.04
BeaconState.hashTreeRoot - 32 full validator 2.1930 ms/op 2.2002 ms/op 1.00
BeaconState.hashTreeRoot - 512 full validator 30.712 ms/op 29.850 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 156.71 us/op 154.78 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.7801 ms/op 2.3791 ms/op 1.17
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 33.948 ms/op 32.560 ms/op 1.04
BeaconState.hashTreeRoot - 1 balances 104.25 us/op 108.57 us/op 0.96
BeaconState.hashTreeRoot - 32 balances 970.42 us/op 895.10 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 8.8337 ms/op 8.6031 ms/op 1.03
BeaconState.hashTreeRoot - 250000 balances 153.28 ms/op 151.79 ms/op 1.01
processSlot - 1 slots 67.739 us/op 52.298 us/op 1.30
processSlot - 32 slots 3.7830 ms/op 3.2853 ms/op 1.15
getCommitteeAssignments - req 1 vs - 250000 vc 5.2632 ms/op 6.0643 ms/op 0.87
getCommitteeAssignments - req 100 vs - 250000 vc 7.0668 ms/op 8.3339 ms/op 0.85
getCommitteeAssignments - req 1000 vs - 250000 vc 7.6583 ms/op 8.9891 ms/op 0.85
computeProposers - vc 250000 24.436 ms/op 24.309 ms/op 1.01
computeEpochShuffling - vc 250000 199.76 ms/op 211.06 ms/op 0.95
getNextSyncCommittee - vc 250000 428.07 ms/op 391.88 ms/op 1.09
altair processAttestation - 250000 vs - 7PWei normalcase 56.228 ms/op 49.343 ms/op 1.14
altair processAttestation - 250000 vs - 7PWei worstcase 53.548 ms/op 53.491 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 16.408 ms/op 13.119 ms/op 1.25
altair processAttestation - setStatus - 1/3 committees join 36.874 ms/op 27.323 ms/op 1.35
altair processAttestation - setStatus - 1/2 committees join 50.354 ms/op 40.375 ms/op 1.25
altair processAttestation - setStatus - 2/3 committees join 68.792 ms/op 54.084 ms/op 1.27
altair processAttestation - setStatus - 4/5 committees join 77.993 ms/op 66.815 ms/op 1.17
altair processAttestation - setStatus - 100% committees join 99.088 ms/op 81.041 ms/op 1.22
altair processAttestation - updateEpochParticipants - 1/6 committees join 17.373 ms/op 13.510 ms/op 1.29
altair processAttestation - updateEpochParticipants - 1/3 committees join 32.050 ms/op 32.180 ms/op 1.00
altair processAttestation - updateEpochParticipants - 1/2 committees join 41.090 ms/op 27.840 ms/op 1.48
altair processAttestation - updateEpochParticipants - 2/3 committees join 42.665 ms/op 29.114 ms/op 1.47
altair processAttestation - updateEpochParticipants - 4/5 committees join 46.818 ms/op 38.711 ms/op 1.21
altair processAttestation - updateEpochParticipants - 100% committees join 61.285 ms/op 32.637 ms/op 1.88
altair processAttestation - updateAllStatus 32.215 ms/op 24.198 ms/op 1.33
altair processBlock - 250000 vs - 7PWei normalcase 53.706 ms/op 49.330 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase 147.96 ms/op 134.98 ms/op 1.10
altair processEpoch - mainnet_e81889 1.3903 s/op 1.3283 s/op 1.05
mainnet_e81889 - altair beforeProcessEpoch 396.68 ms/op 322.28 ms/op 1.23
mainnet_e81889 - altair processJustificationAndFinalization 118.28 us/op 79.728 us/op 1.48
mainnet_e81889 - altair processInactivityUpdates 19.966 ms/op 21.285 ms/op 0.94
mainnet_e81889 - altair processRewardsAndPenalties 292.04 ms/op 310.15 ms/op 0.94
mainnet_e81889 - altair processRegistryUpdates 20.840 us/op 13.515 us/op 1.54
mainnet_e81889 - altair processSlashings 6.0840 us/op 3.7430 us/op 1.63
mainnet_e81889 - altair processEth1DataReset 5.8890 us/op 3.1990 us/op 1.84
mainnet_e81889 - altair processEffectiveBalanceUpdates 13.421 ms/op 13.382 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 36.016 us/op 20.458 us/op 1.76
mainnet_e81889 - altair processRandaoMixesReset 39.223 us/op 20.371 us/op 1.93
mainnet_e81889 - altair processHistoricalRootsUpdate 8.3800 us/op 4.8370 us/op 1.73
mainnet_e81889 - altair processParticipationFlagUpdates 176.11 ms/op 184.23 ms/op 0.96
mainnet_e81889 - altair processSyncCommitteeUpdates 5.8610 us/op 3.6400 us/op 1.61
mainnet_e81889 - altair afterProcessEpoch 229.65 ms/op 258.46 ms/op 0.89
altair processInactivityUpdates - 250000 normalcase 80.376 ms/op 74.798 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 94.216 ms/op 76.869 ms/op 1.23
altair processParticipationFlagUpdates - 250000 anycase 112.73 ms/op 103.71 ms/op 1.09
altair processRewardsAndPenalties - 250000 normalcase 307.33 ms/op 260.95 ms/op 1.18
altair processRewardsAndPenalties - 250000 worstcase 261.98 ms/op 295.07 ms/op 0.89
altair processSyncCommitteeUpdates - 250000 471.84 ms/op 425.99 ms/op 1.11
Tree 40 250000 create 1.2042 s/op 914.33 ms/op 1.32
Tree 40 250000 get(125000) 335.53 ns/op 382.07 ns/op 0.88
Tree 40 250000 set(125000) 2.6081 us/op 2.0475 us/op 1.27
Tree 40 250000 toArray() 56.480 ms/op 49.522 ms/op 1.14
Tree 40 250000 iterate all - toArray() + loop 49.312 ms/op 43.477 ms/op 1.13
Tree 40 250000 iterate all - get(i) 157.84 ms/op 140.57 ms/op 1.12
MutableVector 250000 create 30.684 ms/op 29.639 ms/op 1.04
MutableVector 250000 get(125000) 16.020 ns/op 17.001 ns/op 0.94
MutableVector 250000 set(125000) 873.14 ns/op 609.37 ns/op 1.43
MutableVector 250000 toArray() 10.188 ms/op 10.269 ms/op 0.99
MutableVector 250000 iterate all - toArray() + loop 10.449 ms/op 10.772 ms/op 0.97
MutableVector 250000 iterate all - get(i) 3.7405 ms/op 3.9794 ms/op 0.94
Array 250000 create 6.6968 ms/op 6.3763 ms/op 1.05
Array 250000 clone - spread 2.9462 ms/op 2.5696 ms/op 1.15
Array 250000 get(125000) 1.3520 ns/op 1.2150 ns/op 1.11
Array 250000 set(125000) 1.4100 ns/op 1.1920 ns/op 1.18
Array 250000 iterate all - loop 151.09 us/op 193.66 us/op 0.78
aggregationBits - 2048 els - readonlyValues 295.54 us/op 272.27 us/op 1.09
aggregationBits - 2048 els - zipIndexesInBitList 50.113 us/op 48.998 us/op 1.02
regular array get 100000 times 61.071 us/op 78.810 us/op 0.77
wrappedArray get 100000 times 60.033 us/op 76.796 us/op 0.78
arrayWithProxy get 100000 times 36.562 ms/op 32.318 ms/op 1.13
ssz.Root.equals 1.4460 us/op 1.3470 us/op 1.07
ssz.Root.equals with valueOf() 1.6650 us/op 1.5440 us/op 1.08
byteArrayEquals with valueOf() 1.6730 us/op 1.5120 us/op 1.11
phase0 processBlock - 250000 vs - 7PWei normalcase 13.618 ms/op 12.713 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei worstcase 101.05 ms/op 88.329 ms/op 1.14
phase0 afterProcessEpoch - 250000 vs - 7PWei 234.05 ms/op 247.38 ms/op 0.95
phase0 beforeProcessEpoch - 250000 vs - 7PWei 777.57 ms/op 666.25 ms/op 1.17
phase0 processEpoch - mainnet_e58758 1.1386 s/op 943.96 ms/op 1.21
mainnet_e58758 - phase0 beforeProcessEpoch 553.60 ms/op 530.94 ms/op 1.04
mainnet_e58758 - phase0 processJustificationAndFinalization 109.95 us/op 79.457 us/op 1.38
mainnet_e58758 - phase0 processRewardsAndPenalties 159.66 ms/op 156.18 ms/op 1.02
mainnet_e58758 - phase0 processRegistryUpdates 80.564 us/op 60.538 us/op 1.33
mainnet_e58758 - phase0 processSlashings 7.0930 us/op 3.9470 us/op 1.80
mainnet_e58758 - phase0 processEth1DataReset 5.5730 us/op 3.8220 us/op 1.46
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 10.713 ms/op 11.689 ms/op 0.92
mainnet_e58758 - phase0 processSlashingsReset 27.678 us/op 17.998 us/op 1.54
mainnet_e58758 - phase0 processRandaoMixesReset 37.811 us/op 20.532 us/op 1.84
mainnet_e58758 - phase0 processHistoricalRootsUpdate 7.5860 us/op 4.6000 us/op 1.65
mainnet_e58758 - phase0 processParticipationRecordUpdates 25.949 us/op 16.368 us/op 1.59
mainnet_e58758 - phase0 afterProcessEpoch 212.77 ms/op 255.33 ms/op 0.83
phase0 processEffectiveBalanceUpdates - 250000 normalcase 16.775 ms/op 13.134 ms/op 1.28
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.8142 s/op 1.5473 s/op 1.17
phase0 processRegistryUpdates - 250000 normalcase 85.718 us/op 55.970 us/op 1.53
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.9608 ms/op 3.1573 ms/op 1.25
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.2786 s/op 1.9429 s/op 1.17
phase0 getAttestationDeltas - 250000 normalcase 94.055 ms/op 85.457 ms/op 1.10
phase0 getAttestationDeltas - 250000 worstcase 96.382 ms/op 85.754 ms/op 1.12
phase0 processSlashings - 250000 worstcase 47.114 ms/op 42.172 ms/op 1.12
shuffle list - 16384 els 14.358 ms/op 15.304 ms/op 0.94
shuffle list - 250000 els 205.01 ms/op 211.15 ms/op 0.97
getEffectiveBalances - 250000 vs - 7PWei 13.146 ms/op 12.251 ms/op 1.07
pass gossip attestations to forkchoice per slot 21.388 ms/op 18.635 ms/op 1.15
computeDeltas 4.1400 ms/op 3.8354 ms/op 1.08
computeProposerBoostScoreFromBalances 291.77 us/op 391.81 us/op 0.74
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.5492 ms/op 2.4306 ms/op 1.05
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 777.14 us/op 795.30 us/op 0.98
BLS verify - blst-native 2.2192 ms/op 2.1899 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 4.7212 ms/op 4.4495 ms/op 1.06
BLS verifyMultipleSignatures 8 - blst-native 10.251 ms/op 9.3119 ms/op 1.10
BLS verifyMultipleSignatures 32 - blst-native 35.885 ms/op 34.777 ms/op 1.03
BLS aggregatePubkeys 32 - blst-native 47.164 us/op 45.880 us/op 1.03
BLS aggregatePubkeys 128 - blst-native 187.43 us/op 175.43 us/op 1.07
getAttestationsForBlock 101.32 ms/op 94.385 ms/op 1.07
CheckpointStateCache - add get delete 23.989 us/op 21.944 us/op 1.09
validate gossip signedAggregateAndProof - struct 5.5752 ms/op 5.0301 ms/op 1.11
validate gossip signedAggregateAndProof - treeBacked 5.7851 ms/op 5.0645 ms/op 1.14
validate gossip attestation - struct 2.6138 ms/op 2.3927 ms/op 1.09
validate gossip attestation - treeBacked 2.5721 ms/op 2.5267 ms/op 1.02
bytes32 toHexString 1.9190 us/op 1.9110 us/op 1.00
bytes32 Buffer.toString(hex) 768.00 ns/op 806.00 ns/op 0.95
bytes32 Buffer.toString(hex) from Uint8Array 1.0330 us/op 1.0700 us/op 0.97
bytes32 Buffer.toString(hex) + 0x 781.00 ns/op 794.00 ns/op 0.98
Object access 1 prop 0.37600 ns/op 0.36000 ns/op 1.04
Map access 1 prop 0.31900 ns/op 0.31700 ns/op 1.01
Object get x1000 16.275 ns/op 20.197 ns/op 0.81
Map get x1000 0.91100 ns/op 1.0910 ns/op 0.84
Object set x1000 123.08 ns/op 121.23 ns/op 1.02
Map set x1000 79.039 ns/op 72.161 ns/op 1.10
Return object 10000 times 0.41080 ns/op 0.41490 ns/op 0.99
Throw Error 10000 times 6.7857 us/op 6.4944 us/op 1.04
enrSubnets - fastDeserialize 64 bits 1.4020 us/op 1.3620 us/op 1.03
enrSubnets - ssz BitVector 64 bits 18.557 us/op 19.049 us/op 0.97
enrSubnets - fastDeserialize 4 bits 519.00 ns/op 507.00 ns/op 1.02
enrSubnets - ssz BitVector 4 bits 3.3080 us/op 3.5820 us/op 0.92
RateTracker 1000000 limit, 1 obj count per request 207.61 ns/op 214.54 ns/op 0.97
RateTracker 1000000 limit, 2 obj count per request 148.79 ns/op 161.92 ns/op 0.92
RateTracker 1000000 limit, 4 obj count per request 120.78 ns/op 131.95 ns/op 0.92
RateTracker 1000000 limit, 8 obj count per request 106.43 ns/op 115.75 ns/op 0.92
RateTracker with prune 4.7330 us/op 4.6570 us/op 1.02

by benchmarkbot/action

@wemeetagain wemeetagain merged commit aedf4a9 into master Feb 7, 2022
@wemeetagain wemeetagain deleted the dapplion/spec-tests-casing branch February 7, 2022 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants