Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: more metrics for sync committee message validation #5516

Merged
merged 1 commit into from
May 23, 2023
Merged

feat: more metrics for sync committee message validation #5516

merged 1 commit into from
May 23, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented May 22, 2023

Motivation

  • More clear about sync committee message validation

Description

  • Track equivocation in sync committee message validation by adding 2 new metrics

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 2b666d6 Previous: 402c46f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 728.70 us/op 538.02 us/op 1.35
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 43.704 us/op 46.888 us/op 0.93
BLS verify - blst-native 1.2076 ms/op 1.2840 ms/op 0.94
BLS verifyMultipleSignatures 3 - blst-native 2.5365 ms/op 2.5627 ms/op 0.99
BLS verifyMultipleSignatures 8 - blst-native 5.2390 ms/op 5.7667 ms/op 0.91
BLS verifyMultipleSignatures 32 - blst-native 19.154 ms/op 20.371 ms/op 0.94
BLS aggregatePubkeys 32 - blst-native 26.004 us/op 26.720 us/op 0.97
BLS aggregatePubkeys 128 - blst-native 100.22 us/op 104.01 us/op 0.96
getAttestationsForBlock 57.018 ms/op 60.285 ms/op 0.95
isKnown best case - 1 super set check 260.00 ns/op 283.00 ns/op 0.92
isKnown normal case - 2 super set checks 255.00 ns/op 276.00 ns/op 0.92
isKnown worse case - 16 super set checks 253.00 ns/op 272.00 ns/op 0.93
CheckpointStateCache - add get delete 5.5170 us/op 5.1070 us/op 1.08
validate gossip signedAggregateAndProof - struct 2.7985 ms/op 2.8112 ms/op 1.00
validate gossip attestation - struct 1.3389 ms/op 1.3523 ms/op 0.99
pickEth1Vote - no votes 1.3053 ms/op 1.2675 ms/op 1.03
pickEth1Vote - max votes 9.7311 ms/op 11.364 ms/op 0.86
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.5329 ms/op 9.3008 ms/op 0.92
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 13.393 ms/op 19.063 ms/op 0.70
pickEth1Vote - Eth1Data fastSerialize value x2048 692.70 us/op 703.60 us/op 0.98
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.6491 ms/op 8.4191 ms/op 0.55
bytes32 toHexString 503.00 ns/op 541.00 ns/op 0.93
bytes32 Buffer.toString(hex) 349.00 ns/op 360.00 ns/op 0.97
bytes32 Buffer.toString(hex) from Uint8Array 548.00 ns/op 571.00 ns/op 0.96
bytes32 Buffer.toString(hex) + 0x 398.00 ns/op 365.00 ns/op 1.09
Object access 1 prop 0.19300 ns/op 0.17000 ns/op 1.14
Map access 1 prop 0.15700 ns/op 0.16400 ns/op 0.96
Object get x1000 6.9140 ns/op 7.7710 ns/op 0.89
Map get x1000 0.62500 ns/op 0.62700 ns/op 1.00
Object set x1000 54.995 ns/op 54.890 ns/op 1.00
Map set x1000 44.742 ns/op 44.222 ns/op 1.01
Return object 10000 times 0.23700 ns/op 0.23740 ns/op 1.00
Throw Error 10000 times 4.1593 us/op 4.3994 us/op 0.95
fastMsgIdFn sha256 / 200 bytes 3.4640 us/op 3.4960 us/op 0.99
fastMsgIdFn h32 xxhash / 200 bytes 324.00 ns/op 285.00 ns/op 1.14
fastMsgIdFn h64 xxhash / 200 bytes 504.00 ns/op 403.00 ns/op 1.25
fastMsgIdFn sha256 / 1000 bytes 11.752 us/op 12.154 us/op 0.97
fastMsgIdFn h32 xxhash / 1000 bytes 449.00 ns/op 416.00 ns/op 1.08
fastMsgIdFn h64 xxhash / 1000 bytes 559.00 ns/op 480.00 ns/op 1.16
fastMsgIdFn sha256 / 10000 bytes 101.54 us/op 110.00 us/op 0.92
fastMsgIdFn h32 xxhash / 10000 bytes 1.9460 us/op 1.9050 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.3890 us/op 1.3590 us/op 1.02
enrSubnets - fastDeserialize 64 bits 1.6360 us/op 1.4700 us/op 1.11
enrSubnets - ssz BitVector 64 bits 584.00 ns/op 528.00 ns/op 1.11
enrSubnets - fastDeserialize 4 bits 204.00 ns/op 187.00 ns/op 1.09
enrSubnets - ssz BitVector 4 bits 625.00 ns/op 531.00 ns/op 1.18
prioritizePeers score -10:0 att 32-0.1 sync 2-0 115.44 us/op 132.47 us/op 0.87
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 129.04 us/op 155.23 us/op 0.83
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 159.49 us/op 185.78 us/op 0.86
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 296.07 us/op 347.10 us/op 0.85
prioritizePeers score 0:0 att 64-1 sync 4-1 361.29 us/op 423.08 us/op 0.85
array of 16000 items push then shift 1.6266 us/op 1.8020 us/op 0.90
LinkedList of 16000 items push then shift 8.7290 ns/op 9.3190 ns/op 0.94
array of 16000 items push then pop 87.964 ns/op 103.91 ns/op 0.85
LinkedList of 16000 items push then pop 8.4620 ns/op 9.2920 ns/op 0.91
array of 24000 items push then shift 2.3359 us/op 2.5588 us/op 0.91
LinkedList of 24000 items push then shift 8.7590 ns/op 10.657 ns/op 0.82
array of 24000 items push then pop 76.492 ns/op 90.347 ns/op 0.85
LinkedList of 24000 items push then pop 8.3970 ns/op 9.1430 ns/op 0.92
intersect bitArray bitLen 8 16.038 ns/op 14.387 ns/op 1.11
intersect array and set length 8 78.203 ns/op 80.082 ns/op 0.98
intersect bitArray bitLen 128 45.623 ns/op 47.058 ns/op 0.97
intersect array and set length 128 1.0193 us/op 1.1052 us/op 0.92
Buffer.concat 32 items 2.5910 us/op 2.7820 us/op 0.93
Uint8Array.set 32 items 2.9790 us/op 3.3110 us/op 0.90
pass gossip attestations to forkchoice per slot 2.8363 ms/op 3.0591 ms/op 0.93
computeDeltas 2.8935 ms/op 3.6506 ms/op 0.79
computeProposerBoostScoreFromBalances 1.7608 ms/op 1.9103 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei normalcase 2.0843 ms/op 2.5318 ms/op 0.82
altair processAttestation - 250000 vs - 7PWei worstcase 3.1628 ms/op 4.2942 ms/op 0.74
altair processAttestation - setStatus - 1/6 committees join 135.38 us/op 150.22 us/op 0.90
altair processAttestation - setStatus - 1/3 committees join 266.26 us/op 297.90 us/op 0.89
altair processAttestation - setStatus - 1/2 committees join 365.24 us/op 396.41 us/op 0.92
altair processAttestation - setStatus - 2/3 committees join 448.29 us/op 495.11 us/op 0.91
altair processAttestation - setStatus - 4/5 committees join 646.44 us/op 697.61 us/op 0.93
altair processAttestation - setStatus - 100% committees join 721.14 us/op 802.53 us/op 0.90
altair processBlock - 250000 vs - 7PWei normalcase 19.820 ms/op 20.020 ms/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.061 ms/op 29.547 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase 53.906 ms/op 61.079 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase hashState 64.738 ms/op 82.583 ms/op 0.78
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9477 ms/op 1.9907 ms/op 0.98
phase0 processBlock - 250000 vs - 7PWei worstcase 26.996 ms/op 29.550 ms/op 0.91
altair processEth1Data - 250000 vs - 7PWei normalcase 458.65 us/op 491.20 us/op 0.93
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.6010 us/op 6.6270 us/op 1.00
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.702 us/op 32.558 us/op 0.61
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.3400 us/op 9.8520 us/op 0.85
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.2590 us/op 7.9560 us/op 0.79
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 74.249 us/op 106.86 us/op 0.69
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 618.40 us/op 1.0674 ms/op 0.58
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 893.97 us/op 1.3692 ms/op 0.65
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 864.61 us/op 1.6843 ms/op 0.51
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3045 ms/op 2.2918 ms/op 1.01
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.6027 ms/op 1.9564 ms/op 0.82
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.6799 ms/op 3.9485 ms/op 0.93
Tree 40 250000 create 306.92 ms/op 362.63 ms/op 0.85
Tree 40 250000 get(125000) 175.04 ns/op 205.38 ns/op 0.85
Tree 40 250000 set(125000) 933.13 ns/op 1.0223 us/op 0.91
Tree 40 250000 toArray() 16.704 ms/op 23.395 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 16.620 ms/op 23.508 ms/op 0.71
Tree 40 250000 iterate all - get(i) 66.365 ms/op 79.218 ms/op 0.84
MutableVector 250000 create 9.0495 ms/op 11.464 ms/op 0.79
MutableVector 250000 get(125000) 6.0830 ns/op 7.3000 ns/op 0.83
MutableVector 250000 set(125000) 243.41 ns/op 291.12 ns/op 0.84
MutableVector 250000 toArray() 2.6080 ms/op 2.9808 ms/op 0.87
MutableVector 250000 iterate all - toArray() + loop 2.6387 ms/op 4.0881 ms/op 0.65
MutableVector 250000 iterate all - get(i) 1.4629 ms/op 1.7181 ms/op 0.85
Array 250000 create 2.4250 ms/op 3.1401 ms/op 0.77
Array 250000 clone - spread 1.1273 ms/op 1.1597 ms/op 0.97
Array 250000 get(125000) 0.55700 ns/op 0.57100 ns/op 0.98
Array 250000 set(125000) 0.61300 ns/op 0.65400 ns/op 0.94
Array 250000 iterate all - loop 97.823 us/op 86.237 us/op 1.13
effectiveBalanceIncrements clone Uint8Array 300000 24.517 us/op 36.489 us/op 0.67
effectiveBalanceIncrements clone MutableVector 300000 359.00 ns/op 357.00 ns/op 1.01
effectiveBalanceIncrements rw all Uint8Array 300000 161.72 us/op 190.30 us/op 0.85
effectiveBalanceIncrements rw all MutableVector 300000 78.359 ms/op 84.220 ms/op 0.93
phase0 afterProcessEpoch - 250000 vs - 7PWei 112.94 ms/op 125.87 ms/op 0.90
phase0 beforeProcessEpoch - 250000 vs - 7PWei 38.528 ms/op 45.726 ms/op 0.84
altair processEpoch - mainnet_e81889 316.28 ms/op 354.80 ms/op 0.89
mainnet_e81889 - altair beforeProcessEpoch 63.041 ms/op 70.077 ms/op 0.90
mainnet_e81889 - altair processJustificationAndFinalization 17.675 us/op 18.386 us/op 0.96
mainnet_e81889 - altair processInactivityUpdates 5.3227 ms/op 5.9956 ms/op 0.89
mainnet_e81889 - altair processRewardsAndPenalties 67.519 ms/op 73.829 ms/op 0.91
mainnet_e81889 - altair processRegistryUpdates 2.8290 us/op 3.3790 us/op 0.84
mainnet_e81889 - altair processSlashings 479.00 ns/op 584.00 ns/op 0.82
mainnet_e81889 - altair processEth1DataReset 516.00 ns/op 659.00 ns/op 0.78
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.1956 ms/op 1.5996 ms/op 0.75
mainnet_e81889 - altair processSlashingsReset 5.2440 us/op 4.9610 us/op 1.06
mainnet_e81889 - altair processRandaoMixesReset 4.7000 us/op 9.9240 us/op 0.47
mainnet_e81889 - altair processHistoricalRootsUpdate 631.00 ns/op 1.6410 us/op 0.38
mainnet_e81889 - altair processParticipationFlagUpdates 2.3980 us/op 2.9730 us/op 0.81
mainnet_e81889 - altair processSyncCommitteeUpdates 467.00 ns/op 1.0110 us/op 0.46
mainnet_e81889 - altair afterProcessEpoch 114.45 ms/op 138.14 ms/op 0.83
phase0 processEpoch - mainnet_e58758 309.76 ms/op 420.21 ms/op 0.74
mainnet_e58758 - phase0 beforeProcessEpoch 119.18 ms/op 170.01 ms/op 0.70
mainnet_e58758 - phase0 processJustificationAndFinalization 16.266 us/op 17.118 us/op 0.95
mainnet_e58758 - phase0 processRewardsAndPenalties 53.909 ms/op 77.875 ms/op 0.69
mainnet_e58758 - phase0 processRegistryUpdates 8.4940 us/op 12.271 us/op 0.69
mainnet_e58758 - phase0 processSlashings 477.00 ns/op 943.00 ns/op 0.51
mainnet_e58758 - phase0 processEth1DataReset 512.00 ns/op 940.00 ns/op 0.54
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 940.22 us/op 1.8743 ms/op 0.50
mainnet_e58758 - phase0 processSlashingsReset 3.2650 us/op 6.1960 us/op 0.53
mainnet_e58758 - phase0 processRandaoMixesReset 4.8260 us/op 6.7240 us/op 0.72
mainnet_e58758 - phase0 processHistoricalRootsUpdate 580.00 ns/op 1.0680 us/op 0.54
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.9780 us/op 5.4060 us/op 0.74
mainnet_e58758 - phase0 afterProcessEpoch 94.302 ms/op 103.73 ms/op 0.91
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2057 ms/op 2.0270 ms/op 0.59
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4405 ms/op 2.7191 ms/op 0.53
altair processInactivityUpdates - 250000 normalcase 18.997 ms/op 32.460 ms/op 0.59
altair processInactivityUpdates - 250000 worstcase 21.361 ms/op 31.045 ms/op 0.69
phase0 processRegistryUpdates - 250000 normalcase 6.5530 us/op 10.862 us/op 0.60
phase0 processRegistryUpdates - 250000 badcase_full_deposits 231.92 us/op 313.32 us/op 0.74
phase0 processRegistryUpdates - 250000 worstcase 0.5 121.00 ms/op 149.98 ms/op 0.81
altair processRewardsAndPenalties - 250000 normalcase 56.283 ms/op 73.676 ms/op 0.76
altair processRewardsAndPenalties - 250000 worstcase 49.413 ms/op 77.709 ms/op 0.64
phase0 getAttestationDeltas - 250000 normalcase 6.3757 ms/op 6.9273 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 6.5769 ms/op 7.2137 ms/op 0.91
phase0 processSlashings - 250000 worstcase 3.4982 ms/op 3.7554 ms/op 0.93
altair processSyncCommitteeUpdates - 250000 169.38 ms/op 177.67 ms/op 0.95
BeaconState.hashTreeRoot - No change 253.00 ns/op 275.00 ns/op 0.92
BeaconState.hashTreeRoot - 1 full validator 51.267 us/op 74.442 us/op 0.69
BeaconState.hashTreeRoot - 32 full validator 520.48 us/op 620.91 us/op 0.84
BeaconState.hashTreeRoot - 512 full validator 5.3597 ms/op 5.2543 ms/op 1.02
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 64.098 us/op 74.045 us/op 0.87
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 882.78 us/op 953.91 us/op 0.93
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.468 ms/op 14.303 ms/op 0.80
BeaconState.hashTreeRoot - 1 balances 49.655 us/op 52.647 us/op 0.94
BeaconState.hashTreeRoot - 32 balances 448.45 us/op 542.19 us/op 0.83
BeaconState.hashTreeRoot - 512 balances 4.4321 ms/op 4.9071 ms/op 0.90
BeaconState.hashTreeRoot - 250000 balances 72.761 ms/op 81.205 ms/op 0.90
aggregationBits - 2048 els - zipIndexesInBitList 16.357 us/op 16.642 us/op 0.98
regular array get 100000 times 32.848 us/op 32.763 us/op 1.00
wrappedArray get 100000 times 41.445 us/op 33.872 us/op 1.22
arrayWithProxy get 100000 times 15.365 ms/op 16.357 ms/op 0.94
ssz.Root.equals 598.00 ns/op 573.00 ns/op 1.04
byteArrayEquals 542.00 ns/op 571.00 ns/op 0.95
shuffle list - 16384 els 7.0182 ms/op 7.1891 ms/op 0.98
shuffle list - 250000 els 100.72 ms/op 105.30 ms/op 0.96
processSlot - 1 slots 8.8640 us/op 9.4630 us/op 0.94
processSlot - 32 slots 1.3331 ms/op 1.4061 ms/op 0.95
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 34.197 ms/op 41.560 ms/op 0.82
getCommitteeAssignments - req 1 vs - 250000 vc 2.9287 ms/op 3.1148 ms/op 0.94
getCommitteeAssignments - req 100 vs - 250000 vc 4.1530 ms/op 4.3136 ms/op 0.96
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5993 ms/op 4.7387 ms/op 0.97
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4300 ns/op 5.3000 ns/op 0.84
state getBlockRootAtSlot - 250000 vs - 7PWei 596.02 ns/op 943.93 ns/op 0.63
computeProposers - vc 250000 10.124 ms/op 10.583 ms/op 0.96
computeEpochShuffling - vc 250000 105.10 ms/op 104.16 ms/op 1.01
getNextSyncCommittee - vc 250000 176.48 ms/op 188.50 ms/op 0.94
computeSigningRoot for AttestationData 13.628 us/op 13.349 us/op 1.02
hash AttestationData serialized data then Buffer.toString(base64) 2.5973 us/op 2.4202 us/op 1.07
toHexString serialized data 1.2248 us/op 1.0795 us/op 1.13
Buffer.toString(base64) 323.98 ns/op 320.34 ns/op 1.01

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review May 22, 2023 08:16
@twoeths twoeths requested a review from a team as a code owner May 22, 2023 08:16
@twoeths twoeths merged commit 42c8097 into unstable May 23, 2023
@twoeths twoeths deleted the tuyen/more_sync_committee_metrics branch May 23, 2023 00:55
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@twoeths @wemeetagain @dapplion