Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Track sync committee metrics #5146

Merged
merged 3 commits into from
Feb 14, 2023
Merged

Track sync committee metrics #5146

merged 3 commits into from
Feb 14, 2023

Conversation

dapplion
Copy link
Contributor

Motivation

Track multiple sync committee metrics relevant for local validator performance

Description

Adds:

  • validator_monitor_validators_in_sync_committee
  • validator_monitor_prev_epoch_sync_committee_hits
  • validator_monitor_prev_epoch_sync_committee_misses
  • validator_monitor_prev_epoch_sync_signature_aggregate_inclusions
  • validator_monitor_sync_signature_in_aggregate_total

Closes #3413

@dapplion dapplion requested a review from a team as a code owner February 14, 2023 08:31
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 144724d Previous: 4cec93e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 677.44 us/op 502.86 us/op 1.35
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.887 us/op 45.782 us/op 1.02
BLS verify - blst-native 1.2249 ms/op 1.2111 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.4846 ms/op 2.4632 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.5109 ms/op 5.2826 ms/op 1.04
BLS verifyMultipleSignatures 32 - blst-native 19.467 ms/op 18.996 ms/op 1.02
BLS aggregatePubkeys 32 - blst-native 26.046 us/op 25.549 us/op 1.02
BLS aggregatePubkeys 128 - blst-native 101.38 us/op 100.18 us/op 1.01
getAttestationsForBlock 58.724 ms/op 57.158 ms/op 1.03
isKnown best case - 1 super set check 259.00 ns/op 277.00 ns/op 0.94
isKnown normal case - 2 super set checks 265.00 ns/op 269.00 ns/op 0.99
isKnown worse case - 16 super set checks 269.00 ns/op 264.00 ns/op 1.02
CheckpointStateCache - add get delete 5.7040 us/op 5.2570 us/op 1.09
validate gossip signedAggregateAndProof - struct 2.7896 ms/op 2.7790 ms/op 1.00
validate gossip attestation - struct 1.3319 ms/op 1.3179 ms/op 1.01
pickEth1Vote - no votes 1.3279 ms/op 1.2642 ms/op 1.05
pickEth1Vote - max votes 10.128 ms/op 8.8716 ms/op 1.14
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.6891 ms/op 8.5898 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.492 ms/op 13.427 ms/op 1.23
pickEth1Vote - Eth1Data fastSerialize value x2048 767.81 us/op 643.47 us/op 1.19
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.3798 ms/op 4.5120 ms/op 1.19
bytes32 toHexString 610.00 ns/op 487.00 ns/op 1.25
bytes32 Buffer.toString(hex) 422.00 ns/op 335.00 ns/op 1.26
bytes32 Buffer.toString(hex) from Uint8Array 623.00 ns/op 552.00 ns/op 1.13
bytes32 Buffer.toString(hex) + 0x 413.00 ns/op 348.00 ns/op 1.19
Object access 1 prop 0.18800 ns/op 0.16900 ns/op 1.11
Map access 1 prop 0.17300 ns/op 0.16200 ns/op 1.07
Object get x1000 6.8670 ns/op 6.7380 ns/op 1.02
Map get x1000 0.63100 ns/op 0.55600 ns/op 1.13
Object set x1000 64.274 ns/op 53.068 ns/op 1.21
Map set x1000 51.094 ns/op 43.205 ns/op 1.18
Return object 10000 times 0.24560 ns/op 0.23980 ns/op 1.02
Throw Error 10000 times 4.2227 us/op 4.0672 us/op 1.04
fastMsgIdFn sha256 / 200 bytes 3.6050 us/op 3.4260 us/op 1.05
fastMsgIdFn h32 xxhash / 200 bytes 308.00 ns/op 274.00 ns/op 1.12
fastMsgIdFn h64 xxhash / 200 bytes 476.00 ns/op 383.00 ns/op 1.24
fastMsgIdFn sha256 / 1000 bytes 11.847 us/op 11.466 us/op 1.03
fastMsgIdFn h32 xxhash / 1000 bytes 446.00 ns/op 393.00 ns/op 1.13
fastMsgIdFn h64 xxhash / 1000 bytes 530.00 ns/op 445.00 ns/op 1.19
fastMsgIdFn sha256 / 10000 bytes 105.68 us/op 102.96 us/op 1.03
fastMsgIdFn h32 xxhash / 10000 bytes 1.9590 us/op 1.9040 us/op 1.03
fastMsgIdFn h64 xxhash / 10000 bytes 1.4040 us/op 1.3310 us/op 1.05
enrSubnets - fastDeserialize 64 bits 1.4990 us/op 1.2770 us/op 1.17
enrSubnets - ssz BitVector 64 bits 589.00 ns/op 481.00 ns/op 1.22
enrSubnets - fastDeserialize 4 bits 199.00 ns/op 169.00 ns/op 1.18
enrSubnets - ssz BitVector 4 bits 582.00 ns/op 479.00 ns/op 1.22
prioritizePeers score -10:0 att 32-0.1 sync 2-0 105.81 us/op 93.220 us/op 1.14
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.06 us/op 123.69 us/op 1.06
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 213.96 us/op 190.08 us/op 1.13
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 394.31 us/op 298.68 us/op 1.32
prioritizePeers score 0:0 att 64-1 sync 4-1 466.56 us/op 353.22 us/op 1.32
array of 16000 items push then shift 1.7671 us/op 1.6024 us/op 1.10
LinkedList of 16000 items push then shift 9.3420 ns/op 8.6120 ns/op 1.08
array of 16000 items push then pop 105.42 ns/op 85.515 ns/op 1.23
LinkedList of 16000 items push then pop 9.0610 ns/op 8.4190 ns/op 1.08
array of 24000 items push then shift 2.4422 us/op 2.2952 us/op 1.06
LinkedList of 24000 items push then shift 9.5020 ns/op 8.7030 ns/op 1.09
array of 24000 items push then pop 89.965 ns/op 77.888 ns/op 1.16
LinkedList of 24000 items push then pop 9.3520 ns/op 8.4980 ns/op 1.10
intersect bitArray bitLen 8 13.677 ns/op 13.081 ns/op 1.05
intersect array and set length 8 80.201 ns/op 73.807 ns/op 1.09
intersect bitArray bitLen 128 45.520 ns/op 43.501 ns/op 1.05
intersect array and set length 128 1.1549 us/op 1.0195 us/op 1.13
Buffer.concat 32 items 2.8900 us/op 2.5980 us/op 1.11
Uint8Array.set 32 items 2.2590 us/op 2.1700 us/op 1.04
pass gossip attestations to forkchoice per slot 2.4276 ms/op 2.2845 ms/op 1.06
computeDeltas 3.0533 ms/op 3.3002 ms/op 0.93
computeProposerBoostScoreFromBalances 1.8226 ms/op 1.7708 ms/op 1.03
altair processAttestation - 250000 vs - 7PWei normalcase 2.7900 ms/op 2.1230 ms/op 1.31
altair processAttestation - 250000 vs - 7PWei worstcase 3.6767 ms/op 3.3531 ms/op 1.10
altair processAttestation - setStatus - 1/6 committees join 142.33 us/op 136.65 us/op 1.04
altair processAttestation - setStatus - 1/3 committees join 294.29 us/op 269.97 us/op 1.09
altair processAttestation - setStatus - 1/2 committees join 375.82 us/op 360.39 us/op 1.04
altair processAttestation - setStatus - 2/3 committees join 474.79 us/op 456.30 us/op 1.04
altair processAttestation - setStatus - 4/5 committees join 657.14 us/op 639.15 us/op 1.03
altair processAttestation - setStatus - 100% committees join 771.50 us/op 762.98 us/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 17.261 ms/op 17.047 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.131 ms/op 25.433 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 50.454 ms/op 51.356 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase hashState 73.367 ms/op 66.497 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1679 ms/op 2.0127 ms/op 1.08
phase0 processBlock - 250000 vs - 7PWei worstcase 29.882 ms/op 28.564 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 496.68 us/op 462.95 us/op 1.07
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 8.2140 us/op 7.3190 us/op 1.12
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 26.004 us/op 25.566 us/op 1.02
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.343 us/op 8.8620 us/op 1.17
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 8.4440 us/op 7.1950 us/op 1.17
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 101.30 us/op 82.178 us/op 1.23
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 664.53 us/op 655.07 us/op 1.01
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 919.20 us/op 898.20 us/op 1.02
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 863.30 us/op 867.25 us/op 1.00
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3436 ms/op 2.3133 ms/op 1.01
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.6185 ms/op 1.4629 ms/op 1.11
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 4.0357 ms/op 4.1564 ms/op 0.97
Tree 40 250000 create 327.17 ms/op 343.73 ms/op 0.95
Tree 40 250000 get(125000) 196.18 ns/op 190.85 ns/op 1.03
Tree 40 250000 set(125000) 1.0205 us/op 1.0179 us/op 1.00
Tree 40 250000 toArray() 21.292 ms/op 22.134 ms/op 0.96
Tree 40 250000 iterate all - toArray() + loop 21.558 ms/op 22.416 ms/op 0.96
Tree 40 250000 iterate all - get(i) 75.172 ms/op 74.668 ms/op 1.01
MutableVector 250000 create 10.634 ms/op 10.178 ms/op 1.04
MutableVector 250000 get(125000) 6.4980 ns/op 6.2340 ns/op 1.04
MutableVector 250000 set(125000) 267.07 ns/op 266.00 ns/op 1.00
MutableVector 250000 toArray() 3.0029 ms/op 3.1123 ms/op 0.96
MutableVector 250000 iterate all - toArray() + loop 2.9583 ms/op 2.9197 ms/op 1.01
MutableVector 250000 iterate all - get(i) 1.5156 ms/op 1.4986 ms/op 1.01
Array 250000 create 2.6284 ms/op 2.4677 ms/op 1.07
Array 250000 clone - spread 1.2507 ms/op 1.0781 ms/op 1.16
Array 250000 get(125000) 0.62400 ns/op 0.55400 ns/op 1.13
Array 250000 set(125000) 0.70600 ns/op 0.61700 ns/op 1.14
Array 250000 iterate all - loop 104.47 us/op 109.32 us/op 0.96
effectiveBalanceIncrements clone Uint8Array 300000 36.894 us/op 29.113 us/op 1.27
effectiveBalanceIncrements clone MutableVector 300000 415.00 ns/op 338.00 ns/op 1.23
effectiveBalanceIncrements rw all Uint8Array 300000 167.98 us/op 166.54 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 86.745 ms/op 78.676 ms/op 1.10
phase0 afterProcessEpoch - 250000 vs - 7PWei 113.37 ms/op 112.54 ms/op 1.01
phase0 beforeProcessEpoch - 250000 vs - 7PWei 34.969 ms/op 41.511 ms/op 0.84
altair processEpoch - mainnet_e81889 333.63 ms/op 335.22 ms/op 1.00
mainnet_e81889 - altair beforeProcessEpoch 69.172 ms/op 65.791 ms/op 1.05
mainnet_e81889 - altair processJustificationAndFinalization 17.207 us/op 18.027 us/op 0.95
mainnet_e81889 - altair processInactivityUpdates 6.0167 ms/op 5.2674 ms/op 1.14
mainnet_e81889 - altair processRewardsAndPenalties 68.583 ms/op 65.025 ms/op 1.05
mainnet_e81889 - altair processRegistryUpdates 3.0050 us/op 3.0340 us/op 0.99
mainnet_e81889 - altair processSlashings 429.00 ns/op 465.00 ns/op 0.92
mainnet_e81889 - altair processEth1DataReset 477.00 ns/op 503.00 ns/op 0.95
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2627 ms/op 1.2407 ms/op 1.02
mainnet_e81889 - altair processSlashingsReset 4.4020 us/op 4.6940 us/op 0.94
mainnet_e81889 - altair processRandaoMixesReset 4.4290 us/op 4.4470 us/op 1.00
mainnet_e81889 - altair processHistoricalRootsUpdate 619.00 ns/op 648.00 ns/op 0.96
mainnet_e81889 - altair processParticipationFlagUpdates 2.7300 us/op 2.4090 us/op 1.13
mainnet_e81889 - altair processSyncCommitteeUpdates 459.00 ns/op 516.00 ns/op 0.89
mainnet_e81889 - altair afterProcessEpoch 129.13 ms/op 115.98 ms/op 1.11
phase0 processEpoch - mainnet_e58758 357.84 ms/op 312.07 ms/op 1.15
mainnet_e58758 - phase0 beforeProcessEpoch 140.77 ms/op 120.57 ms/op 1.17
mainnet_e58758 - phase0 processJustificationAndFinalization 16.386 us/op 15.841 us/op 1.03
mainnet_e58758 - phase0 processRewardsAndPenalties 62.042 ms/op 52.409 ms/op 1.18
mainnet_e58758 - phase0 processRegistryUpdates 8.1180 us/op 7.9270 us/op 1.02
mainnet_e58758 - phase0 processSlashings 495.00 ns/op 509.00 ns/op 0.97
mainnet_e58758 - phase0 processEth1DataReset 557.00 ns/op 502.00 ns/op 1.11
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0076 ms/op 944.78 us/op 1.07
mainnet_e58758 - phase0 processSlashingsReset 4.4010 us/op 3.1480 us/op 1.40
mainnet_e58758 - phase0 processRandaoMixesReset 4.3320 us/op 4.7840 us/op 0.91
mainnet_e58758 - phase0 processHistoricalRootsUpdate 500.00 ns/op 559.00 ns/op 0.89
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.0350 us/op 3.9740 us/op 1.02
mainnet_e58758 - phase0 afterProcessEpoch 96.628 ms/op 95.524 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.1756 ms/op 1.2161 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4820 ms/op 1.4597 ms/op 1.02
altair processInactivityUpdates - 250000 normalcase 24.878 ms/op 22.872 ms/op 1.09
altair processInactivityUpdates - 250000 worstcase 24.800 ms/op 22.782 ms/op 1.09
phase0 processRegistryUpdates - 250000 normalcase 6.9440 us/op 6.2040 us/op 1.12
phase0 processRegistryUpdates - 250000 badcase_full_deposits 242.62 us/op 235.60 us/op 1.03
phase0 processRegistryUpdates - 250000 worstcase 0.5 128.92 ms/op 112.91 ms/op 1.14
altair processRewardsAndPenalties - 250000 normalcase 69.214 ms/op 54.278 ms/op 1.28
altair processRewardsAndPenalties - 250000 worstcase 71.459 ms/op 54.773 ms/op 1.30
phase0 getAttestationDeltas - 250000 normalcase 6.9265 ms/op 6.3509 ms/op 1.09
phase0 getAttestationDeltas - 250000 worstcase 6.6517 ms/op 6.4087 ms/op 1.04
phase0 processSlashings - 250000 worstcase 3.6123 ms/op 3.4977 ms/op 1.03
altair processSyncCommitteeUpdates - 250000 176.10 ms/op 176.58 ms/op 1.00
BeaconState.hashTreeRoot - No change 317.00 ns/op 258.00 ns/op 1.23
BeaconState.hashTreeRoot - 1 full validator 53.530 us/op 49.428 us/op 1.08
BeaconState.hashTreeRoot - 32 full validator 560.95 us/op 485.40 us/op 1.16
BeaconState.hashTreeRoot - 512 full validator 5.7925 ms/op 4.8286 ms/op 1.20
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 62.475 us/op 63.269 us/op 0.99
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 948.62 us/op 856.36 us/op 1.11
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.399 ms/op 10.813 ms/op 1.05
BeaconState.hashTreeRoot - 1 balances 50.708 us/op 46.021 us/op 1.10
BeaconState.hashTreeRoot - 32 balances 465.02 us/op 433.92 us/op 1.07
BeaconState.hashTreeRoot - 512 balances 4.3595 ms/op 4.2742 ms/op 1.02
BeaconState.hashTreeRoot - 250000 balances 80.571 ms/op 71.441 ms/op 1.13
aggregationBits - 2048 els - zipIndexesInBitList 16.708 us/op 16.482 us/op 1.01
regular array get 100000 times 33.246 us/op 41.488 us/op 0.80
wrappedArray get 100000 times 33.272 us/op 32.335 us/op 1.03
arrayWithProxy get 100000 times 16.451 ms/op 15.627 ms/op 1.05
ssz.Root.equals 583.00 ns/op 531.00 ns/op 1.10
byteArrayEquals 546.00 ns/op 525.00 ns/op 1.04
shuffle list - 16384 els 6.9656 ms/op 6.6013 ms/op 1.06
shuffle list - 250000 els 103.11 ms/op 96.839 ms/op 1.06
processSlot - 1 slots 8.6690 us/op 8.3860 us/op 1.03
processSlot - 32 slots 1.4297 ms/op 1.2915 ms/op 1.11
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 198.60 us/op 191.96 us/op 1.03
getCommitteeAssignments - req 1 vs - 250000 vc 2.9703 ms/op 2.8271 ms/op 1.05
getCommitteeAssignments - req 100 vs - 250000 vc 4.3157 ms/op 4.0807 ms/op 1.06
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5911 ms/op 4.4202 ms/op 1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.8100 ns/op 4.5000 ns/op 1.07
state getBlockRootAtSlot - 250000 vs - 7PWei 550.21 ns/op 558.27 ns/op 0.99
computeProposers - vc 250000 10.617 ms/op 10.233 ms/op 1.04
computeEpochShuffling - vc 250000 102.61 ms/op 101.89 ms/op 1.01
getNextSyncCommittee - vc 250000 179.92 ms/op 167.61 ms/op 1.07

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 73ea3f6 into unstable Feb 14, 2023
@wemeetagain wemeetagain deleted the dapplion/sync-committee-metrics branch February 14, 2023 15:51
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Metric to track sync committee members
2 participants
@dapplion @wemeetagain