Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid getTotalBalance() #3096

Merged
merged 2 commits into from
Sep 9, 2021
Merged

Avoid getTotalBalance() #3096

merged 2 commits into from
Sep 9, 2021

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Sep 9, 2021

Motivation

  • getTotalBalance is called per finalized checkpoint, basically it means per epoch
  • getTotalBalance is very slow, it's >3s in the below profiler in my local environment (and usually the server is 2x slower than my environment)
    Screen Shot 2021-09-09 at 15 52 20

Description

  • totalActiveBalanceByIncrement is available for epochProcess and altair, we should calculate, use it and cache it in EpochContext for both phase0 and altair

@codeclimate
Copy link

codeclimate bot commented Sep 9, 2021

Code Climate has analyzed commit 68b968d and detected 0 issues on this pull request.

View more on Code Climate.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2021

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9f1bc6d Previous: 22fb0fe Ratio
BeaconState.hashTreeRoot - No change 705.00 ns/op 865.00 ns/op 0.82
BeaconState.hashTreeRoot - 1 full validator 89.603 us/op 95.639 us/op 0.94
BeaconState.hashTreeRoot - 32 full validator 1.2562 ms/op 1.3235 ms/op 0.95
BeaconState.hashTreeRoot - 512 full validator 16.048 ms/op 17.650 ms/op 0.91
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 86.185 us/op 93.835 us/op 0.92
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2544 ms/op 1.2998 ms/op 0.97
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.679 ms/op 22.043 ms/op 0.71
BeaconState.hashTreeRoot - 1 balances 55.750 us/op 65.539 us/op 0.85
BeaconState.hashTreeRoot - 32 balances 520.69 us/op 567.75 us/op 0.92
BeaconState.hashTreeRoot - 512 balances 5.0754 ms/op 5.8970 ms/op 0.86
BeaconState.hashTreeRoot - 250000 balances 89.621 ms/op 107.29 ms/op 0.84
processSlot - 1 slots 53.072 us/op 47.410 us/op 1.12
processSlot - 32 slots 2.5569 ms/op 2.6186 ms/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 4.8279 ms/op 6.2246 ms/op 0.78
getCommitteeAssignments - req 100 vs - 250000 vc 6.7192 ms/op 8.6104 ms/op 0.78
getCommitteeAssignments - req 1000 vs - 250000 vc 6.9164 ms/op 9.2400 ms/op 0.75
computeProposers - vc 250000 26.408 ms/op 25.218 ms/op 1.05
computeEpochShuffling - vc 250000 193.37 ms/op 230.19 ms/op 0.84
getNextSyncCommittee - vc 250000 416.16 ms/op 413.44 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 103.55 ms/op 94.669 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase 273.53 ms/op 244.27 ms/op 1.12
altair processEpoch - pyrmont_e62330 473.51 ms/op 543.28 ms/op 0.87
pyrmont_e62330 - altair beforeProcessEpoch 104.30 ms/op 112.96 ms/op 0.92
pyrmont_e62330 - altair processJustificationAndFinalization 118.21 us/op 80.592 us/op 1.47
pyrmont_e62330 - altair processInactivityUpdates 67.656 ms/op 84.108 ms/op 0.80
pyrmont_e62330 - altair processRewardsAndPenalties 56.269 ms/op 64.134 ms/op 0.88
pyrmont_e62330 - altair processRegistryUpdates 20.324 us/op 11.387 us/op 1.78
pyrmont_e62330 - altair processSlashings 5.3780 us/op 3.1730 us/op 1.69
pyrmont_e62330 - altair processEth1DataReset 4.6630 us/op 2.9880 us/op 1.56
pyrmont_e62330 - altair processEffectiveBalanceUpdates 6.3876 ms/op 6.4838 ms/op 0.99
pyrmont_e62330 - altair processSlashingsReset 29.149 us/op 15.926 us/op 1.83
pyrmont_e62330 - altair processRandaoMixesReset 41.205 us/op 21.911 us/op 1.88
pyrmont_e62330 - altair processHistoricalRootsUpdate 6.2940 us/op 3.7990 us/op 1.66
pyrmont_e62330 - altair processParticipationFlagUpdates 44.865 ms/op 51.891 ms/op 0.86
pyrmont_e62330 - altair processSyncCommitteeUpdates 4.8750 us/op 2.5810 us/op 1.89
pyrmont_e62330 - altair afterProcessEpoch 110.17 ms/op 138.57 ms/op 0.80
altair processInactivityUpdates - 250000 normalcase 193.90 ms/op 235.96 ms/op 0.82
altair processInactivityUpdates - 250000 worstcase 202.84 ms/op 220.81 ms/op 0.92
altair processParticipationFlagUpdates - 250000 anycase 90.586 ms/op 96.198 ms/op 0.94
altair processRewardsAndPenalties - 250000 normalcase 114.23 ms/op 132.89 ms/op 0.86
altair processRewardsAndPenalties - 250000 worstcase 127.23 ms/op 131.92 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 431.62 ms/op 432.54 ms/op 1.00
Tree 40 250000 create 550.18 ms/op 558.69 ms/op 0.98
Tree 40 250000 get(125000) 248.19 ns/op 321.37 ns/op 0.77
Tree 40 250000 set(125000) 1.5421 us/op 1.6424 us/op 0.94
Tree 40 250000 toArray() 48.658 ms/op 43.425 ms/op 1.12
Tree 40 250000 iterate all - toArray() + loop 41.351 ms/op 43.423 ms/op 0.95
Tree 40 250000 iterate all - get(i) 110.22 ms/op 122.39 ms/op 0.90
MutableVector 250000 create 21.215 ms/op 27.565 ms/op 0.77
MutableVector 250000 get(125000) 13.468 ns/op 15.634 ns/op 0.86
MutableVector 250000 set(125000) 631.15 ns/op 640.78 ns/op 0.98
MutableVector 250000 toArray() 8.6372 ms/op 8.5421 ms/op 1.01
MutableVector 250000 iterate all - toArray() + loop 8.3125 ms/op 8.9717 ms/op 0.93
MutableVector 250000 iterate all - get(i) 3.4179 ms/op 3.8010 ms/op 0.90
Array 250000 create 5.6647 ms/op 6.1746 ms/op 0.92
Array 250000 clone - spread 1.7339 ms/op 2.0806 ms/op 0.83
Array 250000 get(125000) 0.83400 ns/op 1.3300 ns/op 0.63
Array 250000 set(125000) 0.87700 ns/op 1.3170 ns/op 0.67
Array 250000 iterate all - loop 126.28 us/op 201.24 us/op 0.63
aggregationBits - 2048 els - readonlyValues 228.15 us/op 281.40 us/op 0.81
aggregationBits - 2048 els - zipIndexesInBitList 42.777 us/op 45.327 us/op 0.94
ssz.Root.equals 1.5650 us/op 1.6580 us/op 0.94
ssz.Root.equals with valueOf() 1.7440 us/op 1.8720 us/op 0.93
byteArrayEquals with valueOf() 1.7260 us/op 1.8840 us/op 0.92
phase0 processBlock - 250000 vs - 7PWei normalcase 16.044 ms/op 14.344 ms/op 1.12
phase0 processBlock - 250000 vs - 7PWei worstcase 91.313 ms/op 94.030 ms/op 0.97
phase0 afterProcessEpoch - 250000 vs - 7PWei 222.45 ms/op 258.92 ms/op 0.86
phase0 beforeProcessEpoch - 250000 vs - 7PWei 655.26 ms/op 611.36 ms/op 1.07
phase0 processEpoch - mainnet_e58758 804.27 ms/op 885.44 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 476.67 ms/op 482.35 ms/op 0.99
mainnet_e58758 - phase0 processJustificationAndFinalization 105.11 us/op 82.130 us/op 1.28
mainnet_e58758 - phase0 processRewardsAndPenalties 82.517 ms/op 98.862 ms/op 0.83
mainnet_e58758 - phase0 processRegistryUpdates 88.862 us/op 54.231 us/op 1.64
mainnet_e58758 - phase0 processSlashings 5.8990 us/op 3.5730 us/op 1.65
mainnet_e58758 - phase0 processEth1DataReset 5.5510 us/op 3.1190 us/op 1.78
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 9.0098 ms/op 7.0189 ms/op 1.28
mainnet_e58758 - phase0 processSlashingsReset 26.742 us/op 15.617 us/op 1.71
mainnet_e58758 - phase0 processRandaoMixesReset 34.446 us/op 21.143 us/op 1.63
mainnet_e58758 - phase0 processHistoricalRootsUpdate 7.1170 us/op 4.2170 us/op 1.69
mainnet_e58758 - phase0 processParticipationRecordUpdates 29.430 us/op 14.727 us/op 2.00
mainnet_e58758 - phase0 afterProcessEpoch 185.77 ms/op 223.25 ms/op 0.83
phase0 processEffectiveBalanceUpdates - 250000 normalcase 10.073 ms/op 8.3115 ms/op 1.21
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5968 s/op 1.5311 s/op 1.04
phase0 processRegistryUpdates - 250000 normalcase 93.634 us/op 56.423 us/op 1.66
phase0 processRegistryUpdates - 250000 badcase_full_deposits 4.1141 ms/op 3.9454 ms/op 1.04
phase0 processRegistryUpdates - 250000 worstcase 0.5 1.8769 s/op 2.0183 s/op 0.93
phase0 getAttestationDeltas - 250000 normalcase 35.196 ms/op 42.196 ms/op 0.83
phase0 getAttestationDeltas - 250000 worstcase 32.825 ms/op 41.642 ms/op 0.79
phase0 processSlashings - 250000 worstcase 35.790 ms/op 40.934 ms/op 0.87
shuffle list - 16384 els 11.825 ms/op 17.694 ms/op 0.67
shuffle list - 250000 els 170.50 ms/op 224.96 ms/op 0.76
getEffectiveBalances - 250000 vs - 7PWei 10.596 ms/op 13.636 ms/op 0.78
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3461 ms/op 2.6523 ms/op 0.88
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 941.52 us/op 1.2555 ms/op 0.75
BLS verify - blst-native 2.0152 ms/op 2.2320 ms/op 0.90
BLS verifyMultipleSignatures 3 - blst-native 4.6530 ms/op 4.5690 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 9.7335 ms/op 9.8622 ms/op 0.99
BLS verifyMultipleSignatures 32 - blst-native 34.307 ms/op 35.548 ms/op 0.97
BLS aggregatePubkeys 32 - blst-native 47.528 us/op 47.991 us/op 0.99
BLS aggregatePubkeys 128 - blst-native 182.31 us/op 186.10 us/op 0.98
getAttestationsForBlock 78.594 ms/op 89.336 ms/op 0.88
CheckpointStateCache - add get delete 17.746 us/op 19.080 us/op 0.93
validate gossip signedAggregateAndProof - struct 5.4016 ms/op 5.3294 ms/op 1.01
validate gossip signedAggregateAndProof - treeBacked 5.2643 ms/op 5.2656 ms/op 1.00
validate gossip attestation - struct 2.3615 ms/op 2.5076 ms/op 0.94
validate gossip attestation - treeBacked 2.1653 ms/op 2.4925 ms/op 0.87

by benchmarkbot/action

@codecov
Copy link

codecov bot commented Sep 9, 2021

Codecov Report

Merging #3096 (68b968d) into master (392b3d2) will decrease coverage by 1.46%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3096      +/-   ##
==========================================
- Coverage   59.72%   58.26%   -1.47%     
==========================================
  Files         376      379       +3     
  Lines        8958     9416     +458     
  Branches     1320     1412      +92     
==========================================
+ Hits         5350     5486     +136     
- Misses       3225     3544     +319     
- Partials      383      386       +3     

@twoeths twoeths marked this pull request as ready for review September 9, 2021 09:44
@dapplion
Copy link
Contributor

dapplion commented Sep 9, 2021

Oh nice pick, but why is the node computing the weak subjectivity period every epoch? Why does it need that info?

EDIT: Oh, for the lightclient initer

@wemeetagain wemeetagain merged commit 397ce92 into master Sep 9, 2021
@wemeetagain wemeetagain deleted the tuyen/avoid-getTotalBalance branch September 9, 2021 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants