Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check, validate and skip if we got any deposit events already present in DB #3716

Merged
merged 2 commits into from
Feb 7, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Feb 7, 2022

Motivation
Previously, we were checking and throwing error if we got any deposit events which were already in DB. however a "reset" in remote EL/eth1 ' head can cause our tracker to import events which are already in DB causing our tracker to throw error and preventing the node from proposing.

This PR fixes that behavior by checking and validating if we get events which are already in DB. If the event doesn't match up it throws the DUPLICATE_DISTINCT_LOG error as it means either the DB is corrupted or the EL/eth1 did a reorg, both catastrophic.

Closes #3674

Steps to test or reproduce

@codecov
Copy link

codecov bot commented Feb 7, 2022

Codecov Report

Merging #3716 (31a32bf) into master (7f37195) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 31a32bf differs from pull request most recent head 8f1ded3. Consider uploading reports for the commit 8f1ded3 to get more accurate results

@@            Coverage Diff             @@
##           master    #3716      +/-   ##
==========================================
- Coverage   37.16%   37.14%   -0.02%     
==========================================
  Files         321      321              
  Lines        8712     8722      +10     
  Branches     1350     1352       +2     
==========================================
+ Hits         3238     3240       +2     
- Misses       5332     5340       +8     
  Partials      142      142              

@github-actions
Copy link
Contributor

github-actions bot commented Feb 7, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 31a32bf Previous: 14f5284 Ratio
BeaconState.hashTreeRoot - No change 589.00 ns/op 616.00 ns/op 0.96
BeaconState.hashTreeRoot - 1 full validator 124.66 us/op 148.01 us/op 0.84
BeaconState.hashTreeRoot - 32 full validator 1.9298 ms/op 2.2002 ms/op 0.88
BeaconState.hashTreeRoot - 512 full validator 26.619 ms/op 29.850 ms/op 0.89
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 129.90 us/op 154.78 us/op 0.84
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.1233 ms/op 2.3791 ms/op 0.89
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 29.079 ms/op 32.560 ms/op 0.89
BeaconState.hashTreeRoot - 1 balances 88.653 us/op 108.57 us/op 0.82
BeaconState.hashTreeRoot - 32 balances 774.04 us/op 895.10 us/op 0.86
BeaconState.hashTreeRoot - 512 balances 7.9653 ms/op 8.6031 ms/op 0.93
BeaconState.hashTreeRoot - 250000 balances 147.66 ms/op 151.79 ms/op 0.97
processSlot - 1 slots 50.200 us/op 52.298 us/op 0.96
processSlot - 32 slots 2.9576 ms/op 3.2853 ms/op 0.90
getCommitteeAssignments - req 1 vs - 250000 vc 5.2627 ms/op 6.0643 ms/op 0.87
getCommitteeAssignments - req 100 vs - 250000 vc 7.3338 ms/op 8.3339 ms/op 0.88
getCommitteeAssignments - req 1000 vs - 250000 vc 7.8106 ms/op 8.9891 ms/op 0.87
computeProposers - vc 250000 21.978 ms/op 24.309 ms/op 0.90
computeEpochShuffling - vc 250000 190.30 ms/op 211.06 ms/op 0.90
getNextSyncCommittee - vc 250000 359.01 ms/op 391.88 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei normalcase 55.718 ms/op 49.343 ms/op 1.13
altair processAttestation - 250000 vs - 7PWei worstcase 47.618 ms/op 53.491 ms/op 0.89
altair processAttestation - setStatus - 1/6 committees join 13.772 ms/op 13.119 ms/op 1.05
altair processAttestation - setStatus - 1/3 committees join 27.869 ms/op 27.323 ms/op 1.02
altair processAttestation - setStatus - 1/2 committees join 43.106 ms/op 40.375 ms/op 1.07
altair processAttestation - setStatus - 2/3 committees join 57.708 ms/op 54.084 ms/op 1.07
altair processAttestation - setStatus - 4/5 committees join 69.930 ms/op 66.815 ms/op 1.05
altair processAttestation - setStatus - 100% committees join 87.064 ms/op 81.041 ms/op 1.07
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.131 ms/op 13.510 ms/op 1.12
altair processAttestation - updateEpochParticipants - 1/3 committees join 30.329 ms/op 32.180 ms/op 0.94
altair processAttestation - updateEpochParticipants - 1/2 committees join 33.126 ms/op 27.840 ms/op 1.19
altair processAttestation - updateEpochParticipants - 2/3 committees join 29.993 ms/op 29.114 ms/op 1.03
altair processAttestation - updateEpochParticipants - 4/5 committees join 31.404 ms/op 38.711 ms/op 0.81
altair processAttestation - updateEpochParticipants - 100% committees join 39.030 ms/op 32.637 ms/op 1.20
altair processAttestation - updateAllStatus 23.515 ms/op 24.198 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase 49.426 ms/op 49.330 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 125.51 ms/op 134.98 ms/op 0.93
altair processEpoch - mainnet_e81889 1.2144 s/op 1.3283 s/op 0.91
mainnet_e81889 - altair beforeProcessEpoch 307.10 ms/op 322.28 ms/op 0.95
mainnet_e81889 - altair processJustificationAndFinalization 50.641 us/op 79.728 us/op 0.64
mainnet_e81889 - altair processInactivityUpdates 19.369 ms/op 21.285 ms/op 0.91
mainnet_e81889 - altair processRewardsAndPenalties 285.08 ms/op 310.15 ms/op 0.92
mainnet_e81889 - altair processRegistryUpdates 5.3270 us/op 13.515 us/op 0.39
mainnet_e81889 - altair processSlashings 1.4970 us/op 3.7430 us/op 0.40
mainnet_e81889 - altair processEth1DataReset 978.00 ns/op 3.1990 us/op 0.31
mainnet_e81889 - altair processEffectiveBalanceUpdates 12.794 ms/op 13.382 ms/op 0.96
mainnet_e81889 - altair processSlashingsReset 8.9750 us/op 20.458 us/op 0.44
mainnet_e81889 - altair processRandaoMixesReset 11.936 us/op 20.371 us/op 0.59
mainnet_e81889 - altair processHistoricalRootsUpdate 1.4320 us/op 4.8370 us/op 0.30
mainnet_e81889 - altair processParticipationFlagUpdates 173.22 ms/op 184.23 ms/op 0.94
mainnet_e81889 - altair processSyncCommitteeUpdates 980.00 ns/op 3.6400 us/op 0.27
mainnet_e81889 - altair afterProcessEpoch 222.35 ms/op 258.46 ms/op 0.86
altair processInactivityUpdates - 250000 normalcase 91.910 ms/op 74.798 ms/op 1.23
altair processInactivityUpdates - 250000 worstcase 95.325 ms/op 76.869 ms/op 1.24
altair processParticipationFlagUpdates - 250000 anycase 97.139 ms/op 103.71 ms/op 0.94
altair processRewardsAndPenalties - 250000 normalcase 231.58 ms/op 260.95 ms/op 0.89
altair processRewardsAndPenalties - 250000 worstcase 255.62 ms/op 295.07 ms/op 0.87
altair processSyncCommitteeUpdates - 250000 376.31 ms/op 425.99 ms/op 0.88
Tree 40 250000 create 1.0102 s/op 914.33 ms/op 1.10
Tree 40 250000 get(125000) 327.35 ns/op 382.07 ns/op 0.86
Tree 40 250000 set(125000) 2.3044 us/op 2.0475 us/op 1.13
Tree 40 250000 toArray() 42.386 ms/op 49.522 ms/op 0.86
Tree 40 250000 iterate all - toArray() + loop 50.819 ms/op 43.477 ms/op 1.17
Tree 40 250000 iterate all - get(i) 136.93 ms/op 140.57 ms/op 0.97
MutableVector 250000 create 28.920 ms/op 29.639 ms/op 0.98
MutableVector 250000 get(125000) 14.234 ns/op 17.001 ns/op 0.84
MutableVector 250000 set(125000) 666.64 ns/op 609.37 ns/op 1.09
MutableVector 250000 toArray() 9.5476 ms/op 10.269 ms/op 0.93
MutableVector 250000 iterate all - toArray() + loop 9.7192 ms/op 10.772 ms/op 0.90
MutableVector 250000 iterate all - get(i) 3.4135 ms/op 3.9794 ms/op 0.86
Array 250000 create 6.3235 ms/op 6.3763 ms/op 0.99
Array 250000 clone - spread 2.5267 ms/op 2.5696 ms/op 0.98
Array 250000 get(125000) 1.1290 ns/op 1.2150 ns/op 0.93
Array 250000 set(125000) 1.1100 ns/op 1.1920 ns/op 0.93
Array 250000 iterate all - loop 168.42 us/op 193.66 us/op 0.87
aggregationBits - 2048 els - readonlyValues 252.26 us/op 272.27 us/op 0.93
aggregationBits - 2048 els - zipIndexesInBitList 50.093 us/op 48.998 us/op 1.02
regular array get 100000 times 67.425 us/op 78.810 us/op 0.86
wrappedArray get 100000 times 67.541 us/op 76.796 us/op 0.88
arrayWithProxy get 100000 times 28.566 ms/op 32.318 ms/op 0.88
ssz.Root.equals 1.2390 us/op 1.3470 us/op 0.92
ssz.Root.equals with valueOf() 1.5630 us/op 1.5440 us/op 1.01
byteArrayEquals with valueOf() 1.5810 us/op 1.5120 us/op 1.05
phase0 processBlock - 250000 vs - 7PWei normalcase 11.253 ms/op 12.713 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei worstcase 79.734 ms/op 88.329 ms/op 0.90
phase0 afterProcessEpoch - 250000 vs - 7PWei 209.69 ms/op 247.38 ms/op 0.85
phase0 beforeProcessEpoch - 250000 vs - 7PWei 635.24 ms/op 666.25 ms/op 0.95
phase0 processEpoch - mainnet_e58758 878.75 ms/op 943.96 ms/op 0.93
mainnet_e58758 - phase0 beforeProcessEpoch 462.23 ms/op 530.94 ms/op 0.87
mainnet_e58758 - phase0 processJustificationAndFinalization 48.142 us/op 79.457 us/op 0.61
mainnet_e58758 - phase0 processRewardsAndPenalties 166.42 ms/op 156.18 ms/op 1.07
mainnet_e58758 - phase0 processRegistryUpdates 35.798 us/op 60.538 us/op 0.59
mainnet_e58758 - phase0 processSlashings 1.0640 us/op 3.9470 us/op 0.27
mainnet_e58758 - phase0 processEth1DataReset 1.2100 us/op 3.8220 us/op 0.32
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 11.173 ms/op 11.689 ms/op 0.96
mainnet_e58758 - phase0 processSlashingsReset 7.3580 us/op 17.998 us/op 0.41
mainnet_e58758 - phase0 processRandaoMixesReset 10.690 us/op 20.532 us/op 0.52
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4050 us/op 4.6000 us/op 0.31
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.9110 us/op 16.368 us/op 0.48
mainnet_e58758 - phase0 afterProcessEpoch 186.45 ms/op 255.33 ms/op 0.73
phase0 processEffectiveBalanceUpdates - 250000 normalcase 14.848 ms/op 13.134 ms/op 1.13
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5512 s/op 1.5473 s/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 37.589 us/op 55.970 us/op 0.67
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.4577 ms/op 3.1573 ms/op 1.10
phase0 processRegistryUpdates - 250000 worstcase 0.5 1.9472 s/op 1.9429 s/op 1.00
phase0 getAttestationDeltas - 250000 normalcase 77.541 ms/op 85.457 ms/op 0.91
phase0 getAttestationDeltas - 250000 worstcase 77.870 ms/op 85.754 ms/op 0.91
phase0 processSlashings - 250000 worstcase 56.754 ms/op 42.172 ms/op 1.35
shuffle list - 16384 els 12.958 ms/op 15.304 ms/op 0.85
shuffle list - 250000 els 185.16 ms/op 211.15 ms/op 0.88
getEffectiveBalances - 250000 vs - 7PWei 12.516 ms/op 12.251 ms/op 1.02
pass gossip attestations to forkchoice per slot 21.000 ms/op 18.635 ms/op 1.13
computeDeltas 3.7162 ms/op 3.8354 ms/op 0.97
computeProposerBoostScoreFromBalances 337.36 us/op 391.81 us/op 0.86
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.5697 ms/op 2.4306 ms/op 1.06
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 749.15 us/op 795.30 us/op 0.94
BLS verify - blst-native 1.8606 ms/op 2.1899 ms/op 0.85
BLS verifyMultipleSignatures 3 - blst-native 3.8175 ms/op 4.4495 ms/op 0.86
BLS verifyMultipleSignatures 8 - blst-native 8.2354 ms/op 9.3119 ms/op 0.88
BLS verifyMultipleSignatures 32 - blst-native 29.868 ms/op 34.777 ms/op 0.86
BLS aggregatePubkeys 32 - blst-native 40.541 us/op 45.880 us/op 0.88
BLS aggregatePubkeys 128 - blst-native 154.33 us/op 175.43 us/op 0.88
getAttestationsForBlock 92.303 ms/op 94.385 ms/op 0.98
CheckpointStateCache - add get delete 19.731 us/op 21.944 us/op 0.90
validate gossip signedAggregateAndProof - struct 4.4827 ms/op 5.0301 ms/op 0.89
validate gossip signedAggregateAndProof - treeBacked 4.4138 ms/op 5.0645 ms/op 0.87
validate gossip attestation - struct 2.0976 ms/op 2.3927 ms/op 0.88
validate gossip attestation - treeBacked 2.1427 ms/op 2.5267 ms/op 0.85
bytes32 toHexString 1.9070 us/op 1.9110 us/op 1.00
bytes32 Buffer.toString(hex) 676.00 ns/op 806.00 ns/op 0.84
bytes32 Buffer.toString(hex) from Uint8Array 880.00 ns/op 1.0700 us/op 0.82
bytes32 Buffer.toString(hex) + 0x 689.00 ns/op 794.00 ns/op 0.87
Object access 1 prop 0.35300 ns/op 0.36000 ns/op 0.98
Map access 1 prop 0.29100 ns/op 0.31700 ns/op 0.92
Object get x1000 17.868 ns/op 20.197 ns/op 0.88
Map get x1000 1.0880 ns/op 1.0910 ns/op 1.00
Object set x1000 116.25 ns/op 121.23 ns/op 0.96
Map set x1000 69.088 ns/op 72.161 ns/op 0.96
Return object 10000 times 0.36750 ns/op 0.41490 ns/op 0.89
Throw Error 10000 times 5.8559 us/op 6.4944 us/op 0.90
enrSubnets - fastDeserialize 64 bits 1.3730 us/op 1.3620 us/op 1.01
enrSubnets - ssz BitVector 64 bits 16.865 us/op 19.049 us/op 0.89
enrSubnets - fastDeserialize 4 bits 481.00 ns/op 507.00 ns/op 0.95
enrSubnets - ssz BitVector 4 bits 3.0080 us/op 3.5820 us/op 0.84
RateTracker 1000000 limit, 1 obj count per request 186.55 ns/op 214.54 ns/op 0.87
RateTracker 1000000 limit, 2 obj count per request 139.68 ns/op 161.92 ns/op 0.86
RateTracker 1000000 limit, 4 obj count per request 116.60 ns/op 131.95 ns/op 0.88
RateTracker 1000000 limit, 8 obj count per request 105.94 ns/op 115.75 ns/op 0.92
RateTracker with prune 4.7370 us/op 4.6570 us/op 1.02

by benchmarkbot/action

Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really tidy now! Thanks for the logic re-write and the comments ❤️

@wemeetagain wemeetagain merged commit 8d1efce into master Feb 7, 2022
@wemeetagain wemeetagain deleted the g11tech/dupdepositlogfix branch February 7, 2022 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle recurring ETH1_ERROR_DUPLICATE_DISTINCT_LOG
3 participants