Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve UnknownBlock sync #4134

Merged
merged 2 commits into from
Jun 9, 2022
Merged

Improve UnknownBlock sync #4134

merged 2 commits into from
Jun 9, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jun 9, 2022

Motivation

UnknownBlock sync keep downloading blocks if the parent is unknown, we should have some conditions to stop this

Description

UnknownBlock sync: We should only download non-finalized blocks, penalize peers if downloaded blocks is before our finalized slot because it means that the chain we're downloading is conflicting to our finalized chain

For example

0 - 1 - ... - n  -  finalizedSlot
               \
                \
               parent 1 - parent 2 - ... - unknownParent block

@github-actions
Copy link
Contributor

github-actions bot commented Jun 9, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 60d062b Previous: a3e286f Ratio
altair processAttestation - 250000 vs - 7PWei normalcase 3.2125 ms/op 3.1174 ms/op 1.03
altair processAttestation - 250000 vs - 7PWei worstcase 5.2342 ms/op 5.4981 ms/op 0.95
altair processAttestation - setStatus - 1/6 committees join 176.02 us/op 198.87 us/op 0.89
altair processAttestation - setStatus - 1/3 committees join 346.04 us/op 380.65 us/op 0.91
altair processAttestation - setStatus - 1/2 committees join 492.37 us/op 474.59 us/op 1.04
altair processAttestation - setStatus - 2/3 committees join 647.70 us/op 685.12 us/op 0.95
altair processAttestation - setStatus - 4/5 committees join 905.23 us/op 948.34 us/op 0.95
altair processAttestation - setStatus - 100% committees join 1.0952 ms/op 997.26 us/op 1.10
altair processBlock - 250000 vs - 7PWei normalcase 23.234 ms/op 24.796 ms/op 0.94
altair processBlock - 250000 vs - 7PWei normalcase hashState 32.646 ms/op 32.349 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 77.126 ms/op 72.032 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase hashState 97.065 ms/op 88.732 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 3.3373 ms/op 3.1733 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei worstcase 51.311 ms/op 40.707 ms/op 1.26
altair processEth1Data - 250000 vs - 7PWei normalcase 709.54 us/op 703.20 us/op 1.01
Tree 40 250000 create 755.52 ms/op 749.15 ms/op 1.01
Tree 40 250000 get(125000) 232.72 ns/op 287.59 ns/op 0.81
Tree 40 250000 set(125000) 2.5280 us/op 2.2412 us/op 1.13
Tree 40 250000 toArray() 28.536 ms/op 29.418 ms/op 0.97
Tree 40 250000 iterate all - toArray() + loop 28.884 ms/op 29.285 ms/op 0.99
Tree 40 250000 iterate all - get(i) 116.85 ms/op 98.423 ms/op 1.19
MutableVector 250000 create 15.754 ms/op 16.898 ms/op 0.93
MutableVector 250000 get(125000) 10.994 ns/op 14.791 ns/op 0.74
MutableVector 250000 set(125000) 626.80 ns/op 580.61 ns/op 1.08
MutableVector 250000 toArray() 6.7914 ms/op 6.4571 ms/op 1.05
MutableVector 250000 iterate all - toArray() + loop 6.8734 ms/op 6.8925 ms/op 1.00
MutableVector 250000 iterate all - get(i) 2.6125 ms/op 2.8914 ms/op 0.90
Array 250000 create 6.6114 ms/op 6.3872 ms/op 1.04
Array 250000 clone - spread 3.5611 ms/op 3.2634 ms/op 1.09
Array 250000 get(125000) 1.7280 ns/op 1.4550 ns/op 1.19
Array 250000 set(125000) 1.6930 ns/op 1.4160 ns/op 1.20
Array 250000 iterate all - loop 151.00 us/op 170.78 us/op 0.88
effectiveBalanceIncrements clone Uint8Array 300000 67.639 us/op 82.254 us/op 0.82
effectiveBalanceIncrements clone MutableVector 300000 704.00 ns/op 1.0500 us/op 0.67
effectiveBalanceIncrements rw all Uint8Array 300000 248.79 us/op 252.42 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 135.60 ms/op 189.57 ms/op 0.72
phase0 afterProcessEpoch - 250000 vs - 7PWei 196.48 ms/op 172.65 ms/op 1.14
phase0 beforeProcessEpoch - 250000 vs - 7PWei 59.817 ms/op 91.685 ms/op 0.65
altair processEpoch - mainnet_e81889 542.73 ms/op 538.02 ms/op 1.01
mainnet_e81889 - altair beforeProcessEpoch 84.121 ms/op 93.043 ms/op 0.90
mainnet_e81889 - altair processJustificationAndFinalization 16.890 us/op 21.871 us/op 0.77
mainnet_e81889 - altair processInactivityUpdates 9.7407 ms/op 11.003 ms/op 0.89
mainnet_e81889 - altair processRewardsAndPenalties 112.90 ms/op 86.844 ms/op 1.30
mainnet_e81889 - altair processRegistryUpdates 2.7520 us/op 3.5760 us/op 0.77
mainnet_e81889 - altair processSlashings 510.00 ns/op 608.00 ns/op 0.84
mainnet_e81889 - altair processEth1DataReset 598.00 ns/op 879.00 ns/op 0.68
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3264 ms/op 2.2234 ms/op 1.05
mainnet_e81889 - altair processSlashingsReset 4.8920 us/op 6.1730 us/op 0.79
mainnet_e81889 - altair processRandaoMixesReset 4.2060 us/op 7.3370 us/op 0.57
mainnet_e81889 - altair processHistoricalRootsUpdate 585.00 ns/op 852.00 ns/op 0.69
mainnet_e81889 - altair processParticipationFlagUpdates 1.9990 us/op 3.4250 us/op 0.58
mainnet_e81889 - altair processSyncCommitteeUpdates 570.00 ns/op 948.00 ns/op 0.60
mainnet_e81889 - altair afterProcessEpoch 198.57 ms/op 206.56 ms/op 0.96
phase0 processEpoch - mainnet_e58758 474.39 ms/op 549.85 ms/op 0.86
mainnet_e58758 - phase0 beforeProcessEpoch 178.62 ms/op 216.21 ms/op 0.83
mainnet_e58758 - phase0 processJustificationAndFinalization 15.071 us/op 17.471 us/op 0.86
mainnet_e58758 - phase0 processRewardsAndPenalties 97.852 ms/op 100.28 ms/op 0.98
mainnet_e58758 - phase0 processRegistryUpdates 8.3700 us/op 11.680 us/op 0.72
mainnet_e58758 - phase0 processSlashings 548.00 ns/op 801.00 ns/op 0.68
mainnet_e58758 - phase0 processEth1DataReset 575.00 ns/op 776.00 ns/op 0.74
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.8556 ms/op 1.8566 ms/op 1.00
mainnet_e58758 - phase0 processSlashingsReset 3.7910 us/op 6.0700 us/op 0.62
mainnet_e58758 - phase0 processRandaoMixesReset 3.9810 us/op 6.1540 us/op 0.65
mainnet_e58758 - phase0 processHistoricalRootsUpdate 597.00 ns/op 908.00 ns/op 0.66
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.4800 us/op 4.0720 us/op 0.85
mainnet_e58758 - phase0 afterProcessEpoch 166.49 ms/op 144.07 ms/op 1.16
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.2544 ms/op 2.9058 ms/op 0.78
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.1366 ms/op 3.1790 ms/op 0.67
altair processInactivityUpdates - 250000 normalcase 21.908 ms/op 24.956 ms/op 0.88
altair processInactivityUpdates - 250000 worstcase 31.187 ms/op 24.818 ms/op 1.26
phase0 processRegistryUpdates - 250000 normalcase 6.2330 us/op 9.3970 us/op 0.66
phase0 processRegistryUpdates - 250000 badcase_full_deposits 376.31 us/op 397.44 us/op 0.95
phase0 processRegistryUpdates - 250000 worstcase 0.5 178.17 ms/op 196.94 ms/op 0.90
altair processRewardsAndPenalties - 250000 normalcase 92.444 ms/op 79.423 ms/op 1.16
altair processRewardsAndPenalties - 250000 worstcase 93.464 ms/op 102.17 ms/op 0.91
phase0 getAttestationDeltas - 250000 normalcase 12.455 ms/op 12.411 ms/op 1.00
phase0 getAttestationDeltas - 250000 worstcase 12.559 ms/op 12.384 ms/op 1.01
phase0 processSlashings - 250000 worstcase 5.0732 ms/op 5.3509 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 295.09 ms/op 248.78 ms/op 1.19
BeaconState.hashTreeRoot - No change 656.00 ns/op 498.00 ns/op 1.32
BeaconState.hashTreeRoot - 1 full validator 66.430 us/op 57.073 us/op 1.16
BeaconState.hashTreeRoot - 32 full validator 766.39 us/op 575.73 us/op 1.33
BeaconState.hashTreeRoot - 512 full validator 7.7209 ms/op 5.7930 ms/op 1.33
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 88.591 us/op 78.482 us/op 1.13
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2502 ms/op 957.68 us/op 1.31
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.048 ms/op 16.040 ms/op 1.06
BeaconState.hashTreeRoot - 1 balances 74.105 us/op 62.359 us/op 1.19
BeaconState.hashTreeRoot - 32 balances 631.58 us/op 548.09 us/op 1.15
BeaconState.hashTreeRoot - 512 balances 6.4914 ms/op 5.5092 ms/op 1.18
BeaconState.hashTreeRoot - 250000 balances 98.395 ms/op 80.621 ms/op 1.22
aggregationBits - 2048 els - zipIndexesInBitList 29.092 us/op 31.654 us/op 0.92
regular array get 100000 times 60.689 us/op 67.449 us/op 0.90
wrappedArray get 100000 times 61.343 us/op 67.429 us/op 0.91
arrayWithProxy get 100000 times 28.567 ms/op 30.487 ms/op 0.94
ssz.Root.equals 563.00 ns/op 500.00 ns/op 1.13
byteArrayEquals 565.00 ns/op 501.00 ns/op 1.13
shuffle list - 16384 els 11.579 ms/op 11.001 ms/op 1.05
shuffle list - 250000 els 177.96 ms/op 162.20 ms/op 1.10
processSlot - 1 slots 13.969 us/op 12.588 us/op 1.11
processSlot - 32 slots 2.0102 ms/op 1.7231 ms/op 1.17
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 381.38 us/op 387.46 us/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 5.4895 ms/op 5.3251 ms/op 1.03
getCommitteeAssignments - req 100 vs - 250000 vc 8.0029 ms/op 7.3031 ms/op 1.10
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5758 ms/op 7.7721 ms/op 1.10
computeProposers - vc 250000 17.686 ms/op 16.042 ms/op 1.10
computeEpochShuffling - vc 250000 172.22 ms/op 158.83 ms/op 1.08
getNextSyncCommittee - vc 250000 290.79 ms/op 266.31 ms/op 1.09
pass gossip attestations to forkchoice per slot 2.9592 ms/op 3.1170 ms/op 0.95
computeDeltas 3.3015 ms/op 3.0653 ms/op 1.08
computeProposerBoostScoreFromBalances 813.53 us/op 907.80 us/op 0.90
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.8826 ms/op 2.0269 ms/op 0.93
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 68.030 us/op 75.533 us/op 0.90
BLS verify - blst-native 2.1640 ms/op 1.8589 ms/op 1.16
BLS verifyMultipleSignatures 3 - blst-native 4.4740 ms/op 3.8071 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.6843 ms/op 8.2049 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.159 ms/op 29.722 ms/op 1.18
BLS aggregatePubkeys 32 - blst-native 46.911 us/op 40.156 us/op 1.17
BLS aggregatePubkeys 128 - blst-native 182.27 us/op 154.79 us/op 1.18
getAttestationsForBlock 54.926 ms/op 61.431 ms/op 0.89
isKnown best case - 1 super set check 493.00 ns/op 385.00 ns/op 1.28
isKnown normal case - 2 super set checks 483.00 ns/op 374.00 ns/op 1.29
isKnown worse case - 16 super set checks 489.00 ns/op 376.00 ns/op 1.30
CheckpointStateCache - add get delete 10.537 us/op 9.7750 us/op 1.08
validate gossip signedAggregateAndProof - struct 5.0012 ms/op 3.7619 ms/op 1.33
validate gossip attestation - struct 2.3642 ms/op 1.7944 ms/op 1.32
altair verifyImport mainnet_s3766816:31 6.8856 s/op 5.8533 s/op 1.18
pickEth1Vote - no votes 2.1449 ms/op 1.8799 ms/op 1.14
pickEth1Vote - max votes 24.618 ms/op 22.827 ms/op 1.08
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.504 ms/op 11.149 ms/op 1.21
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.917 ms/op 20.973 ms/op 1.05
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4683 ms/op 1.5348 ms/op 0.96
pickEth1Vote - Eth1Data fastSerialize tree x2048 17.638 ms/op 17.426 ms/op 1.01
bytes32 toHexString 1.1080 us/op 1.0700 us/op 1.04
bytes32 Buffer.toString(hex) 823.00 ns/op 714.00 ns/op 1.15
bytes32 Buffer.toString(hex) from Uint8Array 1.0710 us/op 963.00 ns/op 1.11
bytes32 Buffer.toString(hex) + 0x 836.00 ns/op 714.00 ns/op 1.17
Object access 1 prop 0.43400 ns/op 0.29500 ns/op 1.47
Map access 1 prop 0.34000 ns/op 0.25900 ns/op 1.31
Object get x1000 11.203 ns/op 17.837 ns/op 0.63
Map get x1000 0.96700 ns/op 1.0810 ns/op 0.89
Object set x1000 81.174 ns/op 105.21 ns/op 0.77
Map set x1000 55.786 ns/op 63.934 ns/op 0.87
Return object 10000 times 0.43570 ns/op 0.37480 ns/op 1.16
Throw Error 10000 times 5.9520 us/op 5.9901 us/op 0.99
enrSubnets - fastDeserialize 64 bits 3.0280 us/op 2.4750 us/op 1.22
enrSubnets - ssz BitVector 64 bits 941.00 ns/op 732.00 ns/op 1.29
enrSubnets - fastDeserialize 4 bits 474.00 ns/op 382.00 ns/op 1.24
enrSubnets - ssz BitVector 4 bits 917.00 ns/op 716.00 ns/op 1.28
prioritizePeers score -10:0 att 32-0.1 sync 2-0 89.641 us/op 85.914 us/op 1.04
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 129.21 us/op 122.46 us/op 1.06
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 224.43 us/op 199.79 us/op 1.12
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 337.52 us/op 424.44 us/op 0.80
prioritizePeers score 0:0 att 64-1 sync 4-1 407.68 us/op 462.03 us/op 0.88
RateTracker 1000000 limit, 1 obj count per request 186.40 ns/op 177.23 ns/op 1.05
RateTracker 1000000 limit, 2 obj count per request 136.62 ns/op 132.56 ns/op 1.03
RateTracker 1000000 limit, 4 obj count per request 108.27 ns/op 108.99 ns/op 0.99
RateTracker 1000000 limit, 8 obj count per request 95.328 ns/op 96.951 ns/op 0.98
RateTracker with prune 3.7360 us/op 3.7230 us/op 1.00
array of 16000 items push then shift 51.583 us/op 3.0925 us/op 16.68
LinkedList of 16000 items push then shift 16.066 ns/op 23.981 ns/op 0.67
array of 16000 items push then pop 209.87 ns/op 219.17 ns/op 0.96
LinkedList of 16000 items push then pop 14.119 ns/op 20.617 ns/op 0.68
array of 24000 items push then shift 77.350 us/op 4.5668 us/op 16.94
LinkedList of 24000 items push then shift 20.400 ns/op 24.717 ns/op 0.83
array of 24000 items push then pop 190.72 ns/op 189.37 ns/op 1.01
LinkedList of 24000 items push then pop 15.593 ns/op 20.722 ns/op 0.75
intersect bitArray bitLen 8 10.945 ns/op 11.499 ns/op 0.95
intersect array and set length 8 130.78 ns/op 153.05 ns/op 0.85
intersect bitArray bitLen 128 57.747 ns/op 72.326 ns/op 0.80
intersect array and set length 128 1.8345 us/op 2.1402 us/op 0.86

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review June 9, 2022 08:50
@twoeths twoeths requested a review from a team as a code owner June 9, 2022 08:50
dapplion
dapplion reviewed Jun 9, 2022
View reviewed changes
packages/lodestar/src/sync/unknownBlock.ts Outdated
} else if (parentSlot <= finalizedSlot) {
// the common ancestor of the downloading chain and canonical chain should be at least the finalized slot and
// we should found it through forkchoice. If not, we should penalize all peers sending us this block chain
this.logger.error("Downloaded block parent is before finalized slot", {finalizedSlot, parentSlot});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a serious error, I would log the blockRoot of signedBlock too

@dapplion dapplion merged commit f16a76d into unstable Jun 9, 2022
@dapplion dapplion deleted the tuyen/improve-unknown-block-sync branch June 9, 2022 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@twoeths @wemeetagain @dapplion