Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EIP-4844: Handle BlobsSidecar #4882

Merged
merged 1 commit into from
Dec 7, 2022
Merged

EIP-4844: Handle BlobsSidecar #4882

merged 1 commit into from
Dec 7, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 7, 2022

Motivation

Description

Complete handling logic for BlobsSidecar

  • On gossip, handle BlockInput and call processBlock
  • On block verify conditionally run validateBlobsSidecar, skipping for gossip since it's already verified
  • On block import persist BlobsSidecar to db

@dapplion dapplion added the spec-deneb 🐡 Issues targeting the Deneb spec version label Dec 7, 2022
@dapplion dapplion requested a review from a team as a code owner December 7, 2022 09:09
@github-actions
Copy link
Contributor

github-actions bot commented Dec 7, 2022

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: ce4d0eb Previous: 4115f67 Ratio
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.4030 us/op 23.186 us/op 0.28
Full benchmark results
Benchmark suite Current: ce4d0eb Previous: 4115f67 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.7175 ms/op 2.3522 ms/op 0.73
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 62.459 us/op 88.916 us/op 0.70
BLS verify - blst-native 2.1686 ms/op 2.7517 ms/op 0.79
BLS verifyMultipleSignatures 3 - blst-native 4.4949 ms/op 5.5217 ms/op 0.81
BLS verifyMultipleSignatures 8 - blst-native 9.7051 ms/op 12.040 ms/op 0.81
BLS verifyMultipleSignatures 32 - blst-native 35.260 ms/op 40.842 ms/op 0.86
BLS aggregatePubkeys 32 - blst-native 46.698 us/op 55.862 us/op 0.84
BLS aggregatePubkeys 128 - blst-native 182.24 us/op 238.52 us/op 0.76
getAttestationsForBlock 79.560 ms/op 119.79 ms/op 0.66
isKnown best case - 1 super set check 504.00 ns/op 508.00 ns/op 0.99
isKnown normal case - 2 super set checks 483.00 ns/op 493.00 ns/op 0.98
isKnown worse case - 16 super set checks 487.00 ns/op 475.00 ns/op 1.03
CheckpointStateCache - add get delete 8.5640 us/op 10.883 us/op 0.79
validate gossip signedAggregateAndProof - struct 5.0448 ms/op 6.0945 ms/op 0.83
validate gossip attestation - struct 2.3712 ms/op 2.8254 ms/op 0.84
pickEth1Vote - no votes 2.1356 ms/op 2.6825 ms/op 0.80
pickEth1Vote - max votes 19.641 ms/op 22.183 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.100 ms/op 14.179 ms/op 0.92
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.494 ms/op 24.148 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5654 ms/op 1.8585 ms/op 0.84
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.629 ms/op 15.190 ms/op 0.83
bytes32 toHexString 968.00 ns/op 1.2230 us/op 0.79
bytes32 Buffer.toString(hex) 730.00 ns/op 799.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 1.0450 us/op 1.0640 us/op 0.98
bytes32 Buffer.toString(hex) + 0x 733.00 ns/op 778.00 ns/op 0.94
Object access 1 prop 0.37300 ns/op 0.42700 ns/op 0.87
Map access 1 prop 0.30900 ns/op 0.32900 ns/op 0.94
Object get x1000 11.225 ns/op 15.740 ns/op 0.71
Map get x1000 0.92400 ns/op 0.91500 ns/op 1.01
Object set x1000 79.004 ns/op 120.88 ns/op 0.65
Map set x1000 48.584 ns/op 79.547 ns/op 0.61
Return object 10000 times 0.43870 ns/op 0.41480 ns/op 1.06
Throw Error 10000 times 5.9919 us/op 7.1478 us/op 0.84
fastMsgIdFn sha256 / 200 bytes 4.9100 us/op 5.2810 us/op 0.93
fastMsgIdFn h32 xxhash / 200 bytes 626.00 ns/op 616.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 200 bytes 790.00 ns/op 890.00 ns/op 0.89
fastMsgIdFn sha256 / 1000 bytes 15.539 us/op 16.590 us/op 0.94
fastMsgIdFn h32 xxhash / 1000 bytes 806.00 ns/op 804.00 ns/op 1.00
fastMsgIdFn h64 xxhash / 1000 bytes 914.00 ns/op 975.00 ns/op 0.94
fastMsgIdFn sha256 / 10000 bytes 134.03 us/op 153.73 us/op 0.87
fastMsgIdFn h32 xxhash / 10000 bytes 2.6310 us/op 2.8630 us/op 0.92
fastMsgIdFn h64 xxhash / 10000 bytes 1.9470 us/op 2.0510 us/op 0.95
enrSubnets - fastDeserialize 64 bits 2.7670 us/op 2.9950 us/op 0.92
enrSubnets - ssz BitVector 64 bits 854.00 ns/op 806.00 ns/op 1.06
enrSubnets - fastDeserialize 4 bits 391.00 ns/op 475.00 ns/op 0.82
enrSubnets - ssz BitVector 4 bits 848.00 ns/op 886.00 ns/op 0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0 87.660 us/op 109.49 us/op 0.80
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.26 us/op 152.52 us/op 0.86
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 208.22 us/op 287.05 us/op 0.73
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 343.98 us/op 580.28 us/op 0.59
prioritizePeers score 0:0 att 64-1 sync 4-1 438.94 us/op 615.01 us/op 0.71
RateTracker 1000000 limit, 1 obj count per request 182.55 ns/op 206.50 ns/op 0.88
RateTracker 1000000 limit, 2 obj count per request 133.96 ns/op 156.96 ns/op 0.85
RateTracker 1000000 limit, 4 obj count per request 109.08 ns/op 131.21 ns/op 0.83
RateTracker 1000000 limit, 8 obj count per request 96.374 ns/op 111.18 ns/op 0.87
RateTracker with prune 3.9410 us/op 5.3770 us/op 0.73
array of 16000 items push then shift 51.610 us/op 5.0871 us/op 10.15
LinkedList of 16000 items push then shift 12.344 ns/op 19.017 ns/op 0.65
array of 16000 items push then pop 204.86 ns/op 248.49 ns/op 0.82
LinkedList of 16000 items push then pop 12.081 ns/op 18.341 ns/op 0.66
array of 24000 items push then shift 77.392 us/op 7.5670 us/op 10.23
LinkedList of 24000 items push then shift 12.855 ns/op 19.743 ns/op 0.65
array of 24000 items push then pop 192.19 ns/op 249.49 ns/op 0.77
LinkedList of 24000 items push then pop 12.409 ns/op 17.892 ns/op 0.69
intersect bitArray bitLen 8 11.825 ns/op 12.097 ns/op 0.98
intersect array and set length 8 145.33 ns/op 199.97 ns/op 0.73
intersect bitArray bitLen 128 55.878 ns/op 69.326 ns/op 0.81
intersect array and set length 128 1.8088 us/op 2.5573 us/op 0.71
Buffer.concat 32 items 1.7850 ns/op 2.6230 ns/op 0.68
pass gossip attestations to forkchoice per slot 3.5909 ms/op 4.5404 ms/op 0.79
computeDeltas 4.5510 ms/op 5.5700 ms/op 0.82
computeProposerBoostScoreFromBalances 803.85 us/op 856.30 us/op 0.94
altair processAttestation - 250000 vs - 7PWei normalcase 3.3779 ms/op 5.2350 ms/op 0.65
altair processAttestation - 250000 vs - 7PWei worstcase 5.1797 ms/op 9.2972 ms/op 0.56
altair processAttestation - setStatus - 1/6 committees join 186.40 us/op 235.31 us/op 0.79
altair processAttestation - setStatus - 1/3 committees join 366.65 us/op 450.41 us/op 0.81
altair processAttestation - setStatus - 1/2 committees join 524.78 us/op 656.47 us/op 0.80
altair processAttestation - setStatus - 2/3 committees join 694.87 us/op 910.97 us/op 0.76
altair processAttestation - setStatus - 4/5 committees join 969.16 us/op 1.1653 ms/op 0.83
altair processAttestation - setStatus - 100% committees join 1.1648 ms/op 1.4386 ms/op 0.81
altair processBlock - 250000 vs - 7PWei normalcase 24.629 ms/op 30.195 ms/op 0.82
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.867 ms/op 41.124 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase 86.091 ms/op 103.73 ms/op 0.83
altair processBlock - 250000 vs - 7PWei worstcase hashState 99.090 ms/op 121.24 ms/op 0.82
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1790 ms/op 4.0784 ms/op 0.78
phase0 processBlock - 250000 vs - 7PWei worstcase 50.523 ms/op 61.153 ms/op 0.83
altair processEth1Data - 250000 vs - 7PWei normalcase 707.15 us/op 1.0219 ms/op 0.69
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.4030 us/op 23.186 us/op 0.28
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 21.103 us/op 48.326 us/op 0.44
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.4650 us/op 27.610 us/op 0.34
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 8.6920 us/op 22.435 us/op 0.39
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 98.279 us/op 152.19 us/op 0.65
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 936.36 us/op 1.2901 ms/op 0.73
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 12.312 ms/op 17.251 ms/op 0.71
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 22.569 ms/op 27.341 ms/op 0.83
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 72.049 ms/op 88.937 ms/op 0.81
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 40.797 ms/op 55.331 ms/op 0.74
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 117.59 ms/op 144.32 ms/op 0.81
Tree 40 250000 create 699.23 ms/op 1.0225 s/op 0.68
Tree 40 250000 get(125000) 275.80 ns/op 334.77 ns/op 0.82
Tree 40 250000 set(125000) 2.0799 us/op 3.0285 us/op 0.69
Tree 40 250000 toArray() 26.789 ms/op 33.783 ms/op 0.79
Tree 40 250000 iterate all - toArray() + loop 27.179 ms/op 35.296 ms/op 0.77
Tree 40 250000 iterate all - get(i) 108.31 ms/op 126.14 ms/op 0.86
MutableVector 250000 create 12.786 ms/op 16.614 ms/op 0.77
MutableVector 250000 get(125000) 10.715 ns/op 13.525 ns/op 0.79
MutableVector 250000 set(125000) 567.35 ns/op 845.46 ns/op 0.67
MutableVector 250000 toArray() 5.9861 ms/op 7.3368 ms/op 0.82
MutableVector 250000 iterate all - toArray() + loop 6.3276 ms/op 7.6059 ms/op 0.83
MutableVector 250000 iterate all - get(i) 2.7228 ms/op 3.4709 ms/op 0.78
Array 250000 create 5.9324 ms/op 7.1322 ms/op 0.83
Array 250000 clone - spread 3.3967 ms/op 4.1867 ms/op 0.81
Array 250000 get(125000) 1.5310 ns/op 1.7440 ns/op 0.88
Array 250000 set(125000) 1.5340 ns/op 1.7780 ns/op 0.86
Array 250000 iterate all - loop 153.04 us/op 137.41 us/op 1.11
effectiveBalanceIncrements clone Uint8Array 300000 144.13 us/op 113.17 us/op 1.27
effectiveBalanceIncrements clone MutableVector 300000 655.00 ns/op 1.2770 us/op 0.51
effectiveBalanceIncrements rw all Uint8Array 300000 248.66 us/op 286.55 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 140.44 ms/op 276.22 ms/op 0.51
phase0 afterProcessEpoch - 250000 vs - 7PWei 190.26 ms/op 206.47 ms/op 0.92
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.627 ms/op 76.951 ms/op 0.76
altair processEpoch - mainnet_e81889 549.31 ms/op 660.23 ms/op 0.83
mainnet_e81889 - altair beforeProcessEpoch 119.93 ms/op 156.18 ms/op 0.77
mainnet_e81889 - altair processJustificationAndFinalization 22.786 us/op 72.865 us/op 0.31
mainnet_e81889 - altair processInactivityUpdates 8.7509 ms/op 11.698 ms/op 0.75
mainnet_e81889 - altair processRewardsAndPenalties 77.379 ms/op 105.60 ms/op 0.73
mainnet_e81889 - altair processRegistryUpdates 4.5870 us/op 18.375 us/op 0.25
mainnet_e81889 - altair processSlashings 1.3970 us/op 4.6040 us/op 0.30
mainnet_e81889 - altair processEth1DataReset 1.4190 us/op 4.7720 us/op 0.30
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4395 ms/op 2.4133 ms/op 1.01
mainnet_e81889 - altair processSlashingsReset 7.3020 us/op 23.082 us/op 0.32
mainnet_e81889 - altair processRandaoMixesReset 7.9850 us/op 27.887 us/op 0.29
mainnet_e81889 - altair processHistoricalRootsUpdate 945.00 ns/op 4.1220 us/op 0.23
mainnet_e81889 - altair processParticipationFlagUpdates 3.4290 us/op 18.535 us/op 0.19
mainnet_e81889 - altair processSyncCommitteeUpdates 946.00 ns/op 3.5830 us/op 0.26
mainnet_e81889 - altair afterProcessEpoch 201.89 ms/op 211.89 ms/op 0.95
phase0 processEpoch - mainnet_e58758 482.92 ms/op 626.25 ms/op 0.77
mainnet_e58758 - phase0 beforeProcessEpoch 177.44 ms/op 285.78 ms/op 0.62
mainnet_e58758 - phase0 processJustificationAndFinalization 29.666 us/op 68.988 us/op 0.43
mainnet_e58758 - phase0 processRewardsAndPenalties 71.755 ms/op 157.13 ms/op 0.46
mainnet_e58758 - phase0 processRegistryUpdates 14.548 us/op 36.968 us/op 0.39
mainnet_e58758 - phase0 processSlashings 1.2100 us/op 3.7740 us/op 0.32
mainnet_e58758 - phase0 processEth1DataReset 1.0850 us/op 3.8180 us/op 0.28
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0737 ms/op 2.4555 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 4.6960 us/op 20.997 us/op 0.22
mainnet_e58758 - phase0 processRandaoMixesReset 7.4220 us/op 28.811 us/op 0.26
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2700 us/op 4.5820 us/op 0.28
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.5110 us/op 27.352 us/op 0.13
mainnet_e58758 - phase0 afterProcessEpoch 167.70 ms/op 174.31 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.2058 ms/op 2.3961 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.4152 ms/op 2.8524 ms/op 0.85
altair processInactivityUpdates - 250000 normalcase 38.602 ms/op 52.093 ms/op 0.74
altair processInactivityUpdates - 250000 worstcase 39.681 ms/op 67.390 ms/op 0.59
phase0 processRegistryUpdates - 250000 normalcase 13.208 us/op 33.216 us/op 0.40
phase0 processRegistryUpdates - 250000 badcase_full_deposits 585.11 us/op 510.75 us/op 1.15
phase0 processRegistryUpdates - 250000 worstcase 0.5 176.70 ms/op 245.52 ms/op 0.72
altair processRewardsAndPenalties - 250000 normalcase 108.36 ms/op 151.09 ms/op 0.72
altair processRewardsAndPenalties - 250000 worstcase 72.877 ms/op 98.810 ms/op 0.74
phase0 getAttestationDeltas - 250000 normalcase 11.971 ms/op 15.104 ms/op 0.79
phase0 getAttestationDeltas - 250000 worstcase 12.247 ms/op 15.673 ms/op 0.78
phase0 processSlashings - 250000 worstcase 5.6916 ms/op 7.2596 ms/op 0.78
altair processSyncCommitteeUpdates - 250000 286.60 ms/op 342.69 ms/op 0.84
BeaconState.hashTreeRoot - No change 534.00 ns/op 740.00 ns/op 0.72
BeaconState.hashTreeRoot - 1 full validator 73.206 us/op 81.661 us/op 0.90
BeaconState.hashTreeRoot - 32 full validator 651.67 us/op 840.81 us/op 0.78
BeaconState.hashTreeRoot - 512 full validator 8.4354 ms/op 7.7104 ms/op 1.09
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 85.110 us/op 129.14 us/op 0.66
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2737 ms/op 1.5287 ms/op 0.83
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.464 ms/op 18.943 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 63.193 us/op 77.637 us/op 0.81
BeaconState.hashTreeRoot - 32 balances 618.17 us/op 784.71 us/op 0.79
BeaconState.hashTreeRoot - 512 balances 6.0828 ms/op 6.9252 ms/op 0.88
BeaconState.hashTreeRoot - 250000 balances 101.12 ms/op 116.06 ms/op 0.87
aggregationBits - 2048 els - zipIndexesInBitList 27.039 us/op 37.999 us/op 0.71
regular array get 100000 times 62.121 us/op 54.712 us/op 1.14
wrappedArray get 100000 times 60.927 us/op 56.565 us/op 1.08
arrayWithProxy get 100000 times 28.047 ms/op 33.007 ms/op 0.85
ssz.Root.equals 472.00 ns/op 562.00 ns/op 0.84
byteArrayEquals 454.00 ns/op 575.00 ns/op 0.79
shuffle list - 16384 els 11.802 ms/op 12.242 ms/op 0.96
shuffle list - 250000 els 172.55 ms/op 173.84 ms/op 0.99
processSlot - 1 slots 13.151 us/op 19.449 us/op 0.68
processSlot - 32 slots 1.9512 ms/op 2.5994 ms/op 0.75
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 402.65 us/op 578.32 us/op 0.70
getCommitteeAssignments - req 1 vs - 250000 vc 5.3754 ms/op 5.5800 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 7.8682 ms/op 7.8092 ms/op 1.01
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4041 ms/op 8.7725 ms/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.8900 ns/op 9.4900 ns/op 0.83
state getBlockRootAtSlot - 250000 vs - 7PWei 951.89 ns/op 1.3194 us/op 0.72
computeProposers - vc 250000 16.687 ms/op 21.538 ms/op 0.77
computeEpochShuffling - vc 250000 175.18 ms/op 189.10 ms/op 0.93
getNextSyncCommittee - vc 250000 279.25 ms/op 353.99 ms/op 0.79

by benchmarkbot/action

g11tech
g11tech approved these changes Dec 7, 2022
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@g11tech g11tech merged commit 0b31818 into unstable Dec 7, 2022
@g11tech g11tech deleted the dapplion/eip-4844-gossip-handler branch December 7, 2022 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@g11tech g11tech g11tech approved these changes

Assignees
No one assigned
Labels
spec-deneb 🐡 Issues targeting the Deneb spec version
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dapplion @g11tech