Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement produce block v3 and move builder/execution race to beacon #5880

Merged
merged 1 commit into from
Oct 26, 2023
Merged

feat: implement produce block v3 and move builder/execution race to beacon #5880

merged 1 commit into from
Oct 26, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Aug 13, 2023 •
edited by philknows
Loading

Breaking Change

(for annoucements/release notes)

  • builder vs execution block production race has been moved to beacon (from validator) (via both produceBlockV3 and produceBlindedBlock) and hence both validation and beacon will need to be updated
  • validator flag --builder is now deprecated and builder flow activation is solely decided upon --builder.selection
  • See refactor: repurpose --builder flag to alias maxprofit builder.selection and turn builder off by default #6081
    • propose config now also doesn't need enabled flag under builder section. if you do not want to use the builder on a validator that might still be hooked onto a builder enabled beacon, you may provide the builder selection as executiononly (by default its set to maxprofit but default still can be overridden with --builder.selection flag on the validator apart from the proposer config file level overrides) 
           proposer_config:
       '0xa4855c83d868f772a579133d9f23818008417b743e8447e235d8eb78b1d8f8a9f63f98c551beb7de254400f89592314d':
         fee_recipient: '0xbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'
         builder:
           selection: "executiononly"
     default_config:
       fee_recipient: '0xcccccccccccccccccccccccccccccccccccccccc'
       builder:
         selection: "maxprofit"
    • DVTs which are based off on only builder based block production can still continue using --builder.selection builderonly flag on the validator to only trigger block production through the builder url configured on the beacon

Part of

TODO:

  • keep the breaking flags in but hidden and in no-op/backward compatible mode
  • depreacate usage

Implements :

  • switch full and blinded block production to produce block v3
    • add in api routes
    • imp in beacon-node
      • move blinded vs full race to beacon
    • switch validator to use v3
      - [x] move produceblockv2 and produceblindedblock to the standard beacon api reverted after dicusssion in R&D
  • fix tests
    • resolve any tests w.r.t block production
    • move block production race test to beacon node and mock on produceBlockV3
  • house keeping tasks (might need discussion)
    • implement strict fee recipient checks and allow beacon to be configured
    • allow a way to supply builderSelection i.e. with builderOnly
    • flags cleanup
  • breaking change summary (for announcement/release notes benefit)

PS:

this PR doesn't add consensus block value which will be provided in a followup PR

@g11tech g11tech requested a review from a team as a code owner August 13, 2023 13:46
@g11tech g11tech mentioned this pull request Aug 14, 2023
Open
7 tasks
@philknows philknows added this to the v1.11.0 milestone Aug 15, 2023
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 3f85ce9 to 011d1bf Compare August 18, 2023 15:49
@github-actions
Copy link
Contributor

github-actions bot commented Aug 18, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: dc77d5d Previous: 6985cd0 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 968.12 us/op 561.69 us/op 1.72
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 98.671 us/op 84.235 us/op 1.17
BLS verify - blst-native 1.3628 ms/op 1.3978 ms/op 0.97
BLS verifyMultipleSignatures 3 - blst-native 2.8730 ms/op 2.7643 ms/op 1.04
BLS verifyMultipleSignatures 8 - blst-native 6.2874 ms/op 6.0428 ms/op 1.04
BLS verifyMultipleSignatures 32 - blst-native 23.152 ms/op 22.213 ms/op 1.04
BLS verifyMultipleSignatures 64 - blst-native 46.042 ms/op 44.888 ms/op 1.03
BLS verifyMultipleSignatures 128 - blst-native 91.468 ms/op 86.540 ms/op 1.06
BLS deserializing 10000 signatures 955.88 ms/op 920.69 ms/op 1.04
BLS deserializing 100000 signatures 9.4116 s/op 9.2101 s/op 1.02
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.4343 ms/op 1.3645 ms/op 1.05
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5994 ms/op 1.5519 ms/op 1.03
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.5076 ms/op 2.3481 ms/op 1.07
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.6895 ms/op 3.5129 ms/op 1.05
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.5427 ms/op 5.6987 ms/op 1.15
BLS aggregatePubkeys 32 - blst-native 27.731 us/op 26.047 us/op 1.06
BLS aggregatePubkeys 128 - blst-native 107.88 us/op 102.54 us/op 1.05
getAttestationsForBlock 53.058 ms/op 54.230 ms/op 0.98
isKnown best case - 1 super set check 564.00 ns/op 528.00 ns/op 1.07
isKnown normal case - 2 super set checks 564.00 ns/op 526.00 ns/op 1.07
isKnown worse case - 16 super set checks 548.00 ns/op 401.00 ns/op 1.37
CheckpointStateCache - add get delete 6.5880 us/op 6.0120 us/op 1.10
validate api signedAggregateAndProof - struct 3.0235 ms/op 2.8447 ms/op 1.06
validate gossip signedAggregateAndProof - struct 2.9564 ms/op 2.8586 ms/op 1.03
validate gossip attestation - vc 640000 1.4489 ms/op 1.4087 ms/op 1.03
batch validate gossip attestation - vc 640000 - chunk 32 193.11 us/op 169.75 us/op 1.14
batch validate gossip attestation - vc 640000 - chunk 64 178.60 us/op 147.07 us/op 1.21
batch validate gossip attestation - vc 640000 - chunk 128 159.35 us/op 142.11 us/op 1.12
batch validate gossip attestation - vc 640000 - chunk 256 157.96 us/op 138.68 us/op 1.14
pickEth1Vote - no votes 1.4016 ms/op 1.2520 ms/op 1.12
pickEth1Vote - max votes 11.833 ms/op 11.159 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot value x2048 20.186 ms/op 17.595 ms/op 1.15
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 36.638 ms/op 26.377 ms/op 1.39
pickEth1Vote - Eth1Data fastSerialize value x2048 779.67 us/op 618.42 us/op 1.26
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.7849 ms/op 5.3232 ms/op 1.65
bytes32 toHexString 739.00 ns/op 610.00 ns/op 1.21
bytes32 Buffer.toString(hex) 333.00 ns/op 308.00 ns/op 1.08
bytes32 Buffer.toString(hex) from Uint8Array 509.00 ns/op 508.00 ns/op 1.00
bytes32 Buffer.toString(hex) + 0x 350.00 ns/op 294.00 ns/op 1.19
Object access 1 prop 0.21000 ns/op 0.16500 ns/op 1.27
Map access 1 prop 0.17700 ns/op 0.14800 ns/op 1.20
Object get x1000 13.109 ns/op 7.5760 ns/op 1.73
Map get x1000 0.76900 ns/op 0.67500 ns/op 1.14
Object set x1000 74.358 ns/op 55.763 ns/op 1.33
Map set x1000 53.994 ns/op 46.117 ns/op 1.17
Return object 10000 times 0.33700 ns/op 0.24780 ns/op 1.36
Throw Error 10000 times 4.5758 us/op 3.9036 us/op 1.17
fastMsgIdFn sha256 / 200 bytes 3.7070 us/op 3.4170 us/op 1.08
fastMsgIdFn h32 xxhash / 200 bytes 359.00 ns/op 301.00 ns/op 1.19
fastMsgIdFn h64 xxhash / 200 bytes 428.00 ns/op 363.00 ns/op 1.18
fastMsgIdFn sha256 / 1000 bytes 13.186 us/op 11.755 us/op 1.12
fastMsgIdFn h32 xxhash / 1000 bytes 505.00 ns/op 426.00 ns/op 1.19
fastMsgIdFn h64 xxhash / 1000 bytes 470.00 ns/op 438.00 ns/op 1.07
fastMsgIdFn sha256 / 10000 bytes 112.94 us/op 104.96 us/op 1.08
fastMsgIdFn h32 xxhash / 10000 bytes 2.1420 us/op 1.9770 us/op 1.08
fastMsgIdFn h64 xxhash / 10000 bytes 1.5950 us/op 1.3450 us/op 1.19
send data - 1000 256B messages 22.148 ms/op 20.319 ms/op 1.09
send data - 1000 512B messages 30.394 ms/op 27.452 ms/op 1.11
send data - 1000 1024B messages 47.814 ms/op 42.365 ms/op 1.13
send data - 1000 1200B messages 34.810 ms/op 32.876 ms/op 1.06
send data - 1000 2048B messages 37.690 ms/op 32.119 ms/op 1.17
send data - 1000 4096B messages 40.578 ms/op 36.665 ms/op 1.11
send data - 1000 16384B messages 95.118 ms/op 75.521 ms/op 1.26
send data - 1000 65536B messages 340.11 ms/op 318.69 ms/op 1.07
enrSubnets - fastDeserialize 64 bits 1.5880 us/op 1.3580 us/op 1.17
enrSubnets - ssz BitVector 64 bits 500.00 ns/op 460.00 ns/op 1.09
enrSubnets - fastDeserialize 4 bits 212.00 ns/op 200.00 ns/op 1.06
enrSubnets - ssz BitVector 4 bits 507.00 ns/op 476.00 ns/op 1.07
prioritizePeers score -10:0 att 32-0.1 sync 2-0 116.47 us/op 108.35 us/op 1.07
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 149.20 us/op 135.08 us/op 1.10
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 194.03 us/op 180.24 us/op 1.08
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 330.78 us/op 300.70 us/op 1.10
prioritizePeers score 0:0 att 64-1 sync 4-1 397.62 us/op 350.33 us/op 1.14
array of 16000 items push then shift 1.8103 us/op 1.6493 us/op 1.10
LinkedList of 16000 items push then shift 9.9380 ns/op 9.1500 ns/op 1.09
array of 16000 items push then pop 100.97 ns/op 91.462 ns/op 1.10
LinkedList of 16000 items push then pop 10.021 ns/op 8.9490 ns/op 1.12
array of 24000 items push then shift 2.6092 us/op 2.4570 us/op 1.06
LinkedList of 24000 items push then shift 9.9510 ns/op 9.2210 ns/op 1.08
array of 24000 items push then pop 133.94 ns/op 122.76 ns/op 1.09
LinkedList of 24000 items push then pop 9.3500 ns/op 8.9750 ns/op 1.04
intersect bitArray bitLen 8 7.5350 ns/op 7.1040 ns/op 1.06
intersect array and set length 8 70.223 ns/op 56.307 ns/op 1.25
intersect bitArray bitLen 128 35.080 ns/op 32.640 ns/op 1.07
intersect array and set length 128 920.11 ns/op 779.54 ns/op 1.18
bitArray.getTrueBitIndexes() bitLen 128 2.0250 us/op 1.5280 us/op 1.33
bitArray.getTrueBitIndexes() bitLen 248 3.2660 us/op 2.5870 us/op 1.26
bitArray.getTrueBitIndexes() bitLen 512 6.5100 us/op 4.9200 us/op 1.32
Buffer.concat 32 items 1.1390 us/op 962.00 ns/op 1.18
Uint8Array.set 32 items 2.2530 us/op 1.9880 us/op 1.13
Set add up to 64 items then delete first 4.8566 us/op 4.4182 us/op 1.10
OrderedSet add up to 64 items then delete first 6.4702 us/op 5.5563 us/op 1.16
Set add up to 64 items then delete last 5.4464 us/op 4.7073 us/op 1.16
OrderedSet add up to 64 items then delete last 6.6808 us/op 5.7936 us/op 1.15
Set add up to 64 items then delete middle 5.1650 us/op 4.6449 us/op 1.11
OrderedSet add up to 64 items then delete middle 8.1775 us/op 7.0802 us/op 1.15
Set add up to 128 items then delete first 10.898 us/op 9.6421 us/op 1.13
OrderedSet add up to 128 items then delete first 14.300 us/op 12.349 us/op 1.16
Set add up to 128 items then delete last 10.290 us/op 9.2782 us/op 1.11
OrderedSet add up to 128 items then delete last 13.337 us/op 11.899 us/op 1.12
Set add up to 128 items then delete middle 10.219 us/op 9.4615 us/op 1.08
OrderedSet add up to 128 items then delete middle 19.150 us/op 17.402 us/op 1.10
Set add up to 256 items then delete first 21.078 us/op 20.020 us/op 1.05
OrderedSet add up to 256 items then delete first 28.424 us/op 25.813 us/op 1.10
Set add up to 256 items then delete last 20.246 us/op 18.973 us/op 1.07
OrderedSet add up to 256 items then delete last 27.037 us/op 25.220 us/op 1.07
Set add up to 256 items then delete middle 20.374 us/op 19.357 us/op 1.05
OrderedSet add up to 256 items then delete middle 50.746 us/op 48.089 us/op 1.06
transfer serialized Status (84 B) 1.9630 us/op 1.8550 us/op 1.06
copy serialized Status (84 B) 1.6170 us/op 1.5650 us/op 1.03
transfer serialized SignedVoluntaryExit (112 B) 2.0500 us/op 1.9430 us/op 1.06
copy serialized SignedVoluntaryExit (112 B) 1.6990 us/op 1.6380 us/op 1.04
transfer serialized ProposerSlashing (416 B) 2.4770 us/op 2.2770 us/op 1.09
copy serialized ProposerSlashing (416 B) 2.4750 us/op 2.3440 us/op 1.06
transfer serialized Attestation (485 B) 2.4090 us/op 2.4370 us/op 0.99
copy serialized Attestation (485 B) 2.3980 us/op 2.1480 us/op 1.12
transfer serialized AttesterSlashing (33232 B) 2.5250 us/op 2.0990 us/op 1.20
copy serialized AttesterSlashing (33232 B) 7.5700 us/op 5.7040 us/op 1.33
transfer serialized Small SignedBeaconBlock (128000 B) 2.8380 us/op 2.4490 us/op 1.16
copy serialized Small SignedBeaconBlock (128000 B) 20.506 us/op 16.049 us/op 1.28
transfer serialized Avg SignedBeaconBlock (200000 B) 3.8370 us/op 2.8190 us/op 1.36
copy serialized Avg SignedBeaconBlock (200000 B) 28.975 us/op 22.597 us/op 1.28
transfer serialized BlobsSidecar (524380 B) 4.2860 us/op 2.9840 us/op 1.44
copy serialized BlobsSidecar (524380 B) 114.04 us/op 70.908 us/op 1.61
transfer serialized Big SignedBeaconBlock (1000000 B) 4.0290 us/op 2.9430 us/op 1.37
copy serialized Big SignedBeaconBlock (1000000 B) 224.14 us/op 163.38 us/op 1.37
pass gossip attestations to forkchoice per slot 4.1899 ms/op 3.7528 ms/op 1.12
forkChoice updateHead vc 100000 bc 64 eq 0 729.75 us/op 744.41 us/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 0 4.5829 ms/op 5.0487 ms/op 0.91
forkChoice updateHead vc 1000000 bc 64 eq 0 7.5317 ms/op 7.2390 ms/op 1.04
forkChoice updateHead vc 600000 bc 320 eq 0 4.6214 ms/op 4.3669 ms/op 1.06
forkChoice updateHead vc 600000 bc 1200 eq 0 4.8612 ms/op 4.4148 ms/op 1.10
forkChoice updateHead vc 600000 bc 7200 eq 0 6.1309 ms/op 5.4067 ms/op 1.13
forkChoice updateHead vc 600000 bc 64 eq 1000 12.174 ms/op 11.518 ms/op 1.06
forkChoice updateHead vc 600000 bc 64 eq 10000 13.196 ms/op 12.413 ms/op 1.06
forkChoice updateHead vc 600000 bc 64 eq 300000 36.700 ms/op 16.294 ms/op 2.25
computeDeltas 500000 validators 300 proto nodes 6.6854 ms/op 6.4058 ms/op 1.04
computeDeltas 500000 validators 1200 proto nodes 6.5690 ms/op 6.3867 ms/op 1.03
computeDeltas 500000 validators 7200 proto nodes 6.6128 ms/op 6.4193 ms/op 1.03
computeDeltas 750000 validators 300 proto nodes 9.7905 ms/op 9.8418 ms/op 0.99
computeDeltas 750000 validators 1200 proto nodes 9.5601 ms/op 9.6865 ms/op 0.99
computeDeltas 750000 validators 7200 proto nodes 9.3869 ms/op 9.6063 ms/op 0.98
computeDeltas 1400000 validators 300 proto nodes 17.258 ms/op 18.279 ms/op 0.94
computeDeltas 1400000 validators 1200 proto nodes 17.458 ms/op 18.488 ms/op 0.94
computeDeltas 1400000 validators 7200 proto nodes 17.707 ms/op 18.891 ms/op 0.94
computeDeltas 2100000 validators 300 proto nodes 26.770 ms/op 29.097 ms/op 0.92
computeDeltas 2100000 validators 1200 proto nodes 27.264 ms/op 28.395 ms/op 0.96
computeDeltas 2100000 validators 7200 proto nodes 28.361 ms/op 28.611 ms/op 0.99
computeProposerBoostScoreFromBalances 500000 validators 3.5855 ms/op 3.4206 ms/op 1.05
computeProposerBoostScoreFromBalances 750000 validators 3.5861 ms/op 3.5803 ms/op 1.00
computeProposerBoostScoreFromBalances 1400000 validators 4.0751 ms/op 3.5867 ms/op 1.14
computeProposerBoostScoreFromBalances 2100000 validators 3.4313 ms/op 3.8137 ms/op 0.90
altair processAttestation - 250000 vs - 7PWei normalcase 3.4619 ms/op 3.6455 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei worstcase 4.5918 ms/op 5.0208 ms/op 0.91
altair processAttestation - setStatus - 1/6 committees join 152.22 us/op 187.12 us/op 0.81
altair processAttestation - setStatus - 1/3 committees join 298.82 us/op 415.95 us/op 0.72
altair processAttestation - setStatus - 1/2 committees join 398.79 us/op 518.65 us/op 0.77
altair processAttestation - setStatus - 2/3 committees join 500.74 us/op 637.29 us/op 0.79
altair processAttestation - setStatus - 4/5 committees join 695.93 us/op 983.62 us/op 0.71
altair processAttestation - setStatus - 100% committees join 826.71 us/op 1.0515 ms/op 0.79
altair processBlock - 250000 vs - 7PWei normalcase 12.252 ms/op 11.850 ms/op 1.03
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.044 ms/op 42.484 ms/op 0.82
altair processBlock - 250000 vs - 7PWei worstcase 48.016 ms/op 52.584 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase hashState 104.65 ms/op 113.20 ms/op 0.92
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2173 ms/op 4.0947 ms/op 0.79
phase0 processBlock - 250000 vs - 7PWei worstcase 37.312 ms/op 40.788 ms/op 0.91
altair processEth1Data - 250000 vs - 7PWei normalcase 699.08 us/op 767.62 us/op 0.91
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 21.239 us/op 23.060 us/op 0.92
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 73.760 us/op 95.573 us/op 0.77
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 16.768 us/op 21.913 us/op 0.77
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 12.268 us/op 17.072 us/op 0.72
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 206.97 us/op 247.25 us/op 0.84
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.2642 ms/op 1.6355 ms/op 0.77
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.7241 ms/op 2.2468 ms/op 0.77
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8813 ms/op 2.4977 ms/op 0.75
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.4837 ms/op 4.4306 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.6945 ms/op 3.1137 ms/op 0.87
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.7278 ms/op 7.1136 ms/op 0.95
Tree 40 250000 create 542.14 ms/op 440.24 ms/op 1.23
Tree 40 250000 get(125000) 222.23 ns/op 217.08 ns/op 1.02
Tree 40 250000 set(125000) 2.2900 us/op 1.1559 us/op 1.98
Tree 40 250000 toArray() 28.629 ms/op 23.063 ms/op 1.24
Tree 40 250000 iterate all - toArray() + loop 25.526 ms/op 23.052 ms/op 1.11
Tree 40 250000 iterate all - get(i) 78.905 ms/op 79.973 ms/op 0.99
MutableVector 250000 create 16.719 ms/op 17.342 ms/op 0.96
MutableVector 250000 get(125000) 6.9470 ns/op 7.2830 ns/op 0.95
MutableVector 250000 set(125000) 421.62 ns/op 370.45 ns/op 1.14
MutableVector 250000 toArray() 4.3571 ms/op 4.5231 ms/op 0.96
MutableVector 250000 iterate all - toArray() + loop 4.4454 ms/op 4.5357 ms/op 0.98
MutableVector 250000 iterate all - get(i) 1.6896 ms/op 1.6516 ms/op 1.02
Array 250000 create 4.2449 ms/op 4.0883 ms/op 1.04
Array 250000 clone - spread 1.3918 ms/op 1.4278 ms/op 0.97
Array 250000 get(125000) 0.77700 ns/op 0.71700 ns/op 1.08
Array 250000 set(125000) 0.92100 ns/op 0.91600 ns/op 1.01
Array 250000 iterate all - loop 119.39 us/op 100.36 us/op 1.19
effectiveBalanceIncrements clone Uint8Array 300000 65.287 us/op 58.413 us/op 1.12
effectiveBalanceIncrements clone MutableVector 300000 316.00 ns/op 433.00 ns/op 0.73
effectiveBalanceIncrements rw all Uint8Array 300000 197.60 us/op 196.99 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 141.48 ms/op 123.77 ms/op 1.14
phase0 afterProcessEpoch - 250000 vs - 7PWei 128.21 ms/op 137.48 ms/op 0.93
phase0 beforeProcessEpoch - 250000 vs - 7PWei 51.236 ms/op 40.914 ms/op 1.25
altair processEpoch - mainnet_e81889 657.03 ms/op 519.28 ms/op 1.27
mainnet_e81889 - altair beforeProcessEpoch 92.523 ms/op 67.791 ms/op 1.36
mainnet_e81889 - altair processJustificationAndFinalization 28.434 us/op 25.757 us/op 1.10
mainnet_e81889 - altair processInactivityUpdates 10.370 ms/op 6.9565 ms/op 1.49
mainnet_e81889 - altair processRewardsAndPenalties 90.573 ms/op 77.278 ms/op 1.17
mainnet_e81889 - altair processRegistryUpdates 5.2090 us/op 3.5390 us/op 1.47
mainnet_e81889 - altair processSlashings 1.1690 us/op 585.00 ns/op 2.00
mainnet_e81889 - altair processEth1DataReset 1.2970 us/op 1.1000 us/op 1.18
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.9640 ms/op 1.5603 ms/op 1.26
mainnet_e81889 - altair processSlashingsReset 6.5430 us/op 5.4190 us/op 1.21
mainnet_e81889 - altair processRandaoMixesReset 9.8850 us/op 10.140 us/op 0.97
mainnet_e81889 - altair processHistoricalRootsUpdate 1.3100 us/op 1.0650 us/op 1.23
mainnet_e81889 - altair processParticipationFlagUpdates 5.1320 us/op 3.2160 us/op 1.60
mainnet_e81889 - altair processSyncCommitteeUpdates 1.9420 us/op 854.00 ns/op 2.27
mainnet_e81889 - altair afterProcessEpoch 165.67 ms/op 148.47 ms/op 1.12
capella processEpoch - mainnet_e217614 1.6928 s/op 1.7051 s/op 0.99
mainnet_e217614 - capella beforeProcessEpoch 300.39 ms/op 282.07 ms/op 1.06
mainnet_e217614 - capella processJustificationAndFinalization 22.814 us/op 27.356 us/op 0.83
mainnet_e217614 - capella processInactivityUpdates 22.415 ms/op 20.397 ms/op 1.10
mainnet_e217614 - capella processRewardsAndPenalties 327.49 ms/op 349.50 ms/op 0.94
mainnet_e217614 - capella processRegistryUpdates 34.753 us/op 38.880 us/op 0.89
mainnet_e217614 - capella processSlashings 618.00 ns/op 1.3190 us/op 0.47
mainnet_e217614 - capella processEth1DataReset 464.00 ns/op 1.3050 us/op 0.36
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.3123 ms/op 4.8230 ms/op 0.89
mainnet_e217614 - capella processSlashingsReset 2.5730 us/op 6.7350 us/op 0.38
mainnet_e217614 - capella processRandaoMixesReset 5.8910 us/op 11.898 us/op 0.50
mainnet_e217614 - capella processHistoricalRootsUpdate 834.00 ns/op 1.2010 us/op 0.69
mainnet_e217614 - capella processParticipationFlagUpdates 2.3770 us/op 3.5850 us/op 0.66
mainnet_e217614 - capella afterProcessEpoch 332.55 ms/op 331.31 ms/op 1.00
phase0 processEpoch - mainnet_e58758 416.13 ms/op 513.46 ms/op 0.81
mainnet_e58758 - phase0 beforeProcessEpoch 130.55 ms/op 127.98 ms/op 1.02
mainnet_e58758 - phase0 processJustificationAndFinalization 17.479 us/op 21.763 us/op 0.80
mainnet_e58758 - phase0 processRewardsAndPenalties 55.224 ms/op 64.658 ms/op 0.85
mainnet_e58758 - phase0 processRegistryUpdates 10.868 us/op 16.161 us/op 0.67
mainnet_e58758 - phase0 processSlashings 560.00 ns/op 1.0210 us/op 0.55
mainnet_e58758 - phase0 processEth1DataReset 447.00 ns/op 699.00 ns/op 0.64
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0463 ms/op 1.2894 ms/op 0.81
mainnet_e58758 - phase0 processSlashingsReset 2.0290 us/op 2.4870 us/op 0.82
mainnet_e58758 - phase0 processRandaoMixesReset 3.6130 us/op 4.6000 us/op 0.79
mainnet_e58758 - phase0 processHistoricalRootsUpdate 533.00 ns/op 949.00 ns/op 0.56
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.2880 us/op 5.9120 us/op 0.73
mainnet_e58758 - phase0 afterProcessEpoch 100.64 ms/op 118.38 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2742 ms/op 1.6182 ms/op 0.79
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4418 ms/op 2.4558 ms/op 0.59
altair processInactivityUpdates - 250000 normalcase 18.719 ms/op 27.914 ms/op 0.67
altair processInactivityUpdates - 250000 worstcase 20.389 ms/op 26.023 ms/op 0.78
phase0 processRegistryUpdates - 250000 normalcase 15.533 us/op 12.623 us/op 1.23
phase0 processRegistryUpdates - 250000 badcase_full_deposits 368.43 us/op 455.17 us/op 0.81
phase0 processRegistryUpdates - 250000 worstcase 0.5 132.18 ms/op 145.14 ms/op 0.91
altair processRewardsAndPenalties - 250000 normalcase 57.611 ms/op 69.013 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 43.187 ms/op 60.073 ms/op 0.72
phase0 getAttestationDeltas - 250000 normalcase 8.6534 ms/op 9.7384 ms/op 0.89
phase0 getAttestationDeltas - 250000 worstcase 8.7101 ms/op 9.2288 ms/op 0.94
phase0 processSlashings - 250000 worstcase 2.4531 ms/op 2.9683 ms/op 0.83
altair processSyncCommitteeUpdates - 250000 160.59 ms/op 179.03 ms/op 0.90
BeaconState.hashTreeRoot - No change 250.00 ns/op 329.00 ns/op 0.76
BeaconState.hashTreeRoot - 1 full validator 136.48 us/op 156.48 us/op 0.87
BeaconState.hashTreeRoot - 32 full validator 1.2812 ms/op 2.1700 ms/op 0.59
BeaconState.hashTreeRoot - 512 full validator 12.111 ms/op 26.838 ms/op 0.45
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 202.44 us/op 444.73 us/op 0.46
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.3203 ms/op 3.6883 ms/op 0.63
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.367 ms/op 51.582 ms/op 0.55
BeaconState.hashTreeRoot - 1 balances 140.74 us/op 239.88 us/op 0.59
BeaconState.hashTreeRoot - 32 balances 1.1895 ms/op 2.7076 ms/op 0.44
BeaconState.hashTreeRoot - 512 balances 9.7288 ms/op 25.090 ms/op 0.39
BeaconState.hashTreeRoot - 250000 balances 180.62 ms/op 397.42 ms/op 0.45
aggregationBits - 2048 els - zipIndexesInBitList 15.828 us/op 45.097 us/op 0.35
regular array get 100000 times 44.638 us/op 46.468 us/op 0.96
wrappedArray get 100000 times 43.960 us/op 50.537 us/op 0.87
arrayWithProxy get 100000 times 14.296 ms/op 16.454 ms/op 0.87
ssz.Root.equals 216.00 ns/op 368.00 ns/op 0.59
byteArrayEquals 226.00 ns/op 390.00 ns/op 0.58
shuffle list - 16384 els 7.0860 ms/op 8.1152 ms/op 0.87
shuffle list - 250000 els 104.72 ms/op 116.82 ms/op 0.90
processSlot - 1 slots 16.889 us/op 40.218 us/op 0.42
processSlot - 32 slots 2.8259 ms/op 7.5788 ms/op 0.37
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 51.632 ms/op 62.584 ms/op 0.83
getCommitteeAssignments - req 1 vs - 250000 vc 2.5289 ms/op 2.8296 ms/op 0.89
getCommitteeAssignments - req 100 vs - 250000 vc 3.7093 ms/op 4.1391 ms/op 0.90
getCommitteeAssignments - req 1000 vs - 250000 vc 4.0337 ms/op 4.4565 ms/op 0.91
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.6800 ns/op 5.5800 ns/op 1.02
state getBlockRootAtSlot - 250000 vs - 7PWei 574.87 ns/op 779.32 ns/op 0.74
computeProposers - vc 250000 9.9158 ms/op 10.833 ms/op 0.92
computeEpochShuffling - vc 250000 124.84 ms/op 117.88 ms/op 1.06
getNextSyncCommittee - vc 250000 170.68 ms/op 173.97 ms/op 0.98
computeSigningRoot for AttestationData 35.247 us/op 28.243 us/op 1.25
hash AttestationData serialized data then Buffer.toString(base64) 2.4642 us/op 2.6385 us/op 0.93
toHexString serialized data 1.1597 us/op 1.2488 us/op 0.93
Buffer.toString(base64) 236.37 ns/op 255.10 ns/op 0.93

by benchmarkbot/action

@philknows philknows modified the milestones: v1.11.0, v1.12.0 Aug 23, 2023
@philknows philknows added the meta-breaking-change Introduces breaking changes to DB, Validator, Beacon Node, or CLI interfaces. Handle with care! label Aug 23, 2023
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch 2 times, most recently from 0481279 to 62d1e97 Compare August 30, 2023 12:46
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 62d1e97 to 62d39f5 Compare September 3, 2023 07:17
@g11tech g11tech mentioned this pull request Sep 3, 2023
Merged
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 62d39f5 to 615bd59 Compare September 4, 2023 06:40
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch 2 times, most recently from 3fc8b40 to 9ae9801 Compare September 14, 2023 17:08
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch 2 times, most recently from 97aaf8e to 25c4ab5 Compare September 26, 2023 11:41
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch 2 times, most recently from 1e88930 to e9c6aff Compare October 3, 2023 08:34
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch 3 times, most recently from 4779c67 to 9ea7037 Compare October 10, 2023 14:40
dapplion
dapplion reviewed Oct 12, 2023
View reviewed changes
packages/api/src/utils/routes.ts Outdated Show resolved Hide resolved
packages/beacon-node/src/api/impl/beacon/blocks/index.ts Outdated Show resolved Hide resolved
packages/beacon-node/src/api/impl/beacon/blocks/index.ts Outdated Show resolved Hide resolved
packages/beacon-node/src/chain/interface.ts Show resolved Hide resolved
packages/state-transition/src/block/processExecutionPayload.ts
@@ -77,45 +82,3 @@ export function processExecutionPayload(
.getExecutionForkTypes(state.slot)
.ExecutionPayloadHeader.toViewDU(payloadHeader) as typeof state.latestExecutionPayloadHeader;
}

export function executionPayloadToPayloadHeader(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self, util moved to packages/state-transition/src/util/execution.ts

packages/validator/src/services/block.ts Show resolved Hide resolved
packages/validator/src/services/block.ts Outdated Show resolved Hide resolved
packages/validator/test/unit/services/block.test.ts Outdated Show resolved Hide resolved
@dapplion
Copy link
Contributor

Mostly cosmetic comments, otherwise looks good

@g11tech
Copy link
Contributor Author

g11tech commented Oct 12, 2023

Mostly cosmetic comments, otherwise looks good

thank you will address the feedback ❤️ 🙏

@wemeetagain
Copy link
Member

Can you update the description with the lastest?

@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 88a2863 to 41abd3e Compare October 18, 2023 19:14
@dapplion
Copy link
Contributor

Is this really a breaking change if the flag is set to false by default?

@g11tech
Copy link
Contributor Author

g11tech commented Oct 19, 2023

Is this really a breaking change if the flag is set to false by default?

it is a lodestar cross version compatibility issue change as the RACE between execution and builder has now moved to beacon (for non produceblockv3 as well), so its both beacon and validator would need to be updated

@g11tech
Copy link
Contributor Author

g11tech commented Oct 19, 2023

Can you update the description with the lastest?

updated

@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 3daae60 to e6573f0 Compare October 20, 2023 12:03
wemeetagain
wemeetagain previously approved these changes Oct 20, 2023
View reviewed changes
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing this PR up

@g11tech g11tech changed the title feat: switch blinded and full block production to produce block v3 feat: implement blinded and full block production/publish via produce block v3 Oct 20, 2023
@g11tech g11tech changed the title feat: implement blinded and full block production/publish via produce block v3 feat: implement produce block v3 and move builder/execution race to beacon Oct 20, 2023
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 9277614 to acd897d Compare October 23, 2023 14:05
@g11tech
Copy link
Contributor Author

g11tech commented Oct 23, 2023

@dapplion further resolved the feedback 👍 ❤️

@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from acd897d to 676e315 Compare October 24, 2023 15:02
@g11tech
feat: switch blinded and full block production to produce block v3
b643672 
update the impl interfaces

Restore the previous versions

update test

fix oapi spec

fix tests

fix merge tests

fix tests

fix the tests

implement produce blockv3

refac forktypes

plug v3 into vlidator and get it working

mock test produceblockv3 and fix issues

fix lint

fixes

update head opts

cleanup the validator flags

shift fee recipient to extra args as well

typo

fix tests

missed commit

deprecate flag instead of removing

backward produceblock v2 compatability

lint

disable produce blockv3 for mixed tests

run on only execution

sim option fixes

improve log

pass args

fix spell check

sort wordlist

chore: sort wordlist

chore: fix spell check

refactor block types and introduce builder/execution race in produceblindedblock api as well

fix the publish blinded block to pick locally cached blinded data

add flag in get init dev vals

fix unit test

fixes

switch to v2 as default fix issues and run sim tests on mix configurations

remove utf8 graffiti from text log

apply feedback on api

cleanup on the sidecar cache

blinded to full utils

small produced cache cleanup

add metrics

cleanup response parsing

lint

further improv

improve interface

fix

apply feedback
@g11tech g11tech force-pushed the g11tech/produceblock-v3 branch from 676e315 to b643672 Compare October 25, 2023 19:22
dapplion
dapplion approved these changes Oct 26, 2023
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@g11tech g11tech merged commit d130ccd into unstable Oct 26, 2023
14 checks passed
@g11tech g11tech deleted the g11tech/produceblock-v3 branch October 26, 2023 15:22
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.12.0 🎉

@ensi321 ensi321 mentioned this pull request Nov 28, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

@wemeetagain wemeetagain wemeetagain left review comments

Assignees
No one assigned
Labels
meta-breaking-change Introduces breaking changes to DB, Validator, Beacon Node, or CLI interfaces. Handle with care!
Projects
None yet
Milestone
v1.12.0
Development

Successfully merging this pull request may close these issues.
4 participants
@g11tech @dapplion @wemeetagain @philknows