Skip to content

Commit

Permalink
Implement EIP-4844 WIP
Browse files Browse the repository at this point in the history
commit f1e4aaa
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 15:31:01 2022 +0700

    Fix get state for attestation verification logic

commit ad2c300
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 15:14:14 2022 +0700

    Add eip4844 sim test

commit 6bf2e62
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 14:54:45 2022 +0700

    Fix typo in gossipsub coreTopicsAtFork

commit 2e565a8
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 14:54:08 2022 +0700

    Fix typo in updateSyncState currentEpoch

commit 4169498
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 14:53:54 2022 +0700

    Create valid blobs in mock

commit 0f22a80
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Dec 1 00:50:08 2022 +0700

    Debug failing sim tests with more better progress logs

commit 32b1779
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 22:19:54 2022 +0700

    Don't create invalid mock data

commit d77e741
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 15:52:08 2022 +0700

    Allow to run mock EL in sim tests

commit 6fd5d46
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 15:51:25 2022 +0700

    Log chain config on start

commit fb58663
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 15:50:36 2022 +0700

    Add disabledWithdrawals

commit ad9b478
Author: gajinder <develop@g11tech.io>
Date:   Wed Nov 30 13:25:52 2022 +0530

    add eip4844 interop

commit 74287e3
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 00:22:36 2022 +0700

    Skip blobs validation if from gossip

commit b30aa71
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 30 00:09:16 2022 +0700

    Use enum for BlockImport

commit 4bf587d
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 12:38:07 2022 +0700

    Add note on reorg blobs

commit ccdb9bb
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 09:33:29 2022 +0700

    Prune producedBlobsSidecarCache

commit ebf0eae
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 09:28:05 2022 +0700

    Subscribe beacon_block_and_blobs_sidecar on fork

commit 8614743
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 09:27:51 2022 +0700

    Simplify blobsSidecar cache type

commit 58c595a
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 02:00:47 2022 +0700

    Fix rebase issues

commit 804887d
Merge: 22d1f85 c1ab090
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Nov 27 01:56:37 2022 +0700

    Merge remote-tracking branch 'origin/unstable' into dapplion/eip-4844

commit 22d1f85
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 26 23:51:21 2022 +0700

    Fix types

commit 0216099
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 26 23:40:57 2022 +0700

    fix rebase issues

commit 6796207
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 26 23:40:28 2022 +0700

    Subscribe to reqresp methods on forks

commit 99cf400
Merge: d33472b 798ec16
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 26 21:35:52 2022 +0700

    Merge remote-tracking branch 'origin/unstable' into dapplion/eip-4844

commit d33472b
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 26 11:05:06 2022 +0700

    Mock blobs WIP

commit 1dd5e74
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 25 00:02:41 2022 +0300

    Enable blobs reqresp

commit 9c0a0e6
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 25 00:02:18 2022 +0300

    Add yamux

commit 0427435
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 19 01:45:11 2022 +0100

    Temp change FIELD_ELEMENTS_PER_BLOB

commit 370c733
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 19 01:44:30 2022 +0100

    Ensure correct sizes on blob bundle

commit 90f0d4c
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sat Nov 19 00:57:47 2022 +0100

    Fix verifyKzgCommitmentsAgainstTransactions math

commit 5a6652b
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 23:12:46 2022 +0100

    Check allVersionedHashes length

commit 37f3387
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 22:40:36 2022 +0100

    Skip verifyAggregateKzgProof is empty blobs

commit e5d6f4f
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 22:15:30 2022 +0100

    Set SLOTS_PER_EPOCH: 3 for interop tests

commit 8f18bd8
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 21:51:06 2022 +0100

    Move trusted setup file out of src

commit 8d0f1c0
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 02:01:19 2022 +0100

    Revert fork-choice changes

commit 3b82bd7
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 01:57:44 2022 +0100

    Only line JSON

commit b47162b
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 01:49:29 2022 +0100

    Revert preset changes

commit c34e969
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 01:18:23 2022 +0100

    Fix test bellatrix/fork_choice/on_merge_block/pyspec_tests/all_valid

commit 9ea8348
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 00:43:16 2022 +0100

    Fix test Should subscribe to gossip core topics on demand

commit 58fb922
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Fri Nov 18 00:40:26 2022 +0100

    Fix test utils / params / assertEqualParams

commit 0d49fa4
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 23:33:37 2022 +0100

    Fix gossipsub test

commit 7ae69de
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 23:29:23 2022 +0100

    Fix UnknownBlockSync test

commit 4e2a63a
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 23:29:12 2022 +0100

    Remove useless test

commit 4f544df
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 23:23:25 2022 +0100

    Fix enrForkID.nextForkVersion test

commit a3f479e
Author: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 19:09:15 2022 +0100

    Disable parallel for spec tests (#4776)

commit e7eba90
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 19:28:26 2022 +0100

    Fix assertLinearChainSegment

commit 864b7d3
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 18:06:13 2022 +0100

    Review PR

commit 6963c1c
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 16:23:00 2022 +0100

    FIx readme checks

commit ece46f0
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 15:43:03 2022 +0100

    Fix tests

commit 6c878e5
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 14:55:05 2022 +0100

    Fix topic test

commit 0f44e49
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 14:54:05 2022 +0100

    Move kzg test to beacon-node

commit 3f7a0ad
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 14:46:30 2022 +0100

    Fix types

commit 27e503f
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 14:46:15 2022 +0100

    Remove old trusted setup

commit fdeb4f5
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 14:36:35 2022 +0100

    Review PR pt.2

commit 7da555e
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 12:10:04 2022 +0100

    Remove   DOMAIN_BLOBS_SIDECAR

commit ceea759
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 12:06:59 2022 +0100

    Ensure forks are not scheduled in presets

commit 4148785
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 12:06:09 2022 +0100

    Review Dan PR

commit e969457
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 11:03:44 2022 +0100

    Fix types

commit e16a699
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 09:53:17 2022 +0100

    Implement blobsSidecar pruning

commit c90f0fb
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 09:42:05 2022 +0100

    Archive blobs

commit 4632bcf
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Thu Nov 17 09:41:54 2022 +0100

    Fix types

commit 46f58a4
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 16 23:35:37 2022 +0100

    Clean up changes

commit 3d86a02
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 16 21:00:33 2022 +0100

    WIP import block+blobs coupled

commit d268815
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 16 18:57:22 2022 +0100

    WIP block processing

commit 175117d
Merge: ffb6336 bfe5da8
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Nov 16 11:44:02 2022 +0100

    Merge remote-tracking branch 'origin/unstable' into dapplion/eip-4844

commit ffb6336
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Tue Nov 15 21:27:30 2022 +0100

    WIP

commit 611a609
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Tue Nov 15 15:36:42 2022 +0100

    Fix script name

commit 26a0794
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Tue Nov 15 15:33:38 2022 +0100

    De-couple block production

commit 513b873
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Nov 9 10:30:54 2022 -0800

    Bump c-kzg to version that handles empty array of blobs -- remove application code special-case

commit a63b043
Author: dancoffman <coffman@coinbase.com>
Date:   Tue Nov 8 13:00:57 2022 -0800

    Update version of c-kzg to one which does not crash for blobs arrays of length 0 and 1

commit 6a8d536
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 23:33:02 2022 -0800

    Delete unused polynomialCommitments file

commit 656b7d3
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 23:26:44 2022 -0800

    processBlocks for 4844 blocks. Do not compute aggregate proof for empty array (it crashes)

commit aee7d95
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 16:53:02 2022 -0800

    Bump c-kzg again

commit b992fc6
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 16:52:09 2022 -0800

    Add unit test just to verify the c-kzg library works

commit 054cfb6
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 15:17:50 2022 -0800

    Fix trusted_setup file path

commit 7e8f3ab
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 11:55:34 2022 -0800

    Load KZG setup

commit 513e277
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 11:51:04 2022 -0800

    Bump c-kzg to the version that compiles on Linux

commit 78220ef
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Nov 7 11:32:30 2022 -0800

    Bump @types/node

commit 3db4f57
Author: dancoffman <coffman@coinbase.com>
Date:   Sat Nov 5 01:17:08 2022 -0700

    Use computeAggregateKzgProof from the c-kzg package and delete all the code it encapsulates

commit 2fe076b
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Oct 31 17:36:49 2022 -0700

    Drill blobs all the way through

commit ad1a62f
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Oct 31 16:10:33 2022 -0700

    Use SSZ object correctly

commit 28e9f8a
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Oct 31 15:11:59 2022 -0700

    publishBlockWithBlobs implementation. A lot more moon math scaffolding that will be removed.

commit d1e3e4b
Author: dancoffman <coffman@coinbase.com>
Date:   Sat Oct 29 00:23:42 2022 -0700

    Scaffold block proposing w/ KZG commitments

commit f3cedf9
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 22:28:22 2022 -0700

    Add a couple more of the polynomial commitment types

commit 2e9e41a
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 16:32:14 2022 -0700

    Add chain config constants and validator types

commit b166387
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 15:28:25 2022 -0700

    Fix assigning excessDataGas to ExecutionPayload

commit 2a4bedb
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 12:09:21 2022 -0700

    Revert Capella ssz type change

commit 89c43bf
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 12:08:00 2022 -0700

    Revert upgradeStateToCapella

commit 87ecaaf
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 11:57:55 2022 -0700

    Fix allForks SSZ types

commit e6c13e6
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 28 12:13:17 2022 -0700

    Add beacon_block_and_blobs_sidecar gossip type

commit f5c8cfa
Author: dancoffman <coffman@coinbase.com>
Date:   Fri Oct 21 11:54:04 2022 -0700

    Fix: Accidentally removed assigning execution payload to block

commit 78996a9
Author: dancoffman <coffman@coinbase.com>
Date:   Thu Oct 20 15:56:08 2022 -0700

    Fix state upgrade and patch blocks with missing fields when necessary

commit a83df9e
Author: dancoffman <coffman@coinbase.com>
Date:   Thu Oct 20 09:42:02 2022 -0700

    When producing a block to propose, assign blobKzgCommitments from Engine API getBlobsBundle response

commit 6a7630f
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Oct 19 16:33:41 2022 -0700

    Add engine_getBlobsBundleV1 Engine API call

commit 9de5e10
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Oct 19 15:11:08 2022 -0700

    Capella and 4844 state upgrades

commit 71e7e0b
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Oct 19 12:08:06 2022 -0700

    Add DOMAIN_BLOBS_SIDECAR

commit d3fe373
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Oct 19 12:05:04 2022 -0700

    Beacon chain types complete

commit 451f6e9
Author: dancoffman <coffman@coinbase.com>
Date:   Wed Oct 19 11:21:07 2022 -0700

    Spec-matching configs and presets

commit 266ba7a
Author: dancoffman <coffman@coinbase.com>
Date:   Tue Oct 18 18:56:33 2022 -0700

    Sketch out EIP-4844 types

commit 002d850
Author: dancoffman <coffman@coinbase.com>
Date:   Mon Oct 17 15:21:10 2022 -0700

    Updates to allow test:spec-min|main work, some edits to contributing instructions
  • Loading branch information
dapplion committed Dec 7, 2022
1 parent d5b957b commit 310db0b
Show file tree
Hide file tree
Showing 61 changed files with 1,204 additions and 279 deletions.
11 changes: 11 additions & 0 deletions .github/workflows/test-sim-merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ env:
GETH_WITHDRAWALS_IMAGE: g11tech/geth:withdrawals-fix
ETHEREUMJS_WITHDRAWALS_IMAGE: g11tech/ethereumjs:withdrawals
NETHERMIND_WITHDRAWALS_IMAGE: nethermindeth/nethermind:withdrawals_yolo
GETH_4844_IMAGE: g11tech/geth:4844

jobs:
sim-merge-tests:
Expand Down Expand Up @@ -122,6 +123,16 @@ jobs:
# EL_BINARY_DIR: ${{ env.NETHERMIND_WITHDRAWALS_IMAGE }}
# EL_SCRIPT_DIR: netherminddocker

- name: Pull geth eip4844
run: docker pull $GETH_4844_IMAGE

- name: Test Lodestar <> geth 4844
run: yarn test:sim:eip4844
working-directory: packages/beacon-node
env:
EL_BINARY_DIR: ${{ env.GETH_4844_IMAGE }}
EL_SCRIPT_DIR: gethdocker

- name: Upload debug log test files
if: ${{ always() }}
uses: actions/upload-artifact@v2
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/test-sim.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,16 @@ jobs:
if: steps.cache-deps.outputs.cache-hit == 'true'
# </common-build>

- name: Simulation tests for CLI
run: yarn test:sim
- name: Sim tests multifork
run: yarn test:sim:multifork
working-directory: packages/cli

- name: Sim tests endpoints
run: yarn test:sim:endpoints
working-directory: packages/cli

- name: Sim tests eip4844
run: yarn test:sim:eip4844
working-directory: packages/cli

- name: Upload debug log test files for "packages/cli"
Expand Down
2 changes: 2 additions & 0 deletions packages/beacon-node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@
"test:sim:merge-interop": "mocha 'test/sim/merge-interop.test.ts'",
"test:sim:mergemock": "mocha 'test/sim/mergemock.test.ts'",
"test:sim:withdrawals": "mocha 'test/sim/withdrawal-interop.test.ts'",
"test:sim:eip4844": "mocha 'test/sim/4844-interop.test.ts'",
"download-spec-tests": "node --loader=ts-node/esm test/spec/downloadTests.ts",
"check-spec-tests": "mocha test/spec/checkCoverage.ts",
"test:spec-bls-general": "mocha --config .mocharc.spec.cjs 'test/spec/bls/**/*.test.ts' 'test/spec/general/**/*.test.ts'",
Expand All @@ -104,6 +105,7 @@
"@chainsafe/discv5": "^1.4.0",
"@chainsafe/libp2p-gossipsub": "^4.1.1",
"@chainsafe/libp2p-noise": "^10.2.0",
"@chainsafe/libp2p-yamux": "^3.0.3",
"@chainsafe/persistent-merkle-tree": "^0.4.2",
"@chainsafe/snappy-stream": "5.1.1",
"@chainsafe/ssz": "^0.9.2",
Expand Down
4 changes: 2 additions & 2 deletions packages/beacon-node/src/chain/chain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -353,11 +353,11 @@ export class BeaconChain implements IBeaconChain {
return await this.db.block.get(fromHexString(block.blockRoot));
}

async produceBlock(blockAttributes: BlockAttributes): Promise<allForks.BeaconBlock> {
produceBlock(blockAttributes: BlockAttributes): Promise<allForks.BeaconBlock> {
return this.produceBlockWrapper<BlockType.Full>(BlockType.Full, blockAttributes);
}

async produceBlindedBlock(blockAttributes: BlockAttributes): Promise<allForks.BlindedBeaconBlock> {
produceBlindedBlock(blockAttributes: BlockAttributes): Promise<allForks.BlindedBeaconBlock> {
return this.produceBlockWrapper<BlockType.Blinded>(BlockType.Blinded, blockAttributes);
}

Expand Down
2 changes: 1 addition & 1 deletion packages/beacon-node/src/chain/clock/LocalClock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {IBeaconClock} from "./interface.js";
export class LocalClock implements IBeaconClock {
private readonly config: IChainForkConfig;
private readonly genesisTime: number;
private timeoutId: NodeJS.Timeout;
private timeoutId: number | NodeJS.Timeout;
private readonly emitter: ChainEventEmitter;
private readonly signal: AbortSignal;
private _currentSlot: number;
Expand Down
2 changes: 2 additions & 0 deletions packages/beacon-node/src/chain/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ export type BlockProcessOpts = {
* will still issue fcU for block proposal
*/
disableImportExecutionFcU?: boolean;
// TODO EIP-4844: to test without capella
disabledWithdrawals?: boolean;
};

export const defaultChainOptions: IChainOptions = {
Expand Down
18 changes: 17 additions & 1 deletion packages/beacon-node/src/chain/produceBlock/produceBlockBody.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,12 @@ export async function produceBlockBody<T extends BlockType>(
const finalizedBlockHash = this.forkChoice.getFinalizedBlock().executionPayloadBlockHash ?? ZERO_HASH_HEX;
const feeRecipient = this.beaconProposerCache.getOrDefault(proposerIndex);

// Capella and later forks have blsToExecutionChanges on BeaconBlockBody
// It would be nicer to use ForkSeq, but that doesn't narrow the type appropriately
if (fork === ForkName.capella || fork === ForkName.eip4844) {
(blockBody as capella.BeaconBlockBody).blsToExecutionChanges = [];
}

if (blockType === BlockType.Blinded) {
if (!this.executionBuilder) throw Error("Execution Builder not available");

Expand Down Expand Up @@ -181,14 +187,17 @@ export async function produceBlockBody<T extends BlockType>(
// are pre-merge. We don't care the same for builder segment as the execution block
// will takeover if the builder flow was activated and errors
try {
// https://github.com/ethereum/consensus-specs/blob/dev/specs/eip4844/validator.md#constructing-the-beaconblockbody

const prepareRes = await prepareExecutionPayload(
this,
fork,
safeBlockHash,
finalizedBlockHash ?? ZERO_HASH_HEX,
currentState as CachedBeaconStateBellatrix,
currentState as CachedBeaconStateExecutions,
feeRecipient
);

if (prepareRes.isPremerge) {
(blockBody as allForks.ExecutionBlockBody).executionPayload = ssz.allForksExecution[
fork
Expand All @@ -203,6 +212,7 @@ export async function produceBlockBody<T extends BlockType>(
// See: https://discord.com/channels/595666850260713488/892088344438255616/1009882079632314469
await sleep(PAYLOAD_GENERATION_TIME_MS);
}

const payload = await this.executionEngine.getPayload(fork, payloadId);
(blockBody as allForks.ExecutionBlockBody).executionPayload = payload;

Expand All @@ -212,6 +222,12 @@ export async function produceBlockBody<T extends BlockType>(
this.metrics?.blockPayload.emptyPayloads.inc({prepType});
}

// Capella and later forks have withdrawals on their ExecutionPayload
if (fork === ForkName.capella || fork === ForkName.eip4844) {
// TODO EIP-4844 Remove this when the EC includes `withdrawals`
(blockBody as capella.BeaconBlockBody).executionPayload.withdrawals = [];
}

if (fork === ForkName.eip4844) {
// SPEC: https://github.com/ethereum/consensus-specs/blob/dev/specs/eip4844/validator.md#blob-kzg-commitments
// After retrieving the execution payload from the execution engine as specified in Bellatrix, use the
Expand Down
21 changes: 19 additions & 2 deletions packages/beacon-node/src/network/gossip/gossipsub.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@ import {SignaturePolicy, TopicStr} from "@chainsafe/libp2p-gossipsub/types";
import {PeerScore, PeerScoreParams} from "@chainsafe/libp2p-gossipsub/score";
import {MetricsRegister, TopicLabel, TopicStrToLabel} from "@chainsafe/libp2p-gossipsub/metrics";
import {IBeaconConfig} from "@lodestar/config";
import {ATTESTATION_SUBNET_COUNT, ForkName, SYNC_COMMITTEE_SUBNET_COUNT} from "@lodestar/params";
import {ATTESTATION_SUBNET_COUNT, ForkName, ForkSeq, SYNC_COMMITTEE_SUBNET_COUNT} from "@lodestar/params";
import {allForks, altair, phase0} from "@lodestar/types";
import {ILogger, Map2d, Map2dArr} from "@lodestar/utils";
import {computeStartSlotAtEpoch} from "@lodestar/state-transition";

import {SignedBeaconBlockAndBlobsSidecar} from "@lodestar/types/eip4844";
import {IMetrics} from "../../metrics/index.js";
import {Eth2Context} from "../../chain/index.js";
import {PeersData} from "../peers/peersData.js";
Expand Down Expand Up @@ -197,9 +198,19 @@ export class Eth2Gossipsub extends GossipSub {

async publishBeaconBlock(signedBlock: allForks.SignedBeaconBlock): Promise<void> {
const fork = this.config.getForkName(signedBlock.message.slot);

await this.publishObject<GossipType.beacon_block>({type: GossipType.beacon_block, fork}, signedBlock);
}

async publishSignedBeaconBlockAndBlobsSidecar(blockWithBlobs: SignedBeaconBlockAndBlobsSidecar): Promise<void> {
const fork = this.config.getForkName(blockWithBlobs.beaconBlock.message.slot);

await this.publishObject<GossipType.beacon_block_and_blobs_sidecar>(
{type: GossipType.beacon_block_and_blobs_sidecar, fork},
blockWithBlobs
);
}

async publishBeaconAggregateAndProof(aggregateAndProof: phase0.SignedAggregateAndProof): Promise<number> {
const fork = this.config.getForkName(aggregateAndProof.message.aggregate.data.slot);
return await this.publishObject<GossipType.beacon_aggregate_and_proof>(
Expand Down Expand Up @@ -415,7 +426,13 @@ function getMetricsTopicStrToLabel(config: IBeaconConfig): TopicStrToLabel {
topics.push({fork, type: GossipType.sync_committee, subnet});
}

topics.push({fork, type: GossipType.beacon_block});
// beacon_block_and_blobs_sidecar replaces beacon_block topic after EIP-4844
// Ref https://github.com/ethereum/consensus-specs/blob/a1e46d1ae47dd9d097725801575b46907c12a1f8/specs/eip4844/p2p-interface.md#beacon_block_and_blobs_sidecar
if (ForkSeq[fork] >= ForkSeq.eip4844) {
topics.push({fork, type: GossipType.beacon_block_and_blobs_sidecar});
} else {
topics.push({fork, type: GossipType.beacon_block});
}
topics.push({fork, type: GossipType.beacon_aggregate_and_proof});
topics.push({fork, type: GossipType.voluntary_exit});
topics.push({fork, type: GossipType.proposer_slashing});
Expand Down
2 changes: 2 additions & 0 deletions packages/beacon-node/src/network/gossip/interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ export type GossipTopicTypeMap = {
[GossipType.sync_committee]: {type: GossipType.sync_committee; subnet: number};
[GossipType.light_client_finality_update]: {type: GossipType.light_client_finality_update};
[GossipType.light_client_optimistic_update]: {type: GossipType.light_client_optimistic_update};
[GossipType.beacon_block_and_blobs_sidecar]: {type: GossipType.beacon_block_and_blobs_sidecar};
};

export type GossipTopicMap = {
Expand All @@ -75,6 +76,7 @@ export type GossipTypeMap = {
[GossipType.sync_committee]: altair.SyncCommitteeMessage;
[GossipType.light_client_finality_update]: altair.LightClientFinalityUpdate;
[GossipType.light_client_optimistic_update]: altair.LightClientOptimisticUpdate;
[GossipType.beacon_block_and_blobs_sidecar]: eip4844.SignedBeaconBlockAndBlobsSidecar;
};

export type GossipFnByType = {
Expand Down
6 changes: 6 additions & 0 deletions packages/beacon-node/src/network/gossip/topic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ function stringifyGossipTopicType(topic: GossipTopic): string {
case GossipType.sync_committee_contribution_and_proof:
case GossipType.light_client_finality_update:
case GossipType.light_client_optimistic_update:
case GossipType.beacon_block_and_blobs_sidecar:
return topic.type;
case GossipType.beacon_attestation:
case GossipType.sync_committee:
Expand Down Expand Up @@ -92,6 +93,10 @@ export function getGossipSSZType(topic: GossipTopic) {
return ssz.altair.LightClientOptimisticUpdate;
case GossipType.light_client_finality_update:
return ssz.altair.LightClientFinalityUpdate;
case GossipType.beacon_block_and_blobs_sidecar:
return ssz.eip4844.SignedBeaconBlockAndBlobsSidecar;
default:
throw new Error(`No ssz gossip type for ${(topic as GossipTopic).type}`);
}
}

Expand Down Expand Up @@ -123,6 +128,7 @@ export function parseGossipTopic(forkDigestContext: IForkDigestContext, topicStr
case GossipType.beacon_block:
case GossipType.beacon_block_and_blobs_sidecar:
case GossipType.beacon_aggregate_and_proof:
case GossipType.beacon_block_and_blobs_sidecar:
case GossipType.voluntary_exit:
case GossipType.proposer_slashing:
case GossipType.attester_slashing:
Expand Down
4 changes: 2 additions & 2 deletions packages/beacon-node/src/network/interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {INetworkEventBus} from "./events.js";
import {Eth2Gossipsub} from "./gossip/index.js";
import {MetadataController} from "./metadata.js";
import {PeerAction} from "./peers/index.js";
import {IReqRespBeaconNode} from "./reqresp/ReqRespBeaconNode.js";
import {ReqRespBeaconNode} from "./reqresp/ReqRespBeaconNode.js";
import {IAttnetsService, ISubnetsService, CommitteeSubscription} from "./subnets/index.js";

export type PeerSearchOptions = {
Expand All @@ -18,7 +18,7 @@ export type PeerSearchOptions = {

export interface INetwork {
events: INetworkEventBus;
reqResp: IReqRespBeaconNode;
reqResp: ReqRespBeaconNode;
attnetsService: IAttnetsService;
syncnetsService: ISubnetsService;
gossip: Eth2Gossipsub;
Expand Down
33 changes: 21 additions & 12 deletions packages/beacon-node/src/network/network.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ import {ILogger, sleep} from "@lodestar/utils";
import {ATTESTATION_SUBNET_COUNT, ForkName, ForkSeq, SYNC_COMMITTEE_SUBNET_COUNT} from "@lodestar/params";
import {Discv5, ENR} from "@chainsafe/discv5";
import {computeEpochAtSlot, computeTimeAtSlot} from "@lodestar/state-transition";
import {altair, Epoch, phase0} from "@lodestar/types";
import {altair, eip4844, Epoch, phase0} from "@lodestar/types";
import {IMetrics} from "../metrics/index.js";
import {ChainEvent, IBeaconChain, IBeaconClock} from "../chain/index.js";
import {BlockInput, BlockInputType, getBlockInput} from "../chain/blocks/types.js";
import {INetworkOptions} from "./options.js";
import {INetwork} from "./interface.js";
import {ReqRespBeaconNode, ReqRespHandlers} from "./reqresp/ReqRespBeaconNode.js";
import {ReqRespBeaconNode, ReqRespHandlers} from "./reqresp/index.js";
import {Eth2Gossipsub, getGossipHandlers, GossipHandlers, GossipTopicTypeMap, GossipType} from "./gossip/index.js";
import {MetadataController} from "./metadata.js";
import {FORK_EPOCH_LOOKAHEAD, getActiveForks} from "./forks.js";
Expand Down Expand Up @@ -207,17 +207,19 @@ export class Network implements INetwork {
return this.peerManager.hasSomeConnectedPeer();
}

publishBeaconBlockMaybeBlobs(blockImport: BlockInput): Promise<void> {
switch (blockImport.type) {
publishBeaconBlockMaybeBlobs(blockInput: BlockInput): Promise<void> {
switch (blockInput.type) {
case BlockInputType.preEIP4844:
return this.gossip.publishBeaconBlock(blockImport.block);
return this.gossip.publishBeaconBlock(blockInput.block);

case BlockInputType.postEIP4844:
// TODO EIP-4844: Implement SignedBeaconBlockAndBlobsSidecar publish topic
throw Error("SignedBeaconBlockAndBlobsSidecar publish not implemented");
return this.gossip.publishSignedBeaconBlockAndBlobsSidecar({
beaconBlock: blockInput.block as eip4844.SignedBeaconBlock,
blobsSidecar: blockInput.blobs,
});

case BlockInputType.postEIP4844OldBlobs:
throw Error(`Attempting to broadcast old BlockImport slot ${blockImport.block.message.slot}`);
throw Error(`Attempting to broadcast old BlockInput slot ${blockInput.block.message.slot}`);
}
}

Expand Down Expand Up @@ -245,7 +247,7 @@ export class Network implements INetwork {
throw Error(`blocks.length ${blocks.length} != blobsSidecars.length ${blobsSidecars.length}`);
}

const blockImports: BlockInput[] = [];
const blockInputs: BlockInput[] = [];
for (let i = 0; i < blocks.length; i++) {
const block = blocks[i];
const blobsSidecar = blobsSidecars[i];
Expand All @@ -255,9 +257,9 @@ export class Network implements INetwork {
throw Error(`blob does not match block slot ${block.message.slot} != ${blobsSidecar.beaconBlockSlot}`);
}

blockImports.push(getBlockInput.postEIP4844(this.config, block, blobsSidecar));
blockInputs.push(getBlockInput.postEIP4844(this.config, block, blobsSidecar));
}
return blockImports;
return blockInputs;
}

// Post EIP-4844 but old blobs
Expand Down Expand Up @@ -474,7 +476,7 @@ export class Network implements INetwork {
private coreTopicsAtFork(fork: ForkName): GossipTopicTypeMap[keyof GossipTopicTypeMap][] {
// Common topics for all forks
const topics: GossipTopicTypeMap[keyof GossipTopicTypeMap][] = [
{type: GossipType.beacon_block},
// {type: GossipType.beacon_block}, // Handled below
{type: GossipType.beacon_aggregate_and_proof},
{type: GossipType.voluntary_exit},
{type: GossipType.proposer_slashing},
Expand All @@ -488,6 +490,13 @@ export class Network implements INetwork {
topics.push({type: GossipType.light_client_finality_update});
}

// After EIP4844 only track beacon_block_and_blobs_sidecar topic
if (ForkSeq[fork] < ForkSeq.eip4844) {
topics.push({type: GossipType.beacon_block});
} else {
topics.push({type: GossipType.beacon_block_and_blobs_sidecar});
}

if (this.opts.subscribeAllSubnets) {
for (let subnet = 0; subnet < ATTESTATION_SUBNET_COUNT; subnet++) {
topics.push({type: GossipType.beacon_attestation, subnet});
Expand Down
11 changes: 10 additions & 1 deletion packages/beacon-node/src/network/nodejs/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import {Bootstrap} from "@libp2p/bootstrap";
import {MulticastDNS} from "@libp2p/mdns";
import {PeerId} from "@libp2p/interface-peer-id";
import {Datastore} from "interface-datastore";
import {yamux} from "@chainsafe/libp2p-yamux";
import {createNoise} from "./noise.js";

export interface ILibp2pOptions {
Expand Down Expand Up @@ -35,6 +36,9 @@ export async function createNodejsLibp2p(options: ILibp2pOptions): Promise<Libp2
peerDiscovery.push(new MulticastDNS());
}
}

const yamuxInstance = yamux()();

return await createLibp2p({
peerId: options.peerId,
addresses: {
Expand All @@ -43,7 +47,12 @@ export async function createNodejsLibp2p(options: ILibp2pOptions): Promise<Libp2
},
connectionEncryption: [createNoise()],
transports: [new TCP()],
streamMuxers: [new Mplex({maxInboundStreams: 256})],
streamMuxers: [
//
new Mplex({maxInboundStreams: 256}),
// Cast because yamux.StreamMuxerFactory is different than libp2p.StreamMuxerFactory
yamuxInstance as Mplex,
],
peerDiscovery,
metrics: {
// temporarily disable since there is a performance issue with it
Expand Down
Loading

0 comments on commit 310db0b

Please sign in to comment.