implement getBlobs and bypass blob gossip validation on successful blob retrievals from EL

[agnxsh]

ethereum/execution-apis#559

if EL supports engine_getBlobsV1

• makes the block available as soon as the CL gets all the required blobs from the blob pool
• populates blob quarantine with that info (should probably verify proofs and inclusion once, like gossip?)
• pops from blob quarantine and add blobs to the block processor

else

• does everything the conventional way

Note: getBlobs in nimbus-eth2 here is made to be backward compatible from Electra and NOT Deneb (Deneb is what the spec suggests)

Pros

• doesn't call kzg proof verification and inclusion proof verification for 9 blobs per slot (assuming the EL can be trusted)
• faster block import (in happy cases)

Cons

• relying heavily on the EL mempool can cause threats of various sorts, however, our block processor runs another kzg proof check before persisting the block and blobs, in case of a failure, the RequestManager should be able to fetch back the otherwise malicious blob/proof data sent by the bad EL, EL can be quickly swapped, Nimbus would rely on blob gossip that entire duration and switch back to optimistically pulling blobs from EL.

CI failing: because of some nim-web3 upstream incompatibilities

[github-actions]

Unit Test Results

       15 files  ±0    2 614 suites  ±0   1h 15m 27s ⏱️ - 4m 44s
  6 412 tests ±0    5 891 ✔️ ±0  521 💤 ±0  0 ❌ ±0 
44 616 runs  ±0  43 898 ✔️ ±0  718 💤 ±0  0 ❌ ±0 

Results for commit bd45643. ± Comparison against base commit 7309f7d.

♻️ This comment has been updated with latest results.

[agnxsh]

for devnet testing locally, use ethpandaops/nimbus-eth2:getBlobs

[tersec]

Obligatory comment about debugEcho (but yeah, it's a draft PR and useful for now)

[agnxsh]

just for now, until i see things are ok in the local devnet

[agnxsh]

2025-02-14 13:24:27 INF 2025-02-14 07:54:27.006+00:00 Slot end                                   topics="beacnde" slot=234 nextActionWait=1s993ms123us121ns nextAttestationSlot=235 nextProposalSlot=236 syncCommitteeDuties=current head=4932c763:234
2025-02-14 13:24:28 WRN 2025-02-14 07:54:28.713+00:00 Peer count low, no new peers discovered    topics="networking" discovered_nodes=0 new_peers=@[] current_peers=1 wanted_peers=160
2025-02-14 13:24:29 INF 2025-02-14 07:54:28.999+00:00 Slot start                                 topics="beacnde" nextFork=Fulu:100000001 head=4932c763:234 delay=137us922ns finalized=5:5177fdfd peers=1 slot=235 sync=synced epoch=7
2025-02-14 13:24:29 pulled blobs from el
2025-02-14 13:24:29 9
2025-02-14 13:24:31 INF 2025-02-14 07:54:31.295+00:00 Attestation sent                           topics="beacval" attestation="(committee_index: 0, attester_index: 28, data: (slot: 235, index: 0, beacon_block_root: \"61632bd0\", source: \"6:f956b1f3\", target: \"7:a67fac0a\"), signature: \"929f5919\")" delay=-1s703ms988us327ns subnet_id=11
2025-02-14 13:24:31 INF 2025-02-14 07:54:31.295+00:00 Attestation sent                           topics="beacval" attestation="(committee_index: 0, attester_index: 52, data: (slot: 235, index: 0, beacon_block_root: \"61632bd0\", source: \"6:f956b1f3\", target: \"7:a67fac0a\"), signature: \"b5d6aa5c\")" delay=-1s703ms537us202ns subnet_id=11
2025-02-14 13:24:31 INF 2025-02-14 07:54:31.296+00:00 Sync committee message sent 

we're finalising with nimbus running getBlobs in a devnet.

[agnxsh]

we can get some better decision making on whether to validate blob gossip or pull blobs from EL every slot, by caching engine_exchangeCapabilities, however a good interval should be decided as ELs can be swapped on the run.