feat: add mempool txs to payload #575

fmoletta · 2024-09-27T16:09:15Z

Motivation
Fetching transactions from the mempool during the block building process (engine_GetPayload).
This code can and should be improved in the future, but it sets a basis for block building

Description

Add method build_payload, which fetches transactions from the mempool and adds them to the payload.
Add a way to remove and filter transactions from the mempool
(Misc) Simplify state_trie related StoreEngine api. The StoreEngine now only creates the trie, leaving block and state root fetching to the Store. This removes duplicated code

LEFTOVER WORK:

Save timestamp when a tx is added to the mempool
process blobs in block building pipeline (we are now ignoring them)

TESTING_STATUS:
The pipeline for block_building tests is ForkChoiceUpdated -> (SendRawTransaction) -> GetPayload -> NewPayload -> (Fetch blocks/receipts).
Tests are currently failing at NewPayload because of different blob versioned hashes.

Closes None, but is part of #344

This reverts commit a66dc76.

This reverts commit adf5204.

…ne-hive

fkrause98

Looks good overall, left some comments to discuss, and btw, how are we testing these changes?

crates/blockchain/Cargo.toml

crates/blockchain/payload.rs

crates/core/types/transaction.rs

crates/rpc/engine/payload.rs

crates/storage/engines/in_memory.rs

crates/storage/engines/libmdbx.rs

crates/blockchain/payload.rs

ilitteri

The PR looks good. I agree with @fkrause98 comments. After creating issues for the TODOs it'd be great to link them as a comment in the code where appropriate.

…oad-add-txs

fkrause98 · 2024-10-03T17:47:10Z

Looks good, let me know when the CI is fixed and I'll take another look! @fmoletta

fmoletta · 2024-10-03T18:21:58Z

Looks good, let me know when the CI is fixed and I'll take another look! @fmoletta

Should be working now 🚀

…oad-add-txs

> [!WARNING] > This PR depends on #575 and must be merged after it. **Motivation**  **Description**  - Adds block producer module - Implements `Engine` to advance the blockchain --------- Co-authored-by: fmoletta <fedemoletta@hotmail.com> Co-authored-by: Manuel Bilbao <manuel.bilbao@lambdaclass.com> Co-authored-by: Manuel Iñaki Bilbao <bilbaomanuel98@gmail.com>

**Motivation** Fetching transactions from the mempool during the block building process (engine_GetPayload). This code can and should be improved in the future, but it sets a basis for block building  **Description** * Add method `build_payload`, which fetches transactions from the mempool and adds them to the payload. * Add a way to remove and filter transactions from the mempool * (Misc) Simplify state_trie related `StoreEngine` api. The `StoreEngine` now only creates the trie, leaving block and state root fetching to the `Store`. This removes duplicated code LEFTOVER WORK: * Save timestamp when a tx is added to the mempool * process blobs in block building pipeline (we are now ignoring them)   TESTING_STATUS: The pipeline for block_building tests is ForkChoiceUpdated -> (SendRawTransaction) -> GetPayload -> NewPayload -> (Fetch blocks/receipts). Tests are currently failing at NewPayload because of different blob versioned hashes. Closes None, but is part of lambdaclass#344

> [!WARNING] > This PR depends on lambdaclass#575 and must be merged after it. **Motivation**  **Description**  - Adds block producer module - Implements `Engine` to advance the blockchain --------- Co-authored-by: fmoletta <fedemoletta@hotmail.com> Co-authored-by: Manuel Bilbao <manuel.bilbao@lambdaclass.com> Co-authored-by: Manuel Iñaki Bilbao <bilbaomanuel98@gmail.com>

**Motivation** Fetching transactions from the mempool during the block building process (engine_GetPayload). This code can and should be improved in the future, but it sets a basis for block building  **Description** * Add method `build_payload`, which fetches transactions from the mempool and adds them to the payload. * Add a way to remove and filter transactions from the mempool * (Misc) Simplify state_trie related `StoreEngine` api. The `StoreEngine` now only creates the trie, leaving block and state root fetching to the `Store`. This removes duplicated code LEFTOVER WORK: * Save timestamp when a tx is added to the mempool * process blobs in block building pipeline (we are now ignoring them)   TESTING_STATUS: The pipeline for block_building tests is ForkChoiceUpdated -> (SendRawTransaction) -> GetPayload -> NewPayload -> (Fetch blocks/receipts). Tests are currently failing at NewPayload because of different blob versioned hashes. Closes None, but is part of lambdaclass#344

> [!WARNING] > This PR depends on lambdaclass#575 and must be merged after it. **Motivation**  **Description**  - Adds block producer module - Implements `Engine` to advance the blockchain --------- Co-authored-by: fmoletta <fedemoletta@hotmail.com> Co-authored-by: Manuel Bilbao <manuel.bilbao@lambdaclass.com> Co-authored-by: Manuel Iñaki Bilbao <bilbaomanuel98@gmail.com>

**Motivation** Fetching transactions from the mempool during the block building process (engine_GetPayload). This code can and should be improved in the future, but it sets a basis for block building  **Description** * Add method `build_payload`, which fetches transactions from the mempool and adds them to the payload. * Add a way to remove and filter transactions from the mempool * (Misc) Simplify state_trie related `StoreEngine` api. The `StoreEngine` now only creates the trie, leaving block and state root fetching to the `Store`. This removes duplicated code LEFTOVER WORK: * Save timestamp when a tx is added to the mempool * process blobs in block building pipeline (we are now ignoring them)   TESTING_STATUS: The pipeline for block_building tests is ForkChoiceUpdated -> (SendRawTransaction) -> GetPayload -> NewPayload -> (Fetch blocks/receipts). Tests are currently failing at NewPayload because of different blob versioned hashes. Closes None, but is part of #344

> [!WARNING] > This PR depends on #575 and must be merged after it. **Motivation**  **Description**  - Adds block producer module - Implements `Engine` to advance the blockchain --------- Co-authored-by: fmoletta <fedemoletta@hotmail.com> Co-authored-by: Manuel Bilbao <manuel.bilbao@lambdaclass.com> Co-authored-by: Manuel Iñaki Bilbao <bilbaomanuel98@gmail.com>

fmoletta added 30 commits September 23, 2024 16:18

Revert "fix: remove totalDifficulty (#347)"

bc150e9

This reverts commit a66dc76.

Complete forkChoiceUpdatedV3 up until payload validation

0c2cd4d

Add BuildPayloadArgs + id

1a1b781

Output proper payload id

7386f2b

Move execution payload to core

adf5204

Revert "Move execution payload to core"

772fac1

This reverts commit adf5204.

Push uncommited file

f8409b0

Add block building

7446289

Clippy + fmt

2b2f4dd

Modularize fork choice update fn

5e47673

Fix timestamp

31c380f

Fix ser

1bdf863

Fix

69d280b

Merge branch 'main' of github.com:lambdaclass/ethereum_rust into engi…

677a7ec

…ne-hive

Integrate previous changes (part 1)

9737674

fetch only header instead of block

1e3d710

Fix typo + remove unwrap

a634c13

fetch only header instead of block

995aca8

Clarify hardcoded values

a856b7a

Add doc

3f1e4ed

Add InvalidForkChoiceState RpcErr

8adfe28

revert uneeded changes

69607ae

Change miner to builder

9618e32

rename: local_block -> payload

76e4bdf

Add request + impl ExecutionPayloadV3::from_block

517d45b

Make ExecutionPayloadV3 serializable

b1bb89f

Impl endpoint

dc6da98

Add empty missing fields

5ba2b60

Compute new state root when building payloads

956d7cb

Map error

f3c356f

fkrause98 reviewed Sep 30, 2024

View reviewed changes

ilitteri mentioned this pull request Oct 1, 2024

feat(L2): basic sequencer #586

Merged

ilitteri reviewed Oct 2, 2024

View reviewed changes

crates/blockchain/payload.rs Outdated Show resolved Hide resolved

ilitteri reviewed Oct 2, 2024

View reviewed changes

crates/blockchain/payload.rs Outdated Show resolved Hide resolved

ilitteri reviewed Oct 2, 2024

View reviewed changes

Apply suggestion

bd4614b

fmoletta mentioned this pull request Oct 3, 2024

Use builder tip when filtering mempool transactions #680

Open

fmoletta added 7 commits October 3, 2024 11:44

Link github issues in TODOs

6e3bd4c

Merge branch 'main' of github.com:lambdaclass/ethereum_rust into payl…

68718f0

…oad-add-txs

Add doc link

49751b5

fix

8cb1bf1

swap info tracing for debug tracing

dddb869

rename txs -> txs_by_sender

c28ed99

use saturating sub and add comment

fc51790

jrchatruc approved these changes Oct 4, 2024

View reviewed changes

Merge branch 'main' of github.com:lambdaclass/ethereum_rust into payl…

92a2cef

…oad-add-txs

jrchatruc enabled auto-merge October 4, 2024 18:43

jrchatruc added this pull request to the merge queue Oct 4, 2024

Merged via the queue into main with commit b17a7ef Oct 4, 2024
7 checks passed

jrchatruc deleted the payload-add-txs branch October 4, 2024 19:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add mempool txs to payload #575

feat: add mempool txs to payload #575

fmoletta commented Sep 27, 2024 •

edited

Loading

fkrause98 left a comment

ilitteri left a comment

fkrause98 commented Oct 3, 2024

fmoletta commented Oct 3, 2024

feat: add mempool txs to payload #575

feat: add mempool txs to payload #575

Conversation

fmoletta commented Sep 27, 2024 • edited Loading

fkrause98 left a comment

Choose a reason for hiding this comment

ilitteri left a comment

Choose a reason for hiding this comment

fkrause98 commented Oct 3, 2024

fmoletta commented Oct 3, 2024

fmoletta commented Sep 27, 2024 •

edited

Loading