Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax Trait Bounds on TransactionPool::Transaction and EthPoolTransaction #11079

Merged
merged 13 commits into from
Oct 4, 2024

Conversation

0xForerunner
Copy link
Contributor

@0xForerunner 0xForerunner commented Sep 20, 2024

I'm working on the World Coin Block builder and ran into a problem implementing TransactionPool which currently requires

    PoolTransaction<
        Consensus = TransactionSignedEcRecovered,
        Pooled = PooledTransactionsElementEcRecovered
    >

which is unnecessarily restrictive. I've created a super trait which looks like this:

/// Super trait for transactions that can be converted to and from Eth transactions
pub trait EthPoolTransaction:
    PoolTransaction<
        Consensus: From<TransactionSignedEcRecovered> + Into<TransactionSignedEcRecovered>,
        Pooled: From<PooledTransactionsElementEcRecovered>
                    + Into<PooledTransactionsElementEcRecovered>,
    > + IntoRecoveredTransaction
{
    /// Extracts the blob sidecar from the transaction.
    fn take_blob(&mut self) -> EthBlobTransactionSidecar;

    /// Returns the number of blobs this transaction has.
    fn blob_count(&self) -> usize;

    /// Validates the blob sidecar of the transaction with the given settings.
    fn validate_blob(
        &self,
        blob: &BlobTransactionSidecar,
        settings: &KzgSettings,
    ) -> Result<(), BlobTransactionValidationError>;

    /// Returns the number of authorizations this transaction has.
    fn authorization_count(&self) -> usize;
}

which should be far more versatile for anyone else looking to implement the trait.

I was considering removing IntoRecoveredTransaction since it seems unnecessary since now you can simply call .into_consensus().into() in every case where you would have called .to_recovered_transaction But I decided to leave it since I wasn't sure if there were any other use cases for it. If you guys want to remove it we can do that in another PR.

Additionally perhaps we should consider renaming trait TransactionPool to trait EthTransactionPool to better illustrate it's bounds.

Would love your feedback, and looking forward to getting this merged in. Thanks!

Copy link
Member

@emhane emhane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly prefer if we leave this completely without trait bounds for now, following design in #10616 and #10997

smthg like

trait PoolTransaction {
    type Error;
    type BlobSideCar;
    etc.
}

associated types for all types used in trait methods except KzgSettings probably

pls change title of pr, it mentions the wrong trait

Pooled = PooledTransactionsElementEcRecovered,
Consensus = TransactionSignedEcRecovered,
>;
type Transaction: EthPoolTransaction;
Copy link
Contributor Author

@0xForerunner 0xForerunner Sep 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emhane I've named the PR because this is specifically the trait bound on TransactionPool I am attempting to relax

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, that's not incorrect, though more helpful would be clarifying it's the trait bound on the associated type TransactionPool::Transaction in the title

@0xForerunner 0xForerunner changed the title Relax Transaction Pool Trait Bounds Relax TransactionPool Trait Bounds to EthPoolTransaction Sep 21, 2024
@0xForerunner
Copy link
Contributor Author

@emhane thanks so much for taking a look.

I'm not 100% sure what you're suggesting though. Are you saying to remove type Consensus from PoolTransaction entirely?

Or are you suggesting removing all bounds from EthPoolTransaction in favour of requiring Into/From on methods only where required?

I had a look at the 2 PRs you linked and I'm not entirely sure how they relate. Thanks for clarification!

@0xForerunner 0xForerunner changed the title Relax TransactionPool Trait Bounds to EthPoolTransaction Relax Trait Bounds on TransactionPool and EthPoolTransaction Sep 21, 2024
@emhane
Copy link
Member

emhane commented Sep 22, 2024

Or are you suggesting removing all bounds from EthPoolTransaction in favour of requiring Into/From on methods only where required?

this :)

@emhane
Copy link
Member

emhane commented Sep 22, 2024

@mattsse tried to move away from those Into/From trait bounds recently, so don't think we want to reintroduce them now

imo worth exploring if it's possible to delay those trait bounds to where you need them and not have them on the trait definition

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supportive in general, luckily these bounds are now possible with stable rust 1.81

only have 1 q re the additional IntoRecoveredTransaction bound

@klkvr I think ideally we replace the EthPool restrictions to something like alloy::Transaction or similar

Comment on lines 930 to 932
Consensus: From<TransactionSignedEcRecovered> + Into<TransactionSignedEcRecovered>,
Pooled: From<PooledTransactionsElementEcRecovered>
+ Into<PooledTransactionsElementEcRecovered>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

relaxing this makes sense, until we have proper trait abstraction for primitives

Consensus: From<TransactionSignedEcRecovered> + Into<TransactionSignedEcRecovered>,
Pooled: From<PooledTransactionsElementEcRecovered>
+ Into<PooledTransactionsElementEcRecovered>,
> + IntoRecoveredTransaction
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't actually need it. I kept it there because the method from this trait was used and rather than using .into_consensus.into() I just left it as is. I can for sure get rid of it though!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to remove this, don't think this is used

@0xForerunner
Copy link
Contributor Author

0xForerunner commented Sep 23, 2024

@emhane @mattsse Thanks again for taking the time to review.

I think ideally we replace the EthPool restrictions to something like alloy::Transaction or similar

Sounds good to me. Assuming you're referencing this trait. I'll take a look into this.

imo worth exploring if it's possible to delay those trait bounds to where you need them and not have them on the trait definition

Okay perfect I'm going to begin work on this modification! Shouldn't take long!

@0xForerunner
Copy link
Contributor Author

Quick update, @mattsse I took a look at alloy::consensus::Transaction but I don't think that quite solves the problem we have, perhaps I'm not quite understanding your suggestion. Anyhow, going to continue adding the trait bounds directly to the methods!

@0xForerunner
Copy link
Contributor Author

@emhane @mattsse So after trying to apply these trait bounds directly to the methods for a couple hours I've found that it just kind of explodes. You end up with

where
    T: PoolTransaction<
        Consensus: From<TransactionSignedEcRecovered> + Into<TransactionSignedEcRecovered>,
        Pooled: From<PooledTransactionsElementEcRecovered>
                    + Into<PooledTransactionsElementEcRecovered>,
    > + IntoRecoveredTransaction

on a lot of function signatures. Personally I don't think this feels like the correct approach, but if you are pretty sure this is the direction you want to head then just let me know and i'll keep going.

I think since the current code essentially requires that these conversions are implemented it feels okay to have the trait also depend on it. In the future I think this could be made even more generic of transaction types, but that seems like the pretty serious overhall.

Let me know either way, thanks!

Comment on lines 975 to 979
impl IntoRecoveredTransaction for EthPooledTransaction {
fn to_recovered_transaction(&self) -> TransactionSignedEcRecovered {
self.transaction.clone()
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we use this on main, so idk why we need this here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify @mattsse, did you want me to remove all traces of IntoRecoveredTransaction from the codebase? Or just remove this impl and the new trait bounds? Either one of these is possible.

Consensus: From<TransactionSignedEcRecovered> + Into<TransactionSignedEcRecovered>,
Pooled: From<PooledTransactionsElementEcRecovered>
+ Into<PooledTransactionsElementEcRecovered>,
> + IntoRecoveredTransaction
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to remove this, don't think this is used

@mattsse
Copy link
Collaborator

mattsse commented Sep 25, 2024

@ewoolsey not worth looking into alloy::transaction rn,

the only thing I'd like to change here is removing the IntoRecoveredTransaction trait bound because I couldn't find any usage on EthPooledTransaction ref #11203

Pooled = PooledTransactionsElementEcRecovered,
Consensus = TransactionSignedEcRecovered,
>;
type Transaction: EthPoolTransaction;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, that's not incorrect, though more helpful would be clarifying it's the trait bound on the associated type TransactionPool::Transaction in the title

Comment on lines 975 to 979
impl IntoRecoveredTransaction for EthPooledTransaction {
fn to_recovered_transaction(&self) -> TransactionSignedEcRecovered {
self.transaction.clone()
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -380,11 +376,11 @@ impl<T: PoolTransaction> ValidPoolTransaction<T> {
}
}

impl<T: PoolTransaction<Consensus = TransactionSignedEcRecovered>> IntoRecoveredTransaction
impl<T: PoolTransaction<Consensus: Into<TransactionSignedEcRecovered>>> IntoRecoveredTransaction
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
impl<T: PoolTransaction<Consensus: Into<TransactionSignedEcRecovered>>> IntoRecoveredTransaction
impl<T> IntoRecoveredTransaction where T: PoolTransaction<Consensus: Into<TransactionSignedEcRecovered>>

we default to using the where clause syntax when the trait bound is more than one name or there is more than one generic

@0xForerunner 0xForerunner changed the title Relax Trait Bounds on TransactionPool and EthPoolTransaction Relax Trait Bounds on TransactionPool::Transaction and EthPoolTransaction Sep 25, 2024
@0xForerunner
Copy link
Contributor Author

@emhane @mattsse I've implemented the changes you've suggested! Specifically I've:

  • Removed the bound on IntoRecoveredTransaction
  • Renamed IntoRecoveredTransaction to ToRecoveredTransaction
  • Switched to using where clause
  • Merged from main

Things should be ready for your final review, and again thanks so much for taking a look!

@0xForerunner
Copy link
Contributor Author

@emhane @mattsse just bumping this!

@mattsse
Copy link
Collaborator

mattsse commented Oct 3, 2024

thanks for the bump,
scheduling this for later today.

@mattsse mattsse enabled auto-merge October 4, 2024 07:33
@mattsse mattsse added this pull request to the merge queue Oct 4, 2024
Merged via the queue into paradigmxyz:main with commit 1fe9f32 Oct 4, 2024
35 checks passed
@0xForerunner
Copy link
Contributor Author

Thanks so much all! have a great weekend!

Thegaram added a commit to scroll-tech/reth that referenced this pull request Oct 10, 2024
* chore(provider): use `get_in_memory_or_storage` on `transactions_by_block_range` (paradigmxyz#11453)

* chore(exex): adjust WAL gauge metric names (paradigmxyz#11454)

* chore(provider): use `get_in_memory_or_storage_by_block` on `fn block_body_indices` (paradigmxyz#11452)

* chore(provider): find `last_database_block_number` with `BlockState` anchor instead (paradigmxyz#11455)

* feat: add metrics for failed deliveries (paradigmxyz#11456)

* chore: release 1.0.8 (paradigmxyz#11457)

* chore(provider): use `block_ref` instead on `BlockState` (paradigmxyz#11458)

* feat(rpc): Add codes in execution witness return (paradigmxyz#11443)

* fix: ensure the request's gas limit does not exceed the target gas limit (paradigmxyz#11462)

* feat(grafana): ExEx WAL (paradigmxyz#11461)

* fix: use correct rpc errors (paradigmxyz#11463)

* chore(provider): clone after filtering on `sealed_headers_while`  (paradigmxyz#11459)

      Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Map `TransferKind::EofCreate` => `OperationType::OpEofCreate` (paradigmxyz#11090)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* fix: windows build (paradigmxyz#11465)

* chore: use `block_ref` on `CanonicalInMemoryState`  (paradigmxyz#11467)

* chore(rpc): remove include_preimage param on debug_execution_witness (paradigmxyz#11466)

* feat: make addons stateful (paradigmxyz#11204)

Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com>

* Relax Trait Bounds on TransactionPool::Transaction and EthPoolTransaction (paradigmxyz#11079)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* test: add unit tests for `CanonicalChain` (paradigmxyz#11472)

* feat(perf): integrate OnStateHook in executor (paradigmxyz#11345)

* feat: cleaned up prepare_call_env() (paradigmxyz#11469)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* chore: use block.body directly (paradigmxyz#11474)

* feat: Add metrics to track transactions by type in txpool  (paradigmxyz#11403)

Co-authored-by: garwah <garwah@garwah>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* chore: op chainspec (paradigmxyz#11415)

* chore(exex): more backfill debug logs (paradigmxyz#11476)

* fix(exex): use thresholds in stream backfill (paradigmxyz#11478)

* chore(db): capture tx opening backtrace in debug mode (paradigmxyz#11477)

* chore(sdk): `SealedHeader` generic over header (paradigmxyz#11429)

* chore(lint): fix lint primitives (paradigmxyz#11487)

* chore(provider): add more test coverage on `BlockchainProvider::*by_tx_range` queries (paradigmxyz#11480)

* test: ensure default hash matches (paradigmxyz#11486)

* chore: rm deposit contract config for op (paradigmxyz#11479)

* chore(lint): fix lint storage (paradigmxyz#11485)

* chore(provider): add more test coverage on `BlockchainProvider::*by_block_range` queries (paradigmxyz#11488)

* fix(rpc-eth-types): incorrect error msg(; -> :) (paradigmxyz#11503)

Signed-off-by: jsvisa <delweng@gmail.com>

* Reexport optimism specific crates from `op-reth` (paradigmxyz#11499)

* feat: rpc replace function created (paradigmxyz#11501)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Add metrics for failed deliveries to Grafana dashboard (paradigmxyz#11481)

* chore: Remove duplicate EthereumChainSpecParser in favor of existing EthChainSpecParser  (paradigmxyz#11412)

Co-authored-by: garwah <garwah@garwah>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* chore(lint): fix `clippy::needles_lifetimes` (paradigmxyz#11496)

* chore: rm from genesis impl (paradigmxyz#11509)

* fix: cap gas limit properly (paradigmxyz#11505)

* feat: add PoolBuilderConfigOverrides (paradigmxyz#11507)

* feat: expose Op node network_config helper (paradigmxyz#11506)

* chore(deps): weekly `cargo update` (paradigmxyz#11518)

Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com>

* test: add unit tests for `PruneLimiter` (paradigmxyz#11517)

* feat(provider): add `test_race` to `BlockchainProvider2` tests (paradigmxyz#11523)

* fix(tree): make state methods work for historical blocks (paradigmxyz#11265)

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
Co-authored-by: Federico Gimenez <fgimenez@users.noreply.github.com>

* rpc: use `eth_api()` method (paradigmxyz#11516)

* chore: delete rpc-types (paradigmxyz#11528)

* feat: add get_highest_tx_by_sender to pools (paradigmxyz#11514)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* ci: add `windows` cargo check  (paradigmxyz#11468)

* fix: acquire permit first (paradigmxyz#11537)

* chore: dont fail on ttd (paradigmxyz#11539)

* grafana: add metrics of all transactions in pool by type  (paradigmxyz#11515)

Co-authored-by: Emilia Hane <emiliaha95@gmail.com>
Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com>

* feat(exex): subscribe to notifications with head using `ExExContext` (paradigmxyz#11500)

* chore: enforce window (paradigmxyz#11540)

* Refactor get_payload_bodies_by_hash_with to be non-blocking (paradigmxyz#11511)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Introduce Eth PayloadTypes Impl (paradigmxyz#11519)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* fix(op-reth): add jemalloc feature to optimism-cli for version (paradigmxyz#11543)

* fix(grafana): remove rate function from panel "Transactions by Type in Pool" (paradigmxyz#11542)

* chore: move ethfiltererror (paradigmxyz#11552)

* chore: rm redundant type hint (paradigmxyz#11557)

* Introduce Op PayloadTypes Impl (paradigmxyz#11558)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* chore: chain manual serialisation implementation (paradigmxyz#11538)

* chore: rm unused optimism feature (paradigmxyz#11559)

* feat(trie): expose storage proofs (paradigmxyz#11550)

* chore: rm unused optimism feature from compat (paradigmxyz#11560)

* Added InternalBlockExecutionError to execute.rs exports (paradigmxyz#11525)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* docs(exex): include code for ExEx book from real files (paradigmxyz#11545)

* fix: actually configure the custom gas limit (paradigmxyz#11565)

* chore: relax trait bound for `EthTransactions` (paradigmxyz#11571)

* fix(provider): fix sub overflow on `tx_range` queries for empty blocks (paradigmxyz#11568)

* chore(provider): add more test coverage on `BlockchainProvider` non-range queries (paradigmxyz#11564)

* feat: adding a new method to network config builder (paradigmxyz#11569)

* chore: rm unused optimism feature from engine api (paradigmxyz#11577)

* chore: replace some revm deps (paradigmxyz#11579)

* chore: rm bad cap function (paradigmxyz#11562)

* feat: impl `Encodable2718` and `Decodable2718` for `PooledTransactionsElement` (paradigmxyz#11482)

* fix(exex): exhaust backfill job when using a stream (paradigmxyz#11578)

* fix: in-memory trie updates pruning (paradigmxyz#11580)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* chore(providers): test race condition on all `BlockchainProvider2` macro tests (paradigmxyz#11574)

* chore: also derive arb for test (paradigmxyz#11588)

* chore: bump alloy primitives 0 8 7 (paradigmxyz#11586)

* chore(ci): remove expected failures related to checksummed addresses (paradigmxyz#11589)

* chore(rpc): use `block_hash` instead on fetching `debug_trace_block` block (paradigmxyz#11587)

* feat: add mul support for SubPoolLimit (paradigmxyz#11591)

Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com>

* docs: delete missing part path (paradigmxyz#11590)

Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com>

* fix: simplify reorg handling (paradigmxyz#11592)

* fix: use original bytes for codes (paradigmxyz#11593)

* perf(rpc): use `Arc<BlockWithSenders` on `full_block_cache` (paradigmxyz#11585)

* fix: active inflight count (paradigmxyz#11598)

* fix(net): max inflight tx reqs default (paradigmxyz#11602)

* fix: set deposit gasprice correctly (paradigmxyz#11603)

* fix: set system tx correctly (paradigmxyz#11601)

* fix(trie): prefix set extension (paradigmxyz#11605)

* feat: add tx propagation mode (paradigmxyz#11594)

* feat: add helper function to provde the tx manager config (paradigmxyz#11608)

* fix(grafana): set instance variable from `reth_info` metric (paradigmxyz#11607)

* fix(net): add concurrency param from config to `TransactionFetcherInfo` (paradigmxyz#11600)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* perf(rpc): optimistically retrieve block if near the tip on `eth_getLogs` (paradigmxyz#11582)

* update fork base commit

* remove new book tests

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: joshieDo <93316087+joshieDo@users.noreply.github.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: Francis Li <francis.li@coinbase.com>
Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com>
Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com>
Co-authored-by: Eric Woolsey <ewoolsey@ualberta.ca>
Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Federico Gimenez <fgimenez@users.noreply.github.com>
Co-authored-by: Varun Doshi <61531351+varun-doshi@users.noreply.github.com>
Co-authored-by: garwah <14845405+garwahl@users.noreply.github.com>
Co-authored-by: garwah <garwah@garwah>
Co-authored-by: greged93 <82421016+greged93@users.noreply.github.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Parikalp Bhardwaj <53660958+Parikalp-Bhardwaj@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com>
Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com>
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
Co-authored-by: crazykissshout <crazykisss111@gmail.com>
Co-authored-by: tedison <76473430+edisontim@users.noreply.github.com>
Co-authored-by: David <39963997+No0key@users.noreply.github.com>
Co-authored-by: Emilia Hane <emiliaha95@gmail.com>
Co-authored-by: Steven <112043913+stevencartavia@users.noreply.github.com>
Co-authored-by: Debjit Bhowal <debjitbhowal.db@gmail.com>
Co-authored-by: Oliver <onbjerg@users.noreply.github.com>
Co-authored-by: Luca Provini <luca.provini@usemerkle.com>
Co-authored-by: John <devthejohn@gmail.com>
vandenbogart pushed a commit to testmachine-ai/reth that referenced this pull request Oct 14, 2024
…tion (paradigmxyz#11079)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants