Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add EVM txs eviction logic #204

Merged
merged 1 commit into from
Feb 26, 2024
Merged

Add EVM txs eviction logic #204

merged 1 commit into from
Feb 26, 2024

Conversation

codchen
Copy link
Collaborator

@codchen codchen commented Feb 21, 2024

Describe your changes and provide context

  • When an EVM transaction is removed from mempool for any reason other than it being successfully processed and committed (e.g. evicted or expired), we want to move all the transactions after it in the queue back to pending.
  • When an EVM transaction is removed from mempool because it's successfully processed, we will only remove itself from the pq without doing anything to the rest of the queue.

Testing performed to validate your change

unit tests

@codchen codchen force-pushed the evm-eviction branch 8 times, most recently from 1e4f2ea to 6b3123e Compare February 21, 2024 11:53
@codchen codchen requested a review from jewei1997 February 21, 2024 15:07
@codchen codchen merged commit 2a85977 into evm Feb 26, 2024
22 checks passed
udpatil pushed a commit that referenced this pull request Feb 28, 2024
@codchen @udpatil
Add EVM txs eviction logic (#204)
c9e2944
udpatil pushed a commit that referenced this pull request Mar 26, 2024
@codchen @udpatil
Add EVM txs eviction logic (#204)
55991cb
udpatil pushed a commit that referenced this pull request Apr 16, 2024
@codchen @udpatil
Add EVM txs eviction logic (#204)
693d289
udpatil pushed a commit that referenced this pull request Apr 16, 2024
@codchen @udpatil
Add EVM txs eviction logic (#204)
e89d46c
udpatil added a commit that referenced this pull request Apr 19, 2024
@udpatil @codchen
@stevenlanders @yzang2019 @jewei1997 @sigv @Kbhat1
Merge seiv2 into main (#232)
754c03a 
* Make ReadMaxTxs atomic (#166)

* Support pending transaction in mempool (#169)

* fix unconfirmed tx to consider pending txs (#172)

* fix pending pop (#173)

* add TTL for pending txs (#174)

* [EVM] Fix evm pending nonce (#179)

* Perf: Increase buffer size for pubsub server to boost performance (#167)

* Increase buffer size for pubsub server

* Add more timeout for test failure

* Add more timeout

* Fix test split scripts

* Fix test split

* Fix unit test

* Unit test

* Unit test

* [P2P] Optimize block pool requester retry and peer pick up logic (#170)

* P2P Improvements: Fix block sync reactor and block pool retry logic

* Revert "Add event data to result event (#165)" (#176)

This reverts commit 72bb29c.

* Fix block sync auto restart not working as expected (#175)

* Fix edge case for blocksync (#178)

* fix evm pending nonce

* fix test

* deflake a test

* de-flake test

* Revert "merge main"

This reverts commit 58b9424, reversing
changes made to 02d1478.

* consider keep-in-cache logic when removing from cache

* undo test tweaks

---------

Co-authored-by: Yiming Zang <50607998+yzang2019@users.noreply.github.com>
Co-authored-by: Jeremy Wei <jeremy.t.wei@gmail.com>

* Fix bug when popping pending TXs (#188)

* Add mempool metrics for number of pending tx and expired txs (#189)

* Add metrics for mempool pending transaction size

* Add expired tx count metrics

* [EVM] Allow multiple txs from same account in a block (#190)

* add mempool prioritization with evm nonce

* fix priority stability

* index fixes

* replace with binary search insert

* impl binary search

* fix removeTx to push next queued evm tx (#191)

* fix expire metric (#193)

* [EVM] Fix duplicate evm txs from priority queue (#195)

* debug duplicate evm tx

* add more logs

* add some \ns

* more logs

* fix swap check

* add-lockable-reap-by-gas

* add invariant checks

* fix invariant parenthesis

* fix log

* remove invalid invariant

* fix nonce ordering pain

* handle ordering of insert

* fix remove

* cleanup

* fix imports

* cleanup

* avoid getTransactionByHash(hash) panic due to index

* use Key() to compare instead of pointer

* [EVM] prevent duplicate txs from getting inserted (#196)

* prevent duplicates in mempool

* use timestamp in priority queue

* [EVM] Add logging for expiration (#198)

* add logging for expired txs

* cleanup

* [EVM] Avoid returning nil transactions on ForEach (#197)

* remove heapIndex to avoid nil scenario

* avoid returning nil in loop (mimic Peek)

* call callback from mempool (#200)

* separate limit for pending tx (#202)

* Add EVM txs eviction logic (#204)

* Fix debug log (#205)

* EVM transaction replacement (#206) (#208)

* Add heapIndex with safety check (#213)

* add heapIndex with safety check

* cleanup

* comment out for perf test

* add back perf improvement

* fix nil test

* Use write-lock in (*TxPriorityQueue).ReapMax funcs (#209)

ReapMaxBytesMaxGas and ReapMaxTxs funcs in TxPriorityQueue claim
> Transactions returned are not removed from the mempool transaction
> store or indexes.

However, they use a priority queue to accomplish the claim
> Transaction are retrieved in priority order.

This is accomplished by popping all items out of the whole heap, and
then pushing then back in sequentially. A copy of the heap cannot be
obtained otherwise. Both of the mentioned functions use a read-lock
(RLock) when doing this. This results in a potential scenario where
multiple executions of the ReapMax can be started in parallel, and
both would be popping items out of the priority queue.

In practice, this can be abused by executing the `unconfirmed_txs` RPC
call repeatedly. Based on our observations, running it multiple times
per millisecond results in multiple threads picking it up at the same
time. Such a scenario can be obtained via the WebSocket interface, and
spamming `unconfirmed_txs` calls there. The behavior that happens is a
`Panic in WSJSONRPC handler` when a queue item unexpectedly disappears
for `mempool.(*TxPriorityQueue).Swap`.
(`runtime error: index out of range [0] with length 0`)

This can additionally lead to a `CONSENSUS FAILURE!!!` if the race
condition occurs for `internal/consensus.(*State).finalizeCommit`
when it tries to do `mempool.(*TxPriorityQueue).RemoveTx`, but
the ReapMax has already removed all elements from the underlying
heap. (`runtime error: index out of range [-1]`)

This commit switches the lock type to a write-lock (Lock) to ensure
no parallel modifications take place. This commit additionally updates
the tests to allow parallel execution of the func calls in testing,
as to prevent regressions (in case someone wants to downgrade the locks
without considering the implications from the underlying heap usage).

---------

Co-authored-by: Valters Jansons <sigv@users.noreply.github.com>

* Pending Txs Update Condition (#214)

* Add metrics for mempool size changes (#220)

* [EVM] Adjust locking for replacement (#224)

* Remove tx from cache when canAddPendingTx fails (#230)

* add tx hash to evm info proto (#231)

---------

Co-authored-by: codchen <codchen03@gmail.com>
Co-authored-by: Steven Landers <steven@seinetwork.io>
Co-authored-by: Yiming Zang <50607998+yzang2019@users.noreply.github.com>
Co-authored-by: Jeremy Wei <jeremy.t.wei@gmail.com>
Co-authored-by: Valters Jansons <sigv@users.noreply.github.com>
Co-authored-by: Kartik Bhat <kartikbhatri@gmail.com>
udpatil pushed a commit that referenced this pull request Apr 19, 2024
@codchen @udpatil
Add EVM txs eviction logic (#204)
3b0ef20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stevenlanders stevenlanders Awaiting requested review from stevenlanders

@jewei1997 jewei1997 Awaiting requested review from jewei1997

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@codchen