Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(sync-v2): Modify TransactionStreamingClient to process vertices asynchronously #865

Merged
merged 1 commit into from
Nov 10, 2023

Conversation

msbrogli
Copy link
Member

@msbrogli msbrogli commented Nov 10, 2023

Motivation

This PR enhances the on_block_complete() method, which initially processed transactions and added blocks to the DAG in a synchronous manner, thus occupying the event loop for the duration of this task. Due to the synchronous execution, the larger the transaction set, the longer the event loop was blocked, preventing the full node from attending to other critical operations like managing connections and processing API requests.

This PR transforms the method into an asynchronous operation. By doing so, after the addition of each transaction, control is yielded back to the event loop, allowing the full node to multitask effectively. This change is particularly beneficial during periods of heavy transaction processing, ensuring that node responsiveness and efficiency are maintained.

Acceptance Criteria

  1. Modify on_block_complete() to yield back to the event loop after calling on_new_tx() for each vertex.
  2. Remove the TransactionStreamingClient._buffer because it was not necessary anymore.
  3. Even if we receive a TRANSACTIONS-END message, we have to wait until all transactions have been processed before resolving the deferred.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@msbrogli msbrogli requested a review from glevco November 10, 2023 00:09
@msbrogli msbrogli self-assigned this Nov 10, 2023
@msbrogli msbrogli requested a review from jansegre as a code owner November 10, 2023 00:09
@msbrogli msbrogli force-pushed the feat/sync-v2-async-tx-processing branch from 5f93c92 to 4a7d448 Compare November 10, 2023 00:17
Copy link

codecov bot commented Nov 10, 2023

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (382cc38) 85.42% compared to head (4a7d448) 85.31%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #865      +/-   ##
==========================================
- Coverage   85.42%   85.31%   -0.11%     
==========================================
  Files         282      282              
  Lines       22270    22308      +38     
  Branches     3367     3377      +10     
==========================================
+ Hits        19025    19033       +8     
- Misses       2583     2602      +19     
- Partials      662      673      +11     
Files Coverage Δ
hathor/p2p/sync_v2/agent.py 78.59% <100.00%> (-0.56%) ⬇️
hathor/p2p/sync_v2/transaction_streaming_client.py 85.50% <88.46%> (-4.69%) ⬇️

... and 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@msbrogli msbrogli force-pushed the feat/sync-v2-async-tx-processing branch from 4a7d448 to e85ad70 Compare November 10, 2023 15:52
@msbrogli msbrogli merged commit e85ad70 into master Nov 10, 2023
8 checks passed
@msbrogli msbrogli deleted the feat/sync-v2-async-tx-processing branch November 10, 2023 15:52
@jansegre jansegre mentioned this pull request Nov 13, 2023
2 tasks
This was referenced Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants