Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: get rid of heavy DB query to start Arbitrum missed messages discovery process #10767

Merged
merged 2 commits into from
Sep 20, 2024

Conversation

akolotov
Copy link
Contributor

@akolotov akolotov commented Sep 13, 2024

Motivation

As per the approach introduced in #10374, the logic to start the discovery process of missing batches attempted to identify a rollup block where the first missed message was expected. Although this approach seemed intuitive, the corresponding query to the DB became very heavy with an increasing number of rollup transactions. For example, on the ArbitrumOne instance, the query takes hours (based on logs) and likely causes excessive resource consumption, leading to issues such as:

(DBConnection.ConnectionError) ssl recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated)

Changelog

Bug Fixes

To address the issue described above, a more simple approach has been implemented that starts looking for missed messages from the block before the latest indexed block.

While this new method may not be optimal for Blockscout instances without missed messages (which are assumed to be the majority), it offers several advantages:

  1. It uses a finite number of relatively inexpensive queries, which can be tuned with the missed_messages_blocks_depth parameter.
  2. It scales better as the number of indexed transactions grows.
  3. It provides a more stable and predictable performance compared to the previous single, expensive query.

Checklist for your Pull Request (PR)

  • If I added new functionality, I added tests covering it.
  • If I fixed a bug, I added a regression test to prevent the bug from silently reappearing again.
  • I checked whether I should update the docs and did so by submitting a PR to docs repository.
  • If I added/changed/removed ENV var, I submitted a PR to docs repository to update the list of env vars and I updated the version to master in the Version column. If I removed variable, I added it to Deprecated ENV Variables page. After merging docs PR, changes will be reflected in these pages.
  • If I added new DB indices, I checked, that they are not redundant, with PGHero or other tools.
  • If I added/removed chain type, I modified the Github CI matrix and PR labels accordingly.

@akolotov akolotov marked this pull request as ready for review September 13, 2024 20:30
@vbaranov vbaranov merged commit f0ef80b into master Sep 20, 2024
16 checks passed
@vbaranov vbaranov deleted the fix/arbitrum/missed-messages-initial-blocks branch September 20, 2024 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants