Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix caching #985

Closed
Closed

Conversation

notmandatory
Copy link
Member

@notmandatory notmandatory commented May 22, 2023

Description

The Wallet.ensure_addresses_cached() function was always regenerating and saving new scripts pubkeys starting from index 0. This is inefficient and for very large wallets and can cause sync to fail.

Notes to the reviewers

This issue was discovered by @bodymindarts while testing on a 200K+ transaction wallet.

Since this is more of a performance issue than functional/scaling problem I only added a test to verify that with multiple cached batches created during sync, the transaction to addresses in each cached batch are found. An integration test syncing 200K+ tx would take too long to complete.

Changelog notice

  • Fix script pubkey caching during sync to only cache starting from the current highest script index instead of 0.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

Bugfixes:

  • This pull request breaks the existing API
  • I've added tests to reproduce the issue which are now passing
  • I'm linking the issue being fixed by this PR

@notmandatory notmandatory changed the base branch from master to release/0.28 May 22, 2023 03:44
@notmandatory
Copy link
Member Author

@bodymindarts would you be able to test this branch against your large test descriptor?

@notmandatory notmandatory self-assigned this May 22, 2023
@notmandatory notmandatory added this to the 0.28.1 milestone May 22, 2023
@notmandatory notmandatory marked this pull request as ready for review May 22, 2023 16:48
@notmandatory
Copy link
Member Author

I fixed a few spelling typos to trigger a new build. Not sure why github wouldn't run the actions pipeline initially.

@notmandatory
Copy link
Member Author

Looks like some tests in CI are failing, I'll work on fixing those.

@bodymindarts
Copy link
Contributor

Just for clarification. It has > 200k addresses. About half and half from the public and the change descriptor. I'm not sure how many transactions but same order-of-magnitude.

I will try it out as soon as possible!

@notmandatory
Copy link
Member Author

Removed from 0.29 milestone and closing this PR since issues are already fixed in 1.0 syncing mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants