Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filter startpoint option (height only) #227

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

optout21
Copy link

@optout21 optout21 commented Nov 28, 2024

Motivations for this change:

  • to be able to specify a start point for filter download, just by height and not block hash
  • more fine-controlled start point.

Block headers and filter headers build on each other, and the chain is used to check consistency. It makes sense to require a block hash if one wants to start from a later point. This is not true for filters: they can be started processing from any point, without loss of confidence.

With this change, if one specifies an arbitrary height for filter startpoint:

  • the closest earlier known block hash is used for block header and filter header checkpoint
  • the provided height is used as starting point for filter download.

Anchor checkpoint can be set separately.

Background: This is useful in the 'balance checker' wallet use case, when the user may have the wallet birth height as an info (but no block hash); see Icepeek.

@rustaceanrob
Copy link
Owner

Hi, apologies I am just getting back from a holiday. I will review this soon

@rustaceanrob
Copy link
Owner

This is not true for filters: they can be started processing from any point, without loss of confidence.

Filters are keyed by the corresponding block hash. Without such a blockhash, there is no way to construct the filter. Also should be noted that the filter headers commit to the hash of the filter, which in turn commits to the block hash.

I am still unclear on the use case. Are you:

  1. Scanning filters after the anchor block header, ignoring some initial heights
  2. Attempting to check filters before the anchor block header

@optout21
Copy link
Author

optout21 commented Dec 6, 2024

The use case is this:
I, as a user, have the public info for my wallet: XPub, etc. I also have the hint for wallet birth height, i.e. the height of the first transaction in this wallet. Let's say it's 845,000. I don't have any block hashes, nor want I to look them up.
After entering this info in the kyoto-powered balance checker, we should:

  • automatically take a known earlier anchor point, in this case block 840,000.
  • download block headers starting from the anchor
  • download filter headers starting from the anchor
  • for filters we can start downloading later, so we can start with 845,000. Since the filter download is much slower than the header downloads, even a diff of 5000 blocks can make a noticeable difference.

@rustaceanrob
Copy link
Owner

rustaceanrob commented Dec 6, 2024

I see, so the differential between the checkpoint and when the relevant birthday is. May I ask why we must resync from the birthday every time? If you have recovered all your scripts, you may safely sync from the last chain tip you saw.

As a side note, I've tested your app and I concur with you that syncing feels slow. I may fork your app and see if I can improve the sync time in other ways.

@rustaceanrob
Copy link
Owner

Please rebase as there is a new CI script and the testing suite has changed.

@optout21
Copy link
Author

optout21 commented Dec 9, 2024

Please rebase

Rebased

@optout21
Copy link
Author

optout21 commented Dec 9, 2024

May I ask why we must resync from the birthday every time? If you have recovered all your scripts, you may safely sync from the last chain tip you saw.

If state has already synced, it's OK to continue from there. Only at the first sync it starts from the hint height, to speed up the process.

The typical use case is that you don't have a state, but always start from a new one. This may not be the best suited for kyoto (or compact block filters in general).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants