Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to store and retrieve deltas #5921

Open
urtho opened this issue Jan 25, 2024 · 2 comments
Open

Add option to store and retrieve deltas #5921

urtho opened this issue Jan 25, 2024 · 2 comments

Comments

@urtho
Copy link
Contributor

urtho commented Jan 25, 2024

Status

Currently node stores a few deltas in memory only.

Expected

Node optionally stores deltas along with blocks and allows for retrieval from storage.
This could be available only with PebbleDB compressed storage to not go beyond current full archival SQLite storage requirements.

Imagine a follower-less infra once dedicated archivers have this option enabled.

Urgency

Alognode.io built a dedicated pipeline and a block server just for that - to solve the jittery nature of the current conduit/follower pipeline.

https://github.com/AlgoNode/blocksrv
https://github.com/AlgoNode/conduit-plugin-export-blksrv

@jannotti
Copy link
Contributor

Are you wanting to keep the state delta for the entire block, or the detailed deltas for individual transactions?

@urtho
Copy link
Contributor Author

urtho commented Feb 2, 2024

My interest is just deltas for entire block - that is minimal requirement for conduit.
Others are interested in TX deltas but that would probably blow up the storage requirements tenfold.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants