Skip to content
This repository has been archived by the owner on Mar 5, 2024. It is now read-only.

Make serialization/deserialization of ledger PoC records quicker #1134

Open
Vagabond opened this issue Nov 30, 2021 · 3 comments
Open

Make serialization/deserialization of ledger PoC records quicker #1134

Vagabond opened this issue Nov 30, 2021 · 3 comments
Assignees

Comments

@Vagabond
Copy link
Contributor

We currently store a t2b'd list of t2b'd ledger_poc_v2 records. This record itself is full of binaries, so we don't preserve sub binary refernces (using more ram) and t2b over t2b seems silly.

Propose we use a real binary packing scheme to pack the binaries, and the list of the poc records more efficently so we can pull them out of the db faster with less ram usage.

@xandkar xandkar self-assigned this Dec 4, 2021
@evanmcc
Copy link
Contributor

evanmcc commented Jan 12, 2022

I would propose additionally that we calculate and store the maximum height of any poc in any given poc list. This will allow us to GC without decoding, which will speed that process up immensely.

@evanmcc
Copy link
Contributor

evanmcc commented Jan 12, 2022

By store, I mean <<Vers:1/bytes, MaxHeight:64/unsigned-native, KVList/binary>> or something similar, so that we don't need to decode the kvlist at all, just iterate over the CF and delete once the entire list is available for GC.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants