Chain storage (MS5.9)

[sandtreader]

Summary

On-disk storage of blocks and transactions

Intersect Milestone

MS5.9

Description

"On-disk storage of blocks"

Acceptance Criteria

"Able to store all received blocks on disk and efficiently recover them"

Expected Deliverables

"- Block store module - Test reports"

[buddhisthead]

DEMO results:

When you run the omnibus process, all the blocks on the chain are written to a fjall partition, along with some index partitions used for looking up blocks through various different identifiers. Starting with no fjall-blocks directory, and after running omnibus for a bit (in this case to epoch 113), the fjall-blocks directory usage will look something like this:

112612 fjall-blocks/partitions/txs/segments
112628 fjall-blocks/partitions/txs
98780 fjall-blocks/partitions/block-hashes-by-number/segments
98796 fjall-blocks/partitions/block-hashes-by-number
101596 fjall-blocks/partitions/block-hashes-by-epoch-slot/segments
101612 fjall-blocks/partitions/block-hashes-by-epoch-slot
99088 fjall-blocks/partitions/block-hashes-by-slot/segments
99104 fjall-blocks/partitions/block-hashes-by-slot
2822024 fjall-blocks/partitions/blocks/segments
2822040 fjall-blocks/partitions/blocks
3234184 fjall-blocks/partitions
85664 fjall-blocks/journals
3319856 fjall-blocks/

And it will continue to grow as more blocks flow in from the omnibus process. The blockfrost API that omnibus exposes can be used to query this store, and you'll get results like this one:

$ curl -w '\n' http://localhost:4340/blocks/cb73dc0a4a26721e569ba23fa7592d00329a2fb269043b071f1a9bc3ffffd5aa
{
"time": 1516110811,
"height": 495323,
"hash": "cb73dc0a4a26721e569ba23fa7592d00329a2fb269043b071f1a9bc3ffffd5aa",
"slot": 495386,
"epoch": 22,
"epoch_slot": 20186,
"slot_issuer": "Byron genesis slot issuer",
"size": 1741,
"tx_count": 3,
"output": 5701889745091,
"fees": null,
"block_vrf": null,
"op_cert": null,
"op_cert_counter": null,
"previous_block": "041bc319f5524f072e22ce784923049419ef650a943b37bff784ce08078a565c",
"next_block": "689a2fb3f38abc89c379b67aef435105e5139d199d7dbabff55b71cb173913e7",
"confirmations": 1413
}