Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

snapshots: index builder refactorings #1922

Merged
merged 10 commits into from
Mar 26, 2024
Merged

snapshots: index builder refactorings #1922

merged 10 commits into from
Mar 26, 2024

Conversation

battlmonstr
Copy link
Contributor

@battlmonstr battlmonstr commented Mar 19, 2024

The same IndexBuilder is used for all indexes. It needs a descriptor (with settings) and a query (with input data). There are 2 types of index input data queries:

  1. DecompressorIndexInputDataQuery reads a snapshot (based on seg::Decompressor::Iterator) and produces key = word, value = offset.
  2. TransactionToBlockIndexInputDataQuery reads a TxsAndBodiesQuery and produces key = tx word, value = block_number. TxsAndBodiesQuery is a query that JOINs bodies + txs.

Each target index is configured in its own dedicated file. TransactionToBlockIndex is separated from the TransactionIndex.

Transaction indexes require a bodies_segment_path to be provided. capi also passes a bodies_segment_region.

Avoid get_os_temporary_path in tests: the tests were interfering with each other.

@battlmonstr battlmonstr force-pushed the pr/index_refac branch 5 times, most recently from a3f1e8f to b2191a7 Compare March 21, 2024 09:51
@battlmonstr battlmonstr marked this pull request as ready for review March 21, 2024 09:51
@battlmonstr battlmonstr changed the title snapshots: index refactorings snapshots: index builder refactorings Mar 21, 2024
@battlmonstr battlmonstr force-pushed the pr/index_refac branch 2 times, most recently from abd5ca0 to f05bc28 Compare March 21, 2024 17:34
index.build();
break;
}
case SnapshotType::transactions: {
TransactionIndex index{*snap_file};
auto bodies_segment_path = TransactionIndex::bodies_segment_path(*snap_file);
auto index = TransactionIndex::make(bodies_segment_path, *snap_file);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could merge these two lines into just one:

auto index = TransactionIndex::make(*snap_file);

looking exactly similar to the HeaderIndex and BodyIndex above, either by refactoring or overloading the current TransactionIndex::make. Next PR is fine

@canepat canepat merged commit 5dee979 into master Mar 26, 2024
4 checks passed
@canepat canepat deleted the pr/index_refac branch March 26, 2024 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants