Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multilayer sstable #2246

Closed
wants to merge 4 commits into from
Closed

multilayer sstable #2246

wants to merge 4 commits into from

Conversation

trinity-1686a
Copy link
Contributor

fix #1948
related to quickwit-oss/quickwit#3964

implement a new file format for sstable which allows much faster opening of very large table, at the cost of somewhat slower individual queries to the table
current results:

before:
        ord_to_term_suffix           time:   [15.442 µs 15.452 µs 15.464 µs]
        open_and_ord_to_term_suffix  time:   [37.591 ms 37.638 ms 37.688 ms]
        ord_to_term                  time:   [65.466 µs 65.483 µs 65.503 µs]
        open_and_ord_to_term         time:   [8.6174 ms 8.6311 ms 8.6474 ms]
after:
        ord_to_term_suffix           time:   [41.148 µs 41.222 µs 41.320 µs]
        open_and_ord_to_term_suffix  time:   [81.646 µs 81.663 µs 81.679 µs]
        ord_to_term                  time:   [92.304 µs 92.331 µs 92.360 µs]
        open_and_ord_to_term         time:   [99.245 µs 99.653 µs 100.41 µs]

@fulmicoton
Copy link
Collaborator

Can you try an fst table? (It should faster... build times should be impacted however.)

@trinity-1686a
Copy link
Contributor Author

superseded by #2268

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SSTableIndex could be a multi layer SSTable
2 participants