-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
This allows users to implement #12547 (caching metadata, especially the Page Index) themselves without reimplementing all of ParquetOpener
and ParquetExec
Describe the solution you'd like
Add a method to the object returned by ParquetFileReaderFactory
that loads metadata, so that users who overwrite the DefaultParquerFileReaderFactory
with SessionState::register_file_format(..., true)
can provide their own implementation.
Describe alternatives you've considered
Adding caching support directly in Datafusion. I tried in #12548 but it went nowhere.
Support for caching directly in Datafusion can be added later, based on this feature.
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request