Sperate get_next_page_header from get_next_page in PageReader

[Ted-Jiang]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

pub trait PageReader: Iterator<Item = Result<Page>> + Send {
    /// Gets the next page in the column chunk associated with this reader.
    /// Returns `None` if there are no pages left.
    fn get_next_page(&mut self) -> Result<Option<Page>>;
}

change to

pub trait PageReader: Iterator<Item = Result<Page>> + Send {
    /// Gets the next page in the column chunk associated with this reader.
    /// Returns `None` if there are no pages left.
    fn get_next_page(&mut self) -> Result<Option<Page>>;

    /// Gets the next page in the column chunk associated with this reader.
    /// Returns `None` if there are no pages left.
    fn get_next_page_header(&mut self) -> Result<Option<PageHeader>>;
}

i want to add skip_page in PageReader (for page index) need check page header before call skip_page (
can not skip DictionaryPage IndexPage),
but now read header is in get_next_page, so extract it to a new func.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
put skip_page in get_next_page

Additional context
Add any other context or screenshots about the feature request here.

[Ted-Jiang]

@tustvold PTAL

[tustvold]

I could be missing something, but I was under the impression the intention of the ColumnIndex, as opposed to using page-level statistics, was precisely to avoid needing to read any page data for those being skipped? Is this information not encoded in the PageLocation index or something?

Note also, if there is a dictionary page, which there likely will be if ColumnChunkMetaData::encodings suggests it, it MUST be the first one in the column chunk - https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8

[Ted-Jiang]

@tustvold Thanks a lot, i miss the dictionary pateMUST be the first one in the column chunk part.

Last two weeks, i busy about my personal things. I will go back and work on page index!
Thanks again for your info ❤️