Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generify ColumnReaderImpl and RecordReader (#1040) #1041

Merged
merged 20 commits into from
Jan 11, 2022
Merged

Generify ColumnReaderImpl and RecordReader (#1040) #1041

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
9930e50
Simplify record reader
tustvold Dec 10, 2021
be0fc1b
Generify ColumnReaderImpl and RecordReader (#1040)
tustvold Dec 9, 2021
8457562
Tweak count_records predicate
tustvold Dec 13, 2021
cdc9d69
Pre-allocate bitmask
tustvold Dec 14, 2021
2a6b576
Merge branch 'master' into generify-column-reader
tustvold Dec 14, 2021
0469ffc
fix: TypedBuffer::split update len
tustvold Dec 14, 2021
d5c5dd9
Simplify GenericRecordReader
tustvold Dec 15, 2021
dec899c
Move column decoders into module
tustvold Dec 15, 2021
90d1399
Remove `RecordBuffer::create` method
tustvold Dec 15, 2021
e52c569
Remove `TypedBuffer<i16>::count_records`
tustvold Dec 15, 2021
14d917e
Pass null count to `ColumnValueDecoder::read`
tustvold Dec 16, 2021
cb18695
Pull null padding out of column reader
tustvold Dec 16, 2021
cb250ae
Review feedback
tustvold Jan 1, 2022
4c11590
Format
tustvold Jan 1, 2022
0fe966a
License headers
tustvold Jan 1, 2022
6a21ad2
Further doc tweaks
tustvold Jan 1, 2022
28228b2
Further docs
tustvold Jan 4, 2022
5e6bb01
Merge remote-tracking branch 'upstream/master' into generify-column-r…
tustvold Jan 11, 2022
48b6d62
Restrict ScalarBuffer types
tustvold Jan 11, 2022
c6f98f7
Merge remote-tracking branch 'upstream/master' into generify-column-r…
tustvold Jan 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions parquet/src/arrow/array_reader.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,6 @@ pub struct PrimitiveArrayReader<T: DataType> {
rep_levels_buffer: Option<Buffer>,
column_desc: ColumnDescPtr,
record_reader: RecordReader<T>,
_type_marker: PhantomData<T>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seemed to be an orphan so I just removed it

}

impl<T: DataType> PrimitiveArrayReader<T> {
Expand Down Expand Up @@ -230,7 +229,6 @@ impl<T: DataType> PrimitiveArrayReader<T> {
rep_levels_buffer: None,
column_desc,
record_reader,
_type_marker: PhantomData,
})
}
}
Expand Down
Loading