Make `push_batch_with_filter` up to 3x faster#8951

Open

Dandandan wants to merge 106 commits intoapache:mainfrom

Dandandan:coalesce_batches_filter

Commits on Dec 5, 2025

perf
Dandandan
committed
comment
Dandandan
committed

Commits on Dec 6, 2025

Commits on Dec 8, 2025

fix comments
Dandandan
committed

Commits on Dec 9, 2025

Commits on Dec 13, 2025

Commits on Dec 15, 2025

Move / optimize
Dandandan
committed

Commits on Jan 8, 2026

Merge branch 'main' into coalesce_batches_filter
Dandandan
authored

Commits on Jan 15, 2026

docs: Update release schedule in README.md (#9111)

alamb
authored and
Dandandan
committed
feat: add benchmarks for json parser (#9107)

Weijun-H
authored and
Dandandan
committed
chore: switch test from `bincode` to maintained `postcard` crate (RUSTSEC-2025-0141 ) (#9104)

alamb
authored and
Dandandan
committed
Speed up binary kernels (30% faster `and` and `or`), add `BooleanBuffer::from_bitwise_binary_op` (#9090)

alamb
authored and
Dandandan
committed
[Variant] Optimize the object header generation logic in ObjectBuilder::finish (#8031)

klion26
authored and
Dandandan
committed
Update readme for geospatial crate (#9124)

paleolimbot
authored and
Dandandan
committed
Remove parquet arrow_cast dependency (#9077)

authored and
Dandandan
committed
Updated arrow-pyarrow to use pyo3 0.27, updated deprecated code warnings (#8773)

authored and
Dandandan
committed
Fix clippy (#9130)

alamb
authored and
Dandandan
committed
fix: display `0 secs` for empty DayTime/MonthDayNano intervals (#9023)

authored and
Dandandan
committed
Fix IPC roundtripping dicts nested in ListViews (#9126)

brancz
authored and
Dandandan
committed
docs(parquet): add example for preserving dictionary encoding (#9116)

AndreaBozzo
authored and
Dandandan
committed
Add options to skip decoding `Statistics` and `SizeStatistics` in Parquet metadata (#9008)

etseidl
authored and
Dandandan
committed
[arrow] Minimize allocation in GenericViewArray::slice() (#9016)

authored and
Dandandan
committed
bench: added to row_format benchmark conversion of 53 non-nested columns (#9081)

authored and
Dandandan
committed
doc: fix link on FixedSizeListArray doc (#9033)

Jefffrey
authored and
Dandandan
committed
Docs: Add additional documentation and example for `make_array` (#9112)

alamb
authored and
Dandandan
committed
Change FlightSQLClient to return `FlightError` & cleanup code (#8916)

authored and
Dandandan
committed
Avoid overallocating arrays in coalesce primitives / views (#9132)
Dandandan
committed
perf: optimize hex decoding in json (1.8x faster in binary-heavy) (#9091)

Weijun-H
authored and
Dandandan
committed
Add nullif_kernel benchmark (#9089)

alamb
authored and
Dandandan
committed
perf: Avoid ArrayData allocation in PrimitiveArray::reinterpret_cast (#9129)

authored and
Dandandan
committed
Add `BooleanBufferBuilder::extend_trusted_len` (#9137)
Dandandan
committed
docs: Improve main README.md and highlight community (#9119)

authored and
Dandandan
committed
fix: support cast from `Null` to list view/run encoded/union types (#9134)

Jefffrey
authored and
Dandandan
committed
docs(variant): fix VariantObject::get documentation to reflect Option return type (#9139)

mohit7705
authored and
Dandandan
committed
Avoid clones in `make_array` for `StructArray` and `GenericByteViewArray` (#9114)

authored and
Dandandan
committed
Uncomment part of test_utf8_single_column_reader_test (#9148)

sdf-jkl
authored and
Dandandan
committed
Update ASF copyright year in NOTICE (#9145)

mohit7705
authored and
Dandandan
committed
fix:[9018]Fixed RunArray slice offsets (#9036)

manishkr
authored and
Dandandan
committed
arrow-ipc: Add tests for nested dicts for Map and Union arrays (#9146)

brancz
authored and
Dandandan
committed
feat: add `reserve` to `Rows` (#9142)

rluvaton
authored and
Dandandan
committed
perf: improve calculating length performance for view byte array in row conversion (#9080)

authored and
Dandandan
committed
[Parquet] perf: Create `PrimitiveArray`s directly rather than via `ArrayData` (#9122)

alamb
authored and
Dandandan
committed
doc: add example of RowFilter usage (#9115)

sonhmai
authored and
Dandandan
committed
[Parquet] perf: Create StructArrays directly rather than via `ArrayData` (1% improvement) (#9120)

alamb
authored and
Dandandan
committed
perf: improve field indexing in JSON StructArrayDecoder (1.7x speed up) (#9086)

Weijun-H
authored and
Dandandan
committed
Minor: try and avoid an allocation creating `GenericByteViewArray` from `ArrayData` (#9156)

alamb
authored and
Dandandan
committed
Avoid a clone when creating `BooleanArray` from ArrayData (#9159)

alamb
authored and
Dandandan
committed
Avoid a clone when creating StringArray/BinaryArray from ArrayData (#9160)

alamb
authored and
Dandandan
committed
perf: improve calculating length performance for `GenericByteArray` in row conversion (#9078)

authored and
Dandandan
committed
perf: improve calculating length performance for nested arrays in row conversion (#9079)

authored and
Dandandan
committed
lint: remove unused function (fix clippy (#9178)

rluvaton
authored and
Dandandan
committed
Add `TimestampWithOffset` canonical extension type (#8743)

serramatutu
authored and
Dandandan
committed
feat: add null comparison handling in make_comparator (#9150)

Weijun-H
authored and
Dandandan
committed
docs: update examples in ArrowReaderOptions to use in-memory buffers (#9163)

AndreaBozzo
authored and
Dandandan
committed
refactor: streamline date64 tests (#9165)

authored and
Dandandan
committed
Improve `ArrowReaderBuilder::with_row_filter` documentation (#9153)

authored and
Dandandan
committed
add `#[inline]` to `BitIterator` `next` function (#9177)

rluvaton
and
Dandandan
committed
Merge
Dandandan
committed
fix missing utf8 check for conversion from BinaryViewArray to StringViewArray (#9158)

authored and
Dandandan
committed
Add find_nth_set_bit_position (#9151)
Dandandan
committed
Merge
Dandandan
committed
Merge
Dandandan
committed
Doc fix
Dandandan
committed

Commits on Jan 16, 2026

Merge branch 'main' into coalesce_batches_filter
Dandandan
authored

Commits on Jan 20, 2026