Add FilterPredicate::filter_record_batch
#8693
Conversation
There was a problem hiding this comment.
Thank you @pepijnve -- this looks great to me
I verified there is existing coverage for filter_record_batch from which this is now called 👍
https://docs.rs/arrow-select/56.2.0/src/arrow_select/filter.rs.html#1419-1427
https://docs.rs/arrow-select/56.2.0/src/arrow_select/filter.rs.html#2095
| /// Returns a filtered [`RecordBatch`] containing only the rows that are selected by this | ||
| /// [`FilterPredicate`]. | ||
| /// | ||
| /// This is the equivalent of calling [filter] on each column of the [`RecordBatch`]. |
There was a problem hiding this comment.
It is also the equivalent of calling https://docs.rs/arrow/latest/arrow/compute/fn.filter_record_batch.html
| /// This is the equivalent of calling [filter] on each column of the [`RecordBatch`]. | |
| /// This is the equivalent of calling [filter] on each column of the [`RecordBatch`]. | |
| /// | |
| /// See also [`filter_record_batch`] |
There was a problem hiding this comment.
I added some extra documentation on the convenience free standing functions related to FilterPredicate reuse. I'm not sure how useful it is to have circular documentation references from FilterPredicate back to those.
| ) -> Result<RecordBatch, ArrowError> { | ||
| let mut filter_builder = FilterBuilder::new(predicate); | ||
| if record_batch.num_columns() > 1 { | ||
| let num_cols = record_batch.num_columns(); |
There was a problem hiding this comment.
this is also a nice improvement (to also account for arrays with multiple children)
There was a problem hiding this comment.
Stolen from filter(&dyn Array, &BooleanArray) which was already doing this.
Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
3 participants
Which issue does this PR close?
FilterPredicateinstances to be reused for RecordBatches #8692.Rationale for this change
Explained in issue.
What changes are included in this PR?
FilterPredicate::filter_record_batchfilter_record_batchto use the new functionnew_uncheckedto create the filtered result. The rationale for this is identical to Skip redundant validation checks in RecordBatch#project #8583Are these changes tested?
Covered by existing tests for
filter_record_batchAre there any user-facing changes?
No