Fix multicolumn parquet predicate pushdown (#4046) #4048
Conversation
There was a problem hiding this comment.
Thank you @tustvold
This is all so familiar -- we have very similar logic in IOx in https://github.com/influxdata/influxdb_iox/blob/main/schema/src/sort.rs (though I don't see any obvious way to reuse it)
| @@ -278,6 +274,32 @@ impl<'a> ExprRewriter for FilterCandidateBuilder<'a> { | |||
| } | |||
| } | |||
|
|
|||
| /// Computes the projection required to go from the file's schema order to the projected | |||
|
I verified this fixes the issue in my test harness in #3976 -- given this is a non controversial bug fix I am merging it in now to keep that project chugging along |
|
Benchmark runs are scheduled for baseline = c92a38f and contender = ea31da9. ea31da9 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
* Fix multicolumn parquet predicate pushdown (apache#4046) * Format
Which issue does this PR close?
Closes #4046
Rationale for this change
The first incarnation of this in #4021 computed the mapping from the projected schema to the file schema and not the other way round. For the case of two columns these are always the same thing and so it worked for the two column case, but more columns requires additional shenanigans.
What changes are included in this PR?
Are there any user-facing changes?