Add filter pushdown example (#1191) #2201
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tustvold
commented
Jul 27, 2022
| @@ -35,7 +35,7 @@ pub fn read_columns_indexes<R: ChunkReader>( | |||
| let length = lengths.iter().sum::<usize>(); | |||
|
|
|||
| //read all need data into buffer | |||
| let mut reader = reader.get_read(offset, reader.len() as usize)?; | |||
| let mut reader = reader.get_read(offset, length)?; | |||
There was a problem hiding this comment.
These were actually bugs that were only working because the ChunkReader implementation for File doesn't validate the end offset is actually within the range of the file.
* Change CI names to match crates * Update .github/workflows/arrow_flight.yml Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com> Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
|
Rebasing on top of #2199 appears to fix the "failed to skip rows" issue, now just to work out why the data is incorrect 😅 Edit: I think the issue might be in the filtering code 🤦 |
tustvold
force-pushed
the
add-filter-pushdown-example
branch
from
43b5ea3
to
1bd4dc5
Compare
This was referenced Jul 27, 2022
|
Closing in favor or #2335 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Draft as makes APIs public that we probably want to keep private until we have better test coverage #2198
Adds an example of filter evaluation as described in #1191
Possibly relates to #2198