ARROW-9678: [Rust] [DataFusion] Improve projection push down to remove unused columns #7919
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jorgecarleitao
commented
Aug 14, 2020
|
FYI @andygrove and @alamb Again, this is useful mostly to the |
alamb
approved these changes
Aug 15, 2020
This commit makes the projection optimizer remove any projection or aggregation that is not used down the plan.
|
@andygrove , is there anything we need to work this further? |
|
I do have a nagging concern that the logic may not work if the query plan contains aliases that rename columns, but we can address that as a follow up if/when that becomes an issue. |
|
Thank you, @andygrove ! I encapsulated that thought on ARROW-9830, with issue type "Test". :) |
emkornfield
pushed a commit
to emkornfield/arrow
that referenced
this pull request
Oct 16, 2020
…e unused columns This PR makes the projection optimizer remove any projection or aggregation that is not used down the plan, thus improving speed and convenience. This is worked on top of apache#7879 and only the last commit is specific to this PR. Closes apache#7919 from jorgecarleitao/projection Authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com> Signed-off-by: Andy Grove <andygrove73@gmail.com>
GeorgeAp
pushed a commit
to sirensolutions/arrow
that referenced
this pull request
Jun 7, 2021
…e unused columns This PR makes the projection optimizer remove any projection or aggregation that is not used down the plan, thus improving speed and convenience. This is worked on top of apache#7879 and only the last commit is specific to this PR. Closes apache#7919 from jorgecarleitao/projection Authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com> Signed-off-by: Andy Grove <andygrove73@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes the projection optimizer remove any projection or aggregation that is not used down the plan, thus improving speed and convenience.
This is worked on top of #7879 and only the last commit is specific to this PR.