-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow configuring parquet filter pushdown dynamically #3821
Labels
enhancement
New feature or request
Comments
FYI @Ted-Jiang @thinkharderdev and @tustvold -- I should have a PR up with this shortly (it rejiggers the config stuff to be more dynamic) |
This was referenced Oct 13, 2022
This was referenced Nov 29, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I want to test out the parquet filter pushdown on real datasets using datafusion-cli so we can enable it by default -- #3463
To do so I want to both understand what the current setting of the pushdown flags are as well as change them statement by statement.
Describe the solution you'd like
Use the
ConfigOptions
to control parquet scanning options rather than anothr structureAmong other things that will allow the parquet settings to appear here as well as be controlled by environment variable in datafusion-cli.
It will also allow this feature to be turned off if we find an issue
Describe alternatives you've considered
Can just do it programmatically
Additional context
See #3463
The text was updated successfully, but these errors were encountered: