Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-hive partitioning schema for parquet files #1828

Open
junjunjd opened this issue Nov 9, 2023 · 0 comments
Open

Support non-hive partitioning schema for parquet files #1828

junjunjd opened this issue Nov 9, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@junjunjd
Copy link
Contributor

junjunjd commented Nov 9, 2023

Description

Currently DeltaTablePartition::try_from only supports hive partitioning. If a parquet path has non-hive parititioning, DeltaTablePartition::try_from will throw an error. It would be nice to support some non-hive partitioning schemes such as the directory scheme like ./partitioned/2000/part-0.parquet. User will need to provide the column name when they use this scheme.
Use Case
This feature would make ConvertToDelta operation applicable to non-hive parquet tables.
Related Issue(s)
#1041, #1682, #1686

@junjunjd junjunjd added the enhancement New feature or request label Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant