Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TST: DataFrame.to_parquet accepts pathlib.Path with partition_cols defined #36491

Merged
merged 3 commits into from
Sep 23, 2020
Merged

TST: DataFrame.to_parquet accepts pathlib.Path with partition_cols defined #36491

merged 3 commits into from
Sep 23, 2020

Conversation

arw2019
Copy link
Member

@arw2019 arw2019 commented Sep 20, 2020

@WillAyd
Copy link
Member

WillAyd commented Sep 21, 2020

Does this require bumping the minimum pyarrow version? @jorisvandenbossche

@WillAyd WillAyd added the Testing pandas testing functions or related to the test suite label Sep 21, 2020
@jreback jreback added the IO Parquet parquet, feather label Sep 21, 2020
@jreback
Copy link
Contributor

jreback commented Sep 21, 2020

hmm, how / why does this pass on older pyarrow?

jreback
jreback requested changes Sep 21, 2020
View reviewed changes
pandas/tests/io/test_parquet.py Outdated Show resolved Hide resolved
@WillAyd
Copy link
Member

WillAyd commented Sep 22, 2020

code lgtm just unclear on pyarrow versioning and why this would be working now - can you comment @arw2019 ?

@arw2019
Copy link
Member Author

arw2019 commented Sep 23, 2020

code lgtm just unclear on pyarrow versioning and why this would be working now - can you comment @arw2019 ?

It was fixed on the pandas side in #35681.

So it will work in 1.2 with pyarrow 0.15+ (I have checked current master against 0.15, 0.17 and 1.0 pyarrow)

@jorisvandenbossche jorisvandenbossche merged commit 2ed549d into pandas-dev:master Sep 23, 2020
@jorisvandenbossche
Copy link
Member

Thanks @arw2019 !

@simonjayhawkins simonjayhawkins added this to the 1.2 milestone Sep 23, 2020
@arw2019
Copy link
Member Author

arw2019 commented Sep 23, 2020

thanks @jorisvandenbossche @WillAyd @jreback for reviewing!

kesmit13 pushed a commit to kesmit13/pandas that referenced this pull request Nov 2, 2020
@arw2019
TST: DataFrame.to_parquet accepts pathlib.Path with partition_cols de…
eecfb49 
…fined (pandas-dev#36491)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback requested changes

@jorisvandenbossche jorisvandenbossche jorisvandenbossche approved these changes

Assignees
No one assigned
Labels
IO Parquet parquet, feather Testing pandas testing functions or related to the test suite
Projects
None yet
Milestone
1.2
Development

Successfully merging this pull request may close these issues.

BUG: to_parquet does not accept pathlib.PosixPath if partition_cols are defined
5 participants
@arw2019 @WillAyd @jreback @jorisvandenbossche @simonjayhawkins