Skip to content

Conversation

@leoleoasd
Copy link
Contributor

This PR adds support append mode for ArrowFSWrapper.

https://github.com/apache/arrow/blob/apache-arrow-1.0.0/python/pyarrow/_fs.pyx#L673

It seems that from v1.0.0, pyarrow supports open_append_stream.

@martindurant
Copy link
Member

Can we please add a test for this in test_arrow.py?

@leoleoasd leoleoasd force-pushed the patch-1 branch 3 times, most recently from 366f936 to 1f47d8a Compare October 28, 2022 14:14
@leoleoasd
Copy link
Contributor Author

Hi, I've added tests. Works fine in my repo (https://github.com/leoleoasd/filesystem_spec/actions/runs/3346115981), but I need approval for running tests in PR.
image

@martindurant martindurant merged commit c72bf7f into fsspec:master Oct 28, 2022
@leoleoasd
Copy link
Contributor Author

Hi, I would like to ask when will we release this into PyPI, is there an ETA?
Since our cluster only supports installing packages from PyPI, I need to apply a patch into site-packages every time I submit a new job before the release.

@martindurant
Copy link
Member

We have a few things coming in now, so maybe within the next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants