Skip to content

Reading bz2-compressed tables from URL fails with c-engine in Python 2 #14874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dhimmel opened this issue Dec 13, 2016 · 0 comments
Closed

Reading bz2-compressed tables from URL fails with c-engine in Python 2 #14874

dhimmel opened this issue Dec 13, 2016 · 0 comments
Labels
IO CSV read_csv, to_csv IO Data IO issues that don't fit into a more specific label
Milestone
0.20.0

Comments

@dhimmel
Copy link
Contributor

dhimmel commented Dec 13, 2016

In #14576, we consolidated compression code and introduced support for compressed URLs beyond just gzip. However, as described in #14576 (comment), the c-engine cannot read bz2-compressed URLs in python 2. This error is coming from parser.pyx#L615-L621. There may be potential to entirely bypass this offending code as it appears to duplicate the functionality of _get_handle.
@dhimmel dhimmel mentioned this issue Dec 13, 2016
Closed
@jreback jreback added IO Data IO issues that don't fit into a more specific label Difficulty Intermediate IO CSV read_csv, to_csv labels Dec 13, 2016
@jreback jreback added this to the Next Major Release milestone Dec 13, 2016
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016
@dhimmel
Refactor compression code to expand URL support
8798932 
Part of pandas-dev#14576.
Closes pandas-dev#12688.
Closes pandas-dev#14570.
Opens pandas-dev#14874.
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016
@dhimmel
Refactor compression code to expand URL support
b33a500 
Part of pandas-dev#14576.
Closes pandas-dev#12688.
Closes pandas-dev#14570.
Opens pandas-dev#14874.
jreback pushed a commit that referenced this issue Dec 13, 2016
@dhimmel @jreback
CLN: Refactor compression code to expand URL support
3761448 
closes #14576
closes #12688
closes #14570
xref #14874
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 14, 2016
@dhimmel
ENH: Support bz2 compression in PY2 for c engine
9fb40da 
Closes pandas-dev#14874
@dhimmel dhimmel mentioned this issue Dec 14, 2016
Closed
1 task
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 14, 2016
@dhimmel
ENH: Support bz2 compression in PY2 for c engine
aebad5a 
Closes pandas-dev#14874
@jorisvandenbossche jorisvandenbossche modified the milestones: 0.20.0, Next Major Release Dec 15, 2016
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 15, 2016
@dhimmel
ENH: Support bz2 compression in PY2 for c engine
85630ea 
Closes pandas-dev#14874
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 16, 2016
@dhimmel
TST: Read bz2 files from S3 in PY2
6c632db 
Addresses pandas-dev#14874
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 16, 2016
@dhimmel
TST: Read bz2 files from S3 in PY2
8568aed 
Addresses pandas-dev#14874
ischurov pushed a commit to ischurov/pandas that referenced this issue Dec 19, 2016
@dhimmel @ischurov
CLN: Refactor compression code to expand URL support
a0e27fb 
closes pandas-dev#14576
closes pandas-dev#12688
closes pandas-dev#14570
xref pandas-dev#14874
ischurov pushed a commit to ischurov/pandas that referenced this issue Dec 19, 2016
@dhimmel @ischurov
DOC for refactored compression (GH14576) + BUG: bz2-compressed URL wi…
da406b5 
…th C engine (GH14874)

Follow up on pandas-dev#14576, which
refactored compression code to expand URL support.    Fixes up some
small remaining issues and adds a what's new entry.    - [x] Closes
pandas-dev#14874

Author: Daniel Himmelstein <daniel.himmelstein@gmail.com>

Closes pandas-dev#14880 from dhimmel/whats-new and squashes the following commits:

e1b5d42 [Daniel Himmelstein] Address what's new review comments
8568aed [Daniel Himmelstein] TST: Read bz2 files from S3 in PY2
09dcbff [Daniel Himmelstein] DOC: Improve what's new
c4ea3d3 [Daniel Himmelstein] STY: PEP8 fixes
f8a7900 [Daniel Himmelstein] TST: check bz2 compression in PY2 c engine
0e0fa0a [Daniel Himmelstein] DOC: Reword get_filepath_or_buffer docstring
210fb20 [Daniel Himmelstein] DOC: What's New for refactored compression code
cb91007 [Daniel Himmelstein] TST: Read compressed URLs with c engine
85630ea [Daniel Himmelstein] ENH: Support bz2 compression in PY2 for c engine
a7960f6 [Daniel Himmelstein] DOC: Improve _infer_compression docstring
ShaharBental pushed a commit to ShaharBental/pandas that referenced this issue Dec 26, 2016
@dhimmel @ShaharBental
DOC for refactored compression (GH14576) + BUG: bz2-compressed URL wi…
40546f9 
…th C engine (GH14874)

Follow up on pandas-dev#14576, which
refactored compression code to expand URL support.    Fixes up some
small remaining issues and adds a what's new entry.    - [x] Closes
pandas-dev#14874

Author: Daniel Himmelstein <daniel.himmelstein@gmail.com>

Closes pandas-dev#14880 from dhimmel/whats-new and squashes the following commits:

e1b5d42 [Daniel Himmelstein] Address what's new review comments
8568aed [Daniel Himmelstein] TST: Read bz2 files from S3 in PY2
09dcbff [Daniel Himmelstein] DOC: Improve what's new
c4ea3d3 [Daniel Himmelstein] STY: PEP8 fixes
f8a7900 [Daniel Himmelstein] TST: check bz2 compression in PY2 c engine
0e0fa0a [Daniel Himmelstein] DOC: Reword get_filepath_or_buffer docstring
210fb20 [Daniel Himmelstein] DOC: What's New for refactored compression code
cb91007 [Daniel Himmelstein] TST: Read compressed URLs with c engine
85630ea [Daniel Himmelstein] ENH: Support bz2 compression in PY2 for c engine
a7960f6 [Daniel Himmelstein] DOC: Improve _infer_compression docstring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
IO CSV read_csv, to_csv IO Data IO issues that don't fit into a more specific label
Projects
None yet
Milestone
0.20.0
Development

No branches or pull requests
3 participants
@jreback @jorisvandenbossche @dhimmel