-
-
Notifications
You must be signed in to change notification settings - Fork 382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new top-level compression parameter #609
Add new top-level compression parameter #609
Conversation
@piskvorky please review based on design discussion |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this contribution. The changes to the library itself mostly look good. The unit tests need a bit of work. Please have a look at my comments and let me know when you're ready for another round of reviews.
Co-authored-by: Michael Penkov <m@penkov.dev>
Co-authored-by: Michael Penkov <m@penkov.dev>
Co-authored-by: Michael Penkov <m@penkov.dev>
Co-authored-by: Michael Penkov <m@penkov.dev>
Co-authored-by: Michael Penkov <m@penkov.dev>
@mpenkov ready for another round of reviews |
@mpenkov @piskvorky ping |
Co-authored-by: Michael Penkov <m@penkov.dev>
I'll take it from here. Thank you for your contribution. |
Title
Feature to add compression parameter to override inference by file extension
Motivation
A previous PR was closed for being incompatible with the long-term API design for
smart_open.open
. Because the long-term goal, discussed in the linked issue is to add an enum-likecompression
parameter, and deprecate theignore_ext
parameter, which would be a breaking change, this is a backwards-compatible compromise that provides a clean upgrade path. Specifically, it preserves the existingignore_ext
parameter, instead adding aPendingDeprecationWarning
, and makes its use mutually-exclusive with the newcompression
parameter, verified by comprehensive unit testing (the truth tables used to build the test suites are provided in comments for any discussion of refinement needed). On the next major version, theignore_ext
parameter can be removed and the default forcompression
changed toINFER_FROM_EXTENSION
, since it won't be necessary to apply a MUTEX on the parameters, at that point (i.e.: lines 177-182 can just be removed).Fixes #607.
Tests
Added
test_smart_open.HandleS3CompressionTestCase
. This is specific to S3 (where the mocking is the best), but could be easily extended to any other backend. However, it looked as though S3 is the go-to backend for verifying shared behavior.Checklist
Before you create the PR, please make sure you have: