Propagate __exit__ call to underlying filestream #786
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Propagate
__exit__
call to underlying filestream.No more need for
tweak_close
function: the double wrapping will now cause three calls to__exit__
, which in turn will callclose
, first on the outermost filestream, working inward.If a third party compression handler calls
close
on the inner filestream, the inner__exit__
call will do a second call toclose
on the inner filestream, which will be ignored.Motivation
Currently, the compression mechanic has a side-effect that
__exit__
is no longer called on the original filestream. Consequently, eg boto3 abort multipart upload is not called, causing lingering blobs that will be billed.Tests
✅
Work in progress
Checklist
Before you create the PR, please make sure you have:
Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.