Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #684 Abort S3 MultipartUpload if exception is raised #752

Closed
wants to merge 2 commits into from

Conversation

lociko
Copy link

@lociko lociko commented Jan 4, 2023

Motivation

Tests

test_write_gz_with_error

@mpenkov mpenkov marked this pull request as draft September 7, 2023 06:44
@mpenkov mpenkov changed the title WIP: Fix #684 Abort S3 MultipartUpload if exception is raised Fix #684 Abort S3 MultipartUpload if exception is raised Sep 7, 2023
# https://github.com/RaRe-Technologies/smart_open/issues/684
# If exception is raised multipart writer should be terminated to abort file uploading to S3
# If not incomplete file will be saved in bucket
if type(inner) == MultipartWriter and ex:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi 👋 can this be generalized such that it doesn't just handle S3, but also the other adapters like Azure?

Azure exception handling ref #783

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. by refactoring tweak_close to become tweak_exit, that patches .__exit__() instead of .close()?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened #786

@leeprevost
Copy link

What do I need to do to install this? I'm running into situation where larger transfers fail with this error:

botocore.exceptions.ClientError('An error occurred (EntityTooLarge) when calling the UploadPart operation: Your proposed upload exceeds the maximum allowed size')

@ddelange
Copy link
Contributor

ddelange commented Feb 24, 2024

@mpenkov
Copy link
Collaborator

mpenkov commented Feb 26, 2024

Closing, thank you

@mpenkov mpenkov closed this Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3 Multipart uploads comitted on exception when using compression via context manager
4 participants