Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 has limits with respect to quantity of parts uploaded #1096

Closed
mmattel opened this issue Dec 17, 2024 · 0 comments · Fixed by #1097
Closed

S3 has limits with respect to quantity of parts uploaded #1096

mmattel opened this issue Dec 17, 2024 · 0 comments · Fixed by #1097

Comments

@mmattel
Copy link
Contributor

mmattel commented Dec 17, 2024

References: owncloud/ocis#10775 (S3 storage truncated to 250000MB per object without error)

  • S3 has a hard limit of 10.000 parts per multipart upload.
    Defined by the S3 provider !!!
  • The default part size is 16MB.
  • The maximum part size allowed is 5GB (5 * 1024 * 1024 * 1024)

This results in the following:

  • With default part size
    max file size: 160GB
  • With max part size
    max file size: 50TB --> busts the single object limit of 5TB when using Wasabi S3

Configs:

  • Disable multi part upload:
    STORAGE_USERS_S3NG_PUT_OBJECT_CONCURRENT_STREAM_PARTS
  • Define part size:
    STORAGE_USERS_S3NG_PUT_OBJECT_PART_SIZE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant