RClone requires S3 setting no-check-bucket #2447
Labels
Comments
arielshaqed
added
docs
arielshaqed
added a commit
that referenced
this issue
Sep 5, 2021
When copying to a new key, Rclone likes to try to create _all_ the S3 path down (it seems to think it's on a POSIXy filesystem so it `mkdir -p`s). A repository cannot be created by the gateway -- there is not enough info on the request -- but if we return a 409 (conflict) then Rclone continues and succeeds. There is no S3 gateway test, so tested manually using Rclone. Fixes #2447.
arielshaqed
added a commit
that referenced
this issue
Sep 5, 2021
…2451) * [S3 gateway] Return HTTP 409 (Conflict) when creating existing repo When copying to a new key, Rclone likes to try to create _all_ the S3 path down (it seems to think it's on a POSIXy filesystem so it `mkdir -p`s). A repository cannot be created by the gateway -- there is not enough info on the request -- but if we return a 409 (conflict) then Rclone continues and succeeds. There is no S3 gateway test, so tested manually using Rclone. Fixes #2447. * [CR] Require CreateRepositoryAction (not ReadRepositoryAction)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Pointed out by @tvanderwal-lmco as part of #2445.
When rclone does pretty much anything, if the destination does not exist then it tries to create it. Unfortunately it uses what it considers the S3 equivalent of
mkdir -p. Even worse, it (always!) tries to create the bucket. Document that user needs to setno_check_bucket = true(or set the--s3-no-check-bucketflag) in order to work around this.The underlying issue is that S3 will return a 409 (Conflict) when trying to recreate an already-existing bucket. We return 405 (Method Not Allowed), and rclone gives up.
The text was updated successfully, but these errors were encountered: