File not found if S3 key contains special characters

[khacminh]

Issue:

S3 key may contain special characters as mention in this document. In this case, downloading S3 object throws 404 error.

Solution:

The S3 key need to be URL encoded before download.

[ilkinulas]

Hi @khacminh, thank you for reporting this.

Could you please give an example (and valid) s3 prefix where download s3 object request throws 404 error?

[khacminh]

Hi @ilkinulas

The source bucket has the following objects:

test-dir/object-2021-04-04+1.txt
test-dir/object-2021-04-04:2.txt

Then I would like to copy them to the new bucket with the following command

s5cmd cp srcBucket/* dstBucket/

---

According to AWS document, the following character must be handled by URL encoding or referenced as HEX:

• Ampersand ("&")
• Dollar ("$")
• 'At' symbol ("@")
• Equals ("=")
• Semicolon (";")
• Colon (":")
• Plus ("+")
• Space
• Comma (",")
• Question mark ("?")

Encoding the url component fix this issue. However, it breaks another test cases using wildcard.