Can't use multipart copy on files with certain characters #933
Comments
|
What character encoding was used to create the path string? The x-amz-copy-source header uses |
jeskew
added
the
response-requested
|
I'm not sure about the encoding, currently I have no control over the name of the files |
|
Can you check if |
|
@lonormaly Were you able to test out the affected environment using the code in my previous comment? |
|
I'm going to go ahead and close this issue, as it appears to be related to the system running PHP and not to the SDK. Please feel free to reopen if you have any questions or concerns. |
|
Hey, sorry for not responding. Unfortunately this issue is being reproduced time and time again. File name is Beyoncé.jpg Other files the function works perfectly... Here is agist of our function and exception logs: https://gist.github.com/lonormaly/60b5fb5a71f8e27194e142f810941b82 Thank you! |
|
This still looks like an environment-specific error. Were you able to check your system as described above? Does running |
|
true |
|
What output do you get from running |
|
array(59) { |
|
I'm still unable to reproduce. Copying a file with the exact name provided works fine for me. Is there a particular version of PHP you're seeing this error on? |
|
Nope. Did you use the function in the gist to copy the file? On Friday, 22 April 2016, Jonathan Eskew notifications@github.com wrote:
Thanks |
|
Sorry, what I meant to ask was: which version of PHP are you using on the system where the error is appearing? I am unable to reproduce on PHP 5.6.20 or PHP 7.0.5. |
|
PHP 5.5.9-1ubuntu4.14 (cli) (built: Oct 28 2015 01:34:46) |
|
I'm unable to reproduce on PHP 5.5, either. The root cause of the issue is somewhere in how PHP is configured on your system, which is causing the character encoding of the string provided to not be understood by |
|
We're not doing anything special in the PHP installation. I'm pretty sure it has to do with a. the way we use the API or b. a bug with the SDK. Really appreciate the efforts! |
|
Ah, I see that you're directly using the multipart copier. You will need to URL encode the source key. This is something that The multipart copier and uploader take the source as a string and rely on the user (or a higher level of abstraction in the SDK) to take care of encoding. Altering this interface would be a breaking change. |
|
It will be really helpful if you could provide the exact code I need to On Sat, Apr 23, 2016 at 6:18 PM, Jonathan Eskew notifications@github.com
Thanks |
|
The portion of the |
|
Thanks for that, wrapping the $tempSource with rawurlencode returns error from the MultipartCopy function:
We really need the Multipart capabilities, does the ordinary copy implements the multipart issue as well? We need to support really big files (Up to 5TB) |
|
The source needs to be of the form
|
|
We tried to wrap only the filename with rawurlencode and we tried to wrap the key part as you suggested and we experienced no success...
Stack Trace:
|
|
|
|
But we can't use multipart copy with the encoding solution you suggested. Using copy will demand a major change in the flow and it's a sensitive part on our code, also the copy function doesn't support resume from same place. |
|
MultipartCopy is meant to be a low-level abstraction over the different requests that make up a multipart copy. It assumes that the copy source passed to its constructor is the one you mean to send to S3. If the MultipartCopy constructor suddenly started URL encoding portions of the source path, then any code passing in a properly encoded path would break. For example, the string 'Beyonce%CC%81IMG_18022016_111346.png' would be encoded as 'Beyonce%25CC%2581IMG_18022016_111346.png,' as the percent sign is a character that needs to be URL encoded. If you're getting the path in the form of list($bucket, $key) = explode('/', $path, 2);
$path = $bucket . '/' . rawurlencode($key);If the path also contains a version ID, you would also need to separate the query string from the key.
try {
$response = $s3->copy($sourceBucket, $sourceKey, $destinationBucket, $destinationKey, $acl);
} catch (\Aws\S3\Exception\S3Exception $e) {
// This exception is thrown if a single-part copy fails
} catch (\Aws\Exception\MultipartUploadException $e) {
// This exception is thrown if a multipart copy fails
$response = $s3->copy($sourceBucket, $sourceKey, $destinationBucket, $destinationKey, $acl, [
'state' => $e->getState(),
]);
} |
jeskew
removed
the
response-requested
Hello,
The MultipartCopy function doesn't seem to work on file that contains the following character: é
For example: BeyoncéIMG_18022016_111346.png
Same function run without the special characters works fine.
The response I get is:
2016-03-16 12:55:32 Info: initiateMultipartCopy::ArtworkId 1792 Multipart copy started #7
2016-03-16 12:55:33 Info: initiateMultipartCopy:: Uploading chunk {1}
2016-03-16 12:55:33 Info: initiateMultipartCopy::ArtworkId 1792 Multipart exception An exception occurred while uploading parts to a multipart upload. The following parts had errors:
Thanks,
Shai
The text was updated successfully, but these errors were encountered: