The difference between the request time and the current time is too large

[JaderCM]

Describe the bug

I'm getting the error "The difference between the request time and the current time is too large" when trying to upload a big file using AWSSDK.S3 version 3.7.9.21.

If I use AWS Tools for Windows on PowerShell version 3.15.172 it works, but I want to use C# code to upload this file.

That is my code:

using var s3 = new AmazonS3Client(Settings.Default.AWSAccessKey, Settings.Default.AWSSecretKey, Amazon.RegionEndpoint.GetBySystemName(Settings.Default.AWSRegionEndpoint));
s3.PutObject(new PutObjectRequest
{
    BucketName = Settings.Default.AWSS3BucketName,
    FilePath = backupsFileName
});

Expected Behavior

Upload a large file using AWSSDK C# code without any problem.

Current Behavior

The error "The difference between the request time and the current time is too large" is returned.

Reproduction Steps

• Creates a big file (my example is 7GB)
• Using the C# code at description, try to upload the file

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.S3 3.7.9.21

Targeted .NET Platform

.NET Framework 4.8

Operating System and version

Windows Server 2016 (AWS EC2 instance)

[ashishdhingra]

Hi @JaderCM,

Good afternoon.

The article How to fix S3 (RequestTimeTooSkewed) error documents the possible reasoning behind this issue and possible way to fix it.

Also, the recommended way to upload large files is to use multipart uploads. Refer Uploading and copying objects using multipart upload for more details. The page Uploading an object using multipart upload lists the examples using the same using high level (using TransferUtility) or low level API(s) using .NET.

Just FYI, the AWS PowerShell Tools for .NET uses TransferUtility (and hence multipart upload) behind the scenes.

Thanks,
Ashish

[JaderCM]

@ashishdhingra that solutions still workarounds in my opinion.

Do you have some better solution?

[ashishdhingra]

@ashishdhingra that solutions still workarounds in my opinion.

Do you have some better solution?

@JaderCM As you mentioned that the upload works fine using AWS Tools got PowerShell, the recommended way to upload large files is to use TransferUtility since it increases thorough put and recalculates signature for each uploaded part.

[JaderCM]

@JaderCM As you mentioned that the upload works fine using AWS Tools got PowerShell, the recommended way to upload large files is to use TransferUtility since it increases thorough put and recalculated signature for each uploaded part.

@ashishdhingra do you know if it's possible to this signature assertation be refactored to some more smart solution?

At moment, all data trafic sent was lost with a generic error message for a common scenario (sending large file to S3).

[ashishdhingra]

@ashishdhingra do you know if it's possible to this signature assertation be refactored to some more smart solution?

Please elaborate. Nothing I'm aware of in SDK. The error is thrown by the S3 service.

At moment, all data trafic sent was lost with a generic message error for a common scenario (sending large file to S3).

That's the reason it is recommended to use multipart uploads for large files as per documentation.

[JaderCM]

Please elaborate. Nothing I'm aware of in SDK. The error is thrown by the S3 service.

@ashishdhingra my suggestion is could you add some extra info in that error message when the realNow (AWS UTC time) is equivalent with the current machine UTC time? Like: "Try to use an alternative way to upload your file by mutlipart."

[ashishdhingra]

Please elaborate. Nothing I'm aware of in SDK. The error is thrown by the S3 service.

@ashishdhingra my suggestion is could you add some extra info in that error message when the realNow (AWS UTC time) is equivalent with the current machine UTC time? Like: "Try to use an alternative way to upload your file by mutlipart."

@JaderCM The error message is not controlled by the SDK, rather returned by the S3 service. And the message is independent of the mechanism used by caller, single file upload or multi-part upload.

[JaderCM]

@JaderCM The error message is not controlled by the SDK, rather returned by the S3 service. And the message is independent of the mechanism used by caller, single file upload or multi-part upload.

@ashishdhingra my suggestion is to wrap the PutObject function to handle the exception and

1. If is this specific exception (code RequestTimeTooSkewed) and
2. The realNow (AWS UTC time) is equivalent with the current machine UTC time

Then get the original exception and adds an additional message like "Try to use an alternative way to upload your file by mutlipart."

[ashishdhingra]

@ashishdhingra my suggestion is to wrap the PutObject function to handle the exception and

1. If is this specific exception and

2. The realNow (AWS UTC time) is equivalent with the current machine UTC time

Then get the original exception and adds an additional message like "Try to use an alternative way to upload your file by mutlipart."

@JaderCM Many operations in the SDK are autogenerated by service models and it's not feasible to add specific exception handlers. PutObject is a service client operation and such client side exceptions should be handled by the calling code. Simple upload (using PutObject) and multipart upload are separate service level operations without any dependency on each other. Hence, client application should determine what operation is best suited for production load. As mentioned earlier, you may use high level TransferUtility to automatically handle this behavior based on configuration parameters.

[JaderCM]

@JaderCM Many operations in the SDK are autogenerated by service models and it's not feasible to add specific exception handlers. PutObject is a service client operation and such client side exceptions should be handled by the calling code. Simple upload (using PutObject) and multipart upload are separate service level operations without any dependency on each other. Hence, client application should determine what operation is best suited for production load. As mentioned earlier, you may use high level TransferUtility to automatically handle this behavior based on configuration parameters.

@ashishdhingra thanks to explain this important part about SDK code is autogenerated by service models. In that case, I understand your point to avoid specific exception handlers.

In the end, just this discussion can be enough for future references what is the solution for this case: Use TransferUtility.

[ashishdhingra]

@ashishdhingra thanks to explain this important part about SDK code is autogenerated by service models. In that case, I understand your point to avoid specific exception handlers.

In the end, just this discussion can be enough for future references what is the solution for this case: Use TransferUtility.

@JaderCM Use TransferUtility or low level API as mentioned at Uploading an object using multipart upload for large file uploads. Thanks.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.