Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confusion with Date format and V4 signing in s3tests_boto3 test_object_create_date_and_amz_date #346

Open
jamshid opened this issue May 6, 2020 · 1 comment

Comments

@jamshid
Copy link

jamshid commented May 6, 2020

Hi, I'm very confused by the test
s3tests_boto3.functional.test_headers:test_object_create_date_and_amz_date
https://github.com/ceph/s3-tests/blob/master/s3tests_boto3/functional/test_headers.py#L344

  1. Test seems to be attempting to send both the Date and X-Amz-Date header, but only the Date header is actually getting sent. These are the versions from bootstrap, in case that matters:

virtualenv/lib/python2.7/site-packages/boto-2.49.0.dist-info
virtualenv/lib/python2.7/site-packages/boto3-1.13.2.dist-info
virtualenv/lib/python2.7/site-packages/botocore-1.16.2.dist-info

  1. It fails against AWS S3 because the AWS signs with the Date header as-is. I thought V4 sigs require converting the Date value to ISO 8601 (without dashes or colons). The test fails with this 403 from AWS:
The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAJV3VAEW6OFKXWMLQ</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
Wed, 06 May 2020 15:42:30 -0000
20200506/us-east-1/s3/aws4_request

The test computes the StringToSign with the ISO8601 formatting of Date. Note the line after "AWS4-HMAC-SHA256".

botocore.auth: DEBUG: Calculating signature using v4 auth.
botocore.auth: DEBUG: CanonicalRequest:
PUT
/jam-ceph-a1/foo

date:Wed, 06 May 2020 15:42:30 -0000
host:s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

date;host;x-amz-content-sha256
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
botocore.auth: DEBUG: StringToSign:
AWS4-HMAC-SHA256
20200506T154230Z
20200506/us-east-1/s3/aws4_request
dc8caa951c884d6000554256c19076987fa07d3119c5d06a49834977dcf2179f
  1. I don't know how AWS can be "wrong" here but this from the jets3t library implies the Date header should be converted to ISO 8601 in the signature, right? Maye there was a change or there are differences in regions?
    https://jets3t.s3.amazonaws.com/api/org/jets3t/service/utils/SignatureUtils.html#awsV4ParseAndFormatDate-org.apache.http.client.methods.HttpUriRequest-
@jamshid
Copy link
Author

jamshid commented May 6, 2020

This is from Amazon S3 Glacier but fwiw it says Date must be ISO 8601. So I guess the test should expect failure when it sends Date like an HTTP date, without a x-amz-date header?
https://docs.aws.amazon.com/amazonglacier/latest/dev/api-common-request-headers.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant