AWS PHP SDK region parameter

[iby]

I'm trying to marry a local Riak CS instance with the AWS SDK and came across this aws/aws-sdk-php#409. I asked the guys to explain why things might not be working, while I think you might also have an explanation? Would you elaborate on this?

[iby]

In a nutshell, it seems AWS are slowly shifting from {REGION}.s3.amazonaws.com to s3-{REGION}.amazonaws.com. Specifying a new region makes AWS SDK incompatible with Riak CS. If later this rolls across all regions the SDK will stop working completely.

The adequate solution would be to rewrite the url from any valid S3 region-specific to whatever is specified in the Riak CS config (cs_root_host value).

I think if Riak CS is build to be S3 compatible, this issue should be mentioned in the docs somewhere. If there's no better way of doing it, then please close this.

[kuenishi]

Currently Riak CS has no concept of regions like S3 (and it's too big to support). So my workaround suggestion is to configure aws-sdk-php as pointing some region named blah and set cs_root_host as s3-blah.yours.example.com. Then aws-sdk-php hits bucket.s3-blah.yours.example.com and handles very well.

In my understanding, Amazon is doing that shift by CNAME, not by 30x forwarding. So the workaround above will be compatible with S3.

$ dig ap-northeast-1.s3.amazonaws.com

; <<>> DiG 9.8.3-P1 <<>> ap-northeast-1.s3.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57505
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;ap-northeast-1.s3.amazonaws.com. IN    A

;; ANSWER SECTION:
ap-northeast-1.s3.amazonaws.com. 2400 IN CNAME  s3-1-w.amazonaws.com.
s3-1-w.amazonaws.com.   19  IN  A   54.231.1.129

;; Query time: 38 msec
;; SERVER: 192.168.100.1#53(192.168.100.1)
;; WHEN: Wed Nov 26 03:37:19 2014
;; MSG SIZE  rcvd: 86
$ dig foobucket.ap-northeast-1.s3.amazonaws.com

; <<>> DiG 9.8.3-P1 <<>> foobucket.ap-northeast-1.s3.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13851
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;foobucket.ap-northeast-1.s3.amazonaws.com. IN A

;; ANSWER SECTION:
foobucket.ap-northeast-1.s3.amazonaws.com. 7519 IN CNAME s3-directional-w.amazonaws.com.
s3-directional-w.amazonaws.com. 1363 IN CNAME   s3-directional-w.a-geo.amazonaws.com.
s3-directional-w.a-geo.amazonaws.com. 222 IN CNAME s3-2-w.amazonaws.com.
s3-2-w.amazonaws.com.   7   IN  A   54.231.244.5

;; Query time: 34 msec
;; SERVER: 192.168.100.1#53(192.168.100.1)
;; WHEN: Wed Nov 26 03:41:10 2014
;; MSG SIZE  rcvd: 164
$ dig kuenishi.s3.amazonaws.com

; <<>> DiG 9.8.3-P1 <<>> kuenishi.s3.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35404
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;kuenishi.s3.amazonaws.com. IN  A

;; ANSWER SECTION:
kuenishi.s3.amazonaws.com. 7519 IN  CNAME   s3-directional-w.amazonaws.com.
s3-directional-w.amazonaws.com. 1494 IN CNAME   s3-directional-w.a-geo.amazonaws.com.
s3-directional-w.a-geo.amazonaws.com. 298 IN CNAME s3-2-w.amazonaws.com.
s3-2-w.amazonaws.com.   35  IN  A   54.231.244.5

;; Query time: 36 msec
;; SERVER: 192.168.100.1#53(192.168.100.1)
;; WHEN: Wed Nov 26 03:41:36 2014
;; MSG SIZE  rcvd: 148

After a little playing with dig command, it revealed that the domain name change is more complex than I thought. Do you know any official document from Amazon about these domain name shift?

[iby]

Its discussed in the linked issue, from what I understand this is not set in stone and only happens to the new regions, like Frankfurt. This is more of a DNS-related issue than the backend, just wanted to bring this to your attention, as soon this probably will start coming up more.