Signer.Sign creates invalid signatures when host includes a port

[brendanjerwin]

Version of AWS SDK for Go?

1.10.48, but was in previous versions as well.

Version of Go (go version)?

go version go1.9 darwin/amd64

What issue did you see?

http.Requests signed with Signer.Sign were not being accepted. I happened to be testing through a reverse proxy and was using a non-standard port to access the Elasticsearch service. Eventually I found that the root cause was that my host included a port and that the Signer was signing that whole value for the host header. However, the http host header value doesn't allow a port, so the signature didn't match.

Steps to reproduce

Get some code making a request to some service, ES is pretty simple to do with just an http.Request. Sign the request with Signer.Sign and ensure it works.
Then set up a reverse proxy in front of the resource. mitmproxy works well for this. Use a non-standard port. At this point, in order to connect, you'll need something like http://127.0.0.1:9876/blarg whereas before introducing the reverse proxy, you'd have http://real-domain-name/blarg.
At this point, the requests will start to fail with a 403 response.
Before signing the http.Request, set the req.Host to req.URL.Hostname(), try again. The requests should start working again.

The Signer will, when getting the value to include for the host header value, use req.Host first, and if it is blank, req.URL.Host second. Both of those will have the port included. Setting the value explicitly to the req.URL.Hostname() excludes the port and allows the signature to be correct again.

My Diagnosis (take it with a grain of salt)

Here is the area that seems to be the problem: https://github.com/aws/aws-sdk-go/blob/master/aws/signer/v4/v4.go#L606

I would think that, at least in the second case, the thing should take ctx.Request.URL.Hostname(). The first case should account for it too somehow, but I'm not sure what the best strategy is...

[xibz]

hello @brendanjerwin, thank you for reaching out to us. So, from the sound of things, it sounds like the main issue here is that when signing, we are including the port, which is incorrect. Does that sound right?

I'll need to do a little bit of research to see why we choose to go with req.Host and req.URL.Host before we can make the change. Also we are always happy to take a look at PRs :)

[xibz]

@brendanjerwin - I have set up a reverse proxy running this command
mitmdump -R https://s3-us-west-2.amazonaws.com -p 9091

Then I try to hit it using this code

  svc := s3.New(session.New(&aws.Config{                                                                                   
    Region:     aws.String("us-west-2"),                                                                                   
    Endpoint:   aws.String("localhost:9091"),                                                                              
    DisableSSL: aws.Bool(true),                                                                                            
    LogLevel:   aws.LogLevel(aws.LogDebugWithHTTPBody),                                                                    
  }))                                                                                                                      
                                                                                                                           
  resp, err := svc.ListBuckets(&s3.ListBucketsInput{})                                                                     

And looks like I am still getting a signature is invalid with that change of Hostname(). I want to make sure that I am setting this up the same as you. Please let me know.