Connect to RDS using rdsutils.BuildAuthToken not working #1248
Comments
|
Hello @jhwang09, thank you for reaching out to us. When the service team tested our implementation, they said it worked. I would ask on their forums to see if they can spot anything that may not look right. In addition, questions like this may be best suited for stackoverflow or gitter. However, if there is an issue with the implementation, please reopen this ticket. |
|
@jhwang09, I am looking at the docs and I see the confusion. Let me try to test this on my end and see if the example is incorrect. |
|
@jhwang09, I have a feeling your Why is there |
|
I have tried the original solution as proposed by the doc but it wasn't working that's why I start trying different variation. the original doc said that the dbEndpoint should include [scheme]://[host][:port], and the same variable is used for both rdsutils.BuildAuthToken and for the DSN: However, I am not sure if the one in DSN actually requires scheme in there the other thing I find odd is the password used, in this case it's the authToken coming back from rdsutils and it is of this format does this look correct? this looks more like a host with bunch of query params, not sure if this is supposed to be the authToken/password used in DSN |
|
jhwang09 - What is the error you are seeing when you use that? I ran into no issues using the example. |
|
see below is the panic |
This is what I currently have, following the doc code snippet |
|
what will be helpful is if you can show me your final dsn, it will be a huge help for me to debug thanks |
|
@jhwang09 - Please make sure your db endpoint is correct. It should match up with what is in the console. I believe your DSN construction is correct, but your endpoint may be wrong since you are manually constructing it. Can you please verify that it is correct? It should match in the AWS console. |
|
Could you please let me know what the scheme should be? for when we doing the BuildAuthToken? is it https? http? something else? I tried to use http/https when build authToken, and in the actual TCP dial to db, I remove the scheme, and this is what I got: this is what the actual DSN looks like: does this looks correct? |
|
to double confirm, my db endpoint is correct, it is consist of [hostname]:[port] I am suspecting if I should open my RDS up a bit in its security group? it is only allowing |
|
on the console, it has two places it says endpoint, one is with port one is without port... which one is correct? |
|
I would use the port one, because that's what I tested with. And yes, the DSN looks correct. Can I have the errors of when you tried to use the scheme http or https? Also, may want to try static credentials to see if that has anything to do with it. |
|
Please refer to my previous post, If I do http or https, below is the panic: I tried the static credentials and it works fine. I am almost to a point to give up using the IAMAuth because it's too much pain... |
|
Okay, that is pretty important information. Let me try replicating this with an IAM role. If it succeeds, then it may be something in your middleware. Can you write a simple Go program that eliminates jenkins and all your dependencies and only tests the SDK? Or has that been what you've been doing? |
|
@jhwang09 - I just tried this with STS credentials, and it worked. Is this failing on opening a connection with So, yea, I cannot seem to replicate this. What |
|
How do you test STS credentials w/o being on an EC2 host? I tried to assign the role to an user, but it isn't allowing user to assume role for some reason. This is failing on when we do db.Ping(). My application basically can't start because of unable to do ping on startup |
|
@jhwang09 - you shouldnt need to be on an Amazon EC2 host. What is the error when trying to assume the role? Have you have given permissions the correct permissions to Amazon RDS? What does that look like? |
|
@jhwang09 - I have a PR that contains an example of using the |
|
Same problem. The documentation truly is a mess. |
|
Same :( |
|
Hello @sujunzhu, what issues are you running into? Can you provide a code sample? |
|
Hi,
1: passwd, err := rdsutils.BuildAuthToken(fmt.Sprintf("%s:%d", host, 3306), region, cfg.User, creds)2: func RegisterRDSMysqlCerts(c *http.Client) error {
resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
if err != nil {
return errs.Wrap(err)
}
defer fileutil.CloseLoggingAnyError(resp.Body)
pem, err := ioutil.ReadAll(resp.Body)
if err != nil {
return errs.Wrap(err)
}
rootCertPool := x509.NewCertPool()
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
return errs.New("couldn't append certs from pem")
}
err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
if err != nil {
return errs.Wrap(err)
}
return nil
}A full working example (you'll need to fill in the appropriate information for your user/credentials/etc): package main
import (
"crypto/tls"
"crypto/x509"
"database/sql"
"fmt"
"io/ioutil"
"net/http"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/service/rds/rdsutils"
"github.com/go-sql-driver/mysql"
"github.com/richardwilkes/errs"
"jaxf-github.fanatics.corp/forge/furnace/fileutil"
)
func RegisterRDSMysqlCerts(c *http.Client) error {
resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
if err != nil {
return errs.Wrap(err)
}
defer fileutil.CloseLoggingAnyError(resp.Body)
pem, err := ioutil.ReadAll(resp.Body)
if err != nil {
return errs.Wrap(err)
}
rootCertPool := x509.NewCertPool()
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
return errs.New("couldn't append certs from pem")
}
err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
if err != nil {
return errs.Wrap(err)
}
return nil
}
func main() {
// FILL THESE OUT:
host := ""
user := ""
dbName := ""
creds := credentials.NewSharedCredentials("", "")
region := "us-east-1"
host = fmt.Sprintf("%s:%d", host, 3306)
cfg := &mysql.Config{
User: user,
Addr: host,
Net: "tcp",
Params: map[string]string{
"tls": "rds",
},
DBName: dbName,
}
cfg.AllowCleartextPasswords = true
var err error
cfg.Passwd, err = rdsutils.BuildAuthToken(host, region, cfg.User, creds)
if err != nil {
panic(err)
}
err = RegisterRDSMysqlCerts(http.DefaultClient)
if err != nil {
panic(err)
}
db, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
fmt.Println("ok")
} |
|
One additional tidbit - if you fail to provide a port in the connection string, you'll encounter this: go-sql-driver/mysql#717 |
Please fill out the sections below to help us address your issue.
Version of AWS SDK for Go?
v1.8.19-6-g7b500fb
Version of Go (
go version)?go 1.8
What issue did you see?
following the doc but unable to make successful DB connection using IAM role from EC2
Also, this is particularly hard to debug because of not actually able to run code as EC2 on dev to test
Steps to reproduce
I follow the below step in
https://docs.aws.amazon.com/sdk-for-go/api/service/rds/rdsutils/#BuildAuthTokenand use the exact same code snippet but I believe there's some issue with the instruction:I have tried the above instruction but it was throwing signing errors
below is my latest code snippet, I have tried various different fortmat and this is the one i last end up with:
for this one i got:
StdError: invalid DSN: did you forget to escape a param value?the token returned is actually in the following format:
I am not sure if this is the right string that should replace the DSN/DNS password field as indicated in the doc
Can anyone please help me figure out what the actual and correct DSN or DNS will look like? some example will be very helpful as I can just follow the same structure
Thanks
The text was updated successfully, but these errors were encountered: