How to generate AWS RDS auth token

[xanather]

Describe the issue

Previously there was some documentation at https://github.com/awslabs/aws-sdk-rust/blob/060d3c5a22a0b559cc459cbdbbe80b28685630c5/sdk/aws-sig-auth/src/lib.rs that defined how to generate some RDS token credentials.

Now that the code-base has been refactored I'm not sure how to do it?

There should be a helper function somewhere like in other AWS SDK's that provide easy access to this.

Links

https://github.com/awslabs/aws-sdk-rust/blob/060d3c5a22a0b559cc459cbdbbe80b28685630c5/sdk/aws-sig-auth/src/lib.rs
https://docs.aws.amazon.com/cli/latest/reference/rds/generate-db-auth-token.html

(no longer relevant).

[xanather]

Related:

#792
#147

[ysaito1001]

Hi @xanather,

Thank you for bringing this to our attention. The example snippet for generate_rds_iam_token has been removed due to the aws-sig-auth crate being deprecated.

I have put together a gist that ports the example in question to the latest release 0.57.x (at the time of writing). I agree that there should be a helper function for easy access because the above gist exposes types like RuntimeComponentsBuilder or ConfigBag, which are normally hidden when you use the SDK to interact an AWS service, so please keep in mind that the gist is just a temporary workaround.

[xanather]

Thanks for the gist @ysaito1001, should help others that were using Rust SDK directly for generating RDS session passwords.
I have decided to invoke the AWS CLI generate-db-auth-token directly within my app and get the password from stdout to avoid depending on the lower-level parts of the SDK again. A top level helper function definitely should be added as part of API stabilization.

[lcmgh]

I cannot access the gist provided by @ysaito1001 due to corp. firewall/proxy issues.

However I came along with this which I have not tested yet (but it compiles :))

   pub fn generate_rds_iam_token_sdk(
        db_hostname: &str,
        region: Region,
        port: u16,
        db_username: &str,
        credentials: &Credentials,
    ) -> Result<String, SigningError> {
        let expiration = credentials.expiry();
        let region = region.to_string();
        let identity = Identity::new(credentials.clone(), expiration);
        let signing_settings = SigningSettings::default();
        let signing_params = aws_sigv4::sign::v4::SigningParams::builder()
            .identity(&identity)
            .region(&region)
            .name("rds-db")
            .time(SystemTime::now())
            .settings(signing_settings)
            .build()
            .unwrap();

        // Convert the HTTP request into a signable request
        let url = format!(
            "http://{db_hostname}:{port}/?Action=connect&DBUser={db_user}",
            db_hostname = db_hostname,
            port = port,
            db_user = db_username
        );
        let signable_request = SignableRequest::new(
            "GET",
            url.clone(),
            std::iter::empty(),
            SignableBody::Bytes(&[]),
        )
        .expect("signable request");

        let (signing_instructions, _signature) = sign(
            signable_request,
            &aws_sigv4::http_request::SigningParams::V4(signing_params),
        )?
        .into_parts();
        let mut my_req = Request::builder().uri(url).body(()).unwrap();

        signing_instructions.apply_to_request(&mut my_req);
        let mut uri = my_req.uri().to_string();

        assert!(uri.starts_with("http://"));
        let uri = uri.split_off("http://".len());

        Ok(uri)
    }

[jwarlander]

@lcmgh, it was a great starting point! ✨ But didn't get me all the way.. I ended up with the following (tested) version, that applies signing instructions manually to a Url as I'm using http = "1.0.0".

For ease of use, the function below loads AWS config & extracts the credentials, but that could of course be passed in as an argument instead.

use std::time::{Duration, SystemTime};

use aws_config::BehaviorVersion;
use aws_credential_types::provider::ProvideCredentials;
use aws_sigv4::{
    http_request::{sign, SignableBody, SignableRequest, SigningSettings},
    sign::v4,
};

async fn generate_rds_iam_token(
    db_hostname: &str,
    port: u16,
    db_username: &str,
) -> Result<String, Box<dyn Error>> {
    let config = aws_config::load_defaults(BehaviorVersion::v2023_11_09()).await;

    let credentials = config
        .credentials_provider()
        .expect("no credentials provider found")
        .provide_credentials()
        .await
        .expect("unable to load credentials");
    let identity = credentials.into();
    let region = config.region().unwrap().to_string();

    let mut signing_settings = SigningSettings::default();
    signing_settings.expires_in = Some(Duration::from_secs(900));
    signing_settings.signature_location = aws_sigv4::http_request::SignatureLocation::QueryParams;

    let signing_params = v4::SigningParams::builder()
        .identity(&identity)
        .region(&region)
        .name("rds-db")
        .time(SystemTime::now())
        .settings(signing_settings)
        .build()?;

    let url = format!(
        "https://{db_hostname}:{port}/?Action=connect&DBUser={db_user}",
        db_hostname = db_hostname,
        port = port,
        db_user = db_username
    );

    let signable_request =
        SignableRequest::new("GET", &url, std::iter::empty(), SignableBody::Bytes(&[]))
            .expect("signable request");

    let (signing_instructions, _signature) = sign(signable_request, &signing_params.into())?.into_parts();

    let mut url = url::Url::parse(&url).unwrap();
    for (name, value) in signing_instructions.params() {
        url.query_pairs_mut().append_pair(name, &value);
    }

    let response = url.to_string().split_off("https://".len());

    Ok(response)
}

Dependencies involved:

aws-config = "1.0.1"
aws-credential-types = "1.0.1"
aws-sigv4 = "1.0.1"
url = "2.5.0"

[jwarlander]

@ysaito1001, doesn't it make a lot of sense to include something like this as an RDS utility function, given that it's how eg. the Java & Python SDKs do it?

It's a bit fiddly & far from obvious how to work out the IAM authentication, while also being one of the ideal ways that I guess one "should" want to use RDS..

If I can, I definitely want to avoid generating yet another password that I need to fetch for each service that needs it, risking potential exposure etc. Relying on IAM roles for my workloads is so much smoother.

[ysaito1001]

@jwarlander, you have a good point. The reason generate_db_auth_token is not part of the aws-sdk-rds crate is that the function is not a Smithy-modeled operation, but a rather library function that needs to be hand-written (as opposed to code-generated by smithy-rs). For such a feature, you will see the high-level-library label within this repository. Essentially, if labeled as high-level-library, it means that we need to figure out how to house those high-level libraries, separately from a code-generated Rust SDK, and that it is a cross-SDK effort to provide those libraries in a consistent manner across different languages.

[jwarlander]

@ysaito1001, it sounds like some thinking around this is happening, that's good to hear!

If one wants to find an interim place for collecting some of these higher level utilities, would that have to be a non-AWS crate for now? I see that eg. #980 is pretty close to the RDS token issue, and I'm sure there are others with workarounds posted in comments.

[lcmgh]

@lcmgh, it was a great starting point! ✨ But didn't get me all the way.. I ended up with the following (tested) version, that applies signing instructions manually to a Url as I'm using http = "1.0.0".

For ease of use, the function below loads AWS config & extracts the credentials, but that could of course be passed in as an argument instead.

use std::time::{Duration, SystemTime};

use aws_config::BehaviorVersion;
use aws_credential_types::provider::ProvideCredentials;
use aws_sigv4::{
    http_request::{sign, SignableBody, SignableRequest, SigningSettings},
    sign::v4,
};

async fn generate_rds_iam_token(
    db_hostname: &str,
    port: u16,
    db_username: &str,
) -> Result<String, Box<dyn Error>> {
    let config = aws_config::load_defaults(BehaviorVersion::v2023_11_09()).await;

    let credentials = config
        .credentials_provider()
        .expect("no credentials provider found")
        .provide_credentials()
        .await
        .expect("unable to load credentials");
    let identity = credentials.into();
    let region = config.region().unwrap().to_string();

    let mut signing_settings = SigningSettings::default();
    signing_settings.expires_in = Some(Duration::from_secs(900));
    signing_settings.signature_location = aws_sigv4::http_request::SignatureLocation::QueryParams;

    let signing_params = v4::SigningParams::builder()
        .identity(&identity)
        .region(&region)
        .name("rds-db")
        .time(SystemTime::now())
        .settings(signing_settings)
        .build()?;

    let url = format!(
        "https://{db_hostname}:{port}/?Action=connect&DBUser={db_user}",
        db_hostname = db_hostname,
        port = port,
        db_user = db_username
    );

    let signable_request =
        SignableRequest::new("GET", &url, std::iter::empty(), SignableBody::Bytes(&[]))
            .expect("signable request");

    let (signing_instructions, _signature) = sign(signable_request, &signing_params.into())?.into_parts();

    let mut url = url::Url::parse(&url).unwrap();
    for (name, value) in signing_instructions.params() {
        url.query_pairs_mut().append_pair(name, &value);
    }

    let response = url.to_string().split_off("https://".len());

    Ok(response)
}

Dependencies involved:

aws-config = "1.0.1"
aws-credential-types = "1.0.1"
aws-sigv4 = "1.0.1"
url = "2.5.0"

Hi @jwarlander thanks. Did you further encode the password as URL before passing it to the db client?

let db_uri = format!("postgres://127.0.0.1:{local_port}/{db_name}");
let mut uri = Url::parse(&db_uri).unwrap();
uri.set_username(username.as_str()).unwrap();
uri.set_password(Some(password.as_str())).unwrap();
let conn_url = uri.as_str();

Without doing so I am getting "invalid port" errors from sqlx. When decoding it that way my auth fails somehow. I am currently not sure about the root cause of the problem.

[jwarlander]

Here's what I'm doing, @lcmgh -- I think I had issues, too, with encoding it in a URL, so I side-stepped the issue:

    let rds_token = generate_rds_iam_token(db_hostname, db_port, db_username).await?;

    let options = PgConnectOptions::new()
        .host(db_hostname)
        .port(db_port)
        .username(db_username)
        .password(&rds_token)
        .database(db_name);

    let pool = PgPoolOptions::new()
        .max_connections(5)
        .connect_with(options)
        .await?;

[lcmgh]

Issue was on my IAM policies side. I can also confirm it works :) Thanks!

[hef]

I had to urlencode the token before calling set_password() on the url object to get url connect strings to work. It looks something like this:

let db_url = "postgresql://my_user@example.com/dbname?foo=bar";
let mut url = url::Url::parse(db_url).unwrap();
let db_hostname = url.host().unwrap();
let db_port = url.port().unwrap();
let db_username = url.username();
let token = generate_rds_iam_token(&db_hostname, db_port, db_username).await.unwrap();

// This was the step I was missing
let encoded_token = urlencoding::encode(&token);
url.set_password(Some(&encoded_token));


let db_url = url.to_string();

The set_password() call looks like it does some escaping, but reading the password out didn't match what I put into it unless I urlencoded it first.

[Velfi]

Due to an internal push, I've created a util to do this: smithy-lang/smithy-rs#3867