feat(postgres): add iam roles anywhere auth profile #3604

sicoyle · 2024-11-15T23:25:08Z

Description

Add IAM Roles Anywhere auth profile to PostgreSQL components.

I saw where we've had fields like awsAccessKey and awsSecretKey fields specific to postgres. However, in the AWS built in profiles for authentication we have accessKey and secretKey. I've moved things to where we have some TODO comments to remove those aws prefixed fields in Dapr 1.17; however, to not break existing users, we cannot just remove these additional fields. Due to this, I had to manually manipulate the metadata bundle even further unfortunately. This is something we should never do. However, again in the case of the auth profile titled "AWS: Access Key ID and Secret Access Key" I am forced to update the manipulations further to make the normally required accessKey and secretKey fields as non-required as we make a transition in the next two releases to the normalized AWS auth profile fields without the aws prefix.

AWS Auth profiles with these changes:


{
          "title": "AWS: Access Key ID and Secret Access Key",
          "description": "Authenticate using an Access Key ID and Secret Access Key included in the metadata",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "awsAccessKey",
              "description": "Deprecated as of Dapr 1.17. Use 'accessKey' instead if using AWS IAM.\nIf both fields are set, then 'accessKey' value will be used.\nAWS access key associated with an IAM account.",
              "type": "string",
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "awsSecretKey",
              "description": "Deprecated as of Dapr 1.17. Use 'secretKey' instead if using AWS IAM.\nIf both fields are set, then 'secretKey' value will be used.\nThe secret key associated with the access key.",
              "sensitive": true,
              "type": "string",
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "awsRegion",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "required": true,
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "accessKey",
              "description": "AWS access key associated with an IAM account",
              "sensitive": true,
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "secretKey",
              "description": "The secret key associated with the access key",
              "sensitive": true,
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "sessionToken",
              "description": "AWS session token to use. A session token is only required if you are using\ntemporary security credentials.",
              "sensitive": true,
              "type": "string",
              "example": "\"TOKEN\""
            }
          ]
        },
        {
          "title": "AWS: Credentials from Environment Variables",
          "description": "Use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the environment",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            }
          ]
        },
        {
          "title": "AWS: IAM Roles Anywhere",
          "description": "Use X.509 certificates to establish trust between AWS and your AWS account and the Dapr cluster using AWS IAM Roles Anywhere.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "trustAnchorArn",
              "description": "ARN of the AWS Trust Anchor in the AWS account granting trust to the Dapr Certificate Authority.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:trust-anchor/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "trustProfileArn",
              "description": "ARN of the AWS IAM Profile in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:profile/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "assumeRoleArn",
              "description": "ARN of the AWS IAM role to assume in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:iam:012345678910:role/exampleIAMRoleName"
            }
          ]
        }

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle · 2024-11-19T01:57:05Z

/ok-to-test

dapr-bot · 2024-11-19T01:57:36Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: 0e4401c

dapr-bot · 2024-11-19T01:57:38Z

Components conformance test

🔗 Link to Action run

Commit ref: 0e4401c

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.sqlserver

dapr-bot · 2024-11-19T01:57:40Z

Components certification test

🔗 Link to Action run

Commit ref: 0e4401c

❌ Some certification tests failed

These tests failed:

bindings.azure.eventhubs
pubsub.gcp.pubsub
state.azure.cosmosdb

famarting · 2024-11-19T13:57:13Z

common/authentication/aws/static.go

+func (a *StaticAuth) getDatabaseToken(ctx context.Context, poolConfig *pgxpool.Config) (string, error) {
+	dbEndpoint := poolConfig.ConnConfig.Host + ":" + strconv.Itoa(int(poolConfig.ConnConfig.Port))
+	switch {
+	case a.accessKey != nil && a.secretKey != nil:


rather than this I would do a simple if statement, much simpler to read

famarting · 2024-11-19T13:59:31Z

so this PR is missing the assume role auth profile, right?

sicoyle · 2024-11-20T16:31:03Z

https://github.com/dapr/components-contrib/pull/3607/files
yes, didn't want to bring in a bunch into this 1 PR, but after I get the kafka PR merged then this will be an easy add here.

Pls give me a sec on this PR bc I do need to make tweaks, but doing the kafka PR first.

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle · 2024-12-03T17:06:21Z

so this PR is missing the assume role auth profile, right?

This PR includes the ability to assume a role in the auth profile

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle · 2024-12-03T17:25:29Z

latest metadata for postgres components aws auth profiles


 {
          "title": "AWS: Access Key ID and Secret Access Key",
          "description": "Authenticate using an Access Key ID and Secret Access Key included in the metadata",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "awsAccessKey",
              "description": "Deprecated as of Dapr 1.17. Use 'accessKey' instead if using AWS IAM.\nIf both fields are set, then 'accessKey' value will be used.\nAWS access key associated with an IAM account.",
              "type": "string",
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "awsSecretKey",
              "description": "Deprecated as of Dapr 1.17. Use 'secretKey' instead if using AWS IAM.\nIf both fields are set, then 'secretKey' value will be used.\nThe secret key associated with the access key.",
              "sensitive": true,
              "type": "string",
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "awsRegion",
              "description": "This maintains backwards compatibility with existing fields. \nIt will be deprecated as of Dapr 1.17. Use 'region' instead.\nThe AWS Region where the AWS service is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "region",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "accessKey",
              "description": "AWS access key associated with an IAM account",
              "sensitive": true,
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "secretKey",
              "description": "The secret key associated with the access key",
              "sensitive": true,
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "sessionToken",
              "description": "AWS session token to use. A session token is only required if you are using\ntemporary security credentials.",
              "sensitive": true,
              "example": ""
            }
          ]
        },
        {
          "title": "AWS: Assume IAM Role",
          "description": "Assume a specific IAM role. Note: This is only supported for Kafka and PostgreSQL.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "region",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "assumeRoleArn",
              "description": "IAM role that has access to AWS resource.\nThis is another option to authenticate with MSK and RDS Aurora aside from the AWS Credentials.",
              "type": "string",
              "example": "\"arn:aws:iam::123456789:role/mskRole\""
            },
            {
              "name": "sessionName",
              "description": "The session name for assuming a role.",
              "type": "string",
              "default": "\"DaprDefaultSession\"",
              "example": "\"MyAppSession\""
            }
          ]
        },
        {
          "title": "AWS: Credentials from Environment Variables",
          "description": "Use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the environment",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            }
          ]
        },
        {
          "title": "AWS: IAM Roles Anywhere",
          "description": "Use X.509 certificates to establish trust between AWS and your AWS account and the Dapr cluster using AWS IAM Roles Anywhere.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "trustAnchorArn",
              "description": "ARN of the AWS Trust Anchor in the AWS account granting trust to the Dapr Certificate Authority.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:trust-anchor/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "trustProfileArn",
              "description": "ARN of the AWS IAM Profile in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:profile/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "assumeRoleArn",
              "description": "ARN of the AWS IAM role to assume in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:iam:012345678910:role/exampleIAMRoleName"
            }
          ]
        }

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle · 2024-12-03T17:37:20Z

/ok-to-test

dapr-bot · 2024-12-03T17:38:13Z

Components conformance test

🔗 Link to Action run

Commit ref: faf743b

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.azure.cosmosdb

dapr-bot · 2024-12-03T17:38:15Z

Components certification test

🔗 Link to Action run

Commit ref: faf743b

❌ Some certification tests failed

These tests failed:

pubsub.aws.snssqs
pubsub.gcp.pubsub
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-12-03T17:38:43Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: faf743b

sicoyle · 2024-12-03T18:36:42Z

this is ready pls. Conformance/certification test failures are unrelated

sicoyle · 2024-12-03T18:40:37Z

/ok-to-test

dapr-bot · 2024-12-03T18:41:00Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: faf743b

dapr-bot · 2024-12-03T18:41:02Z

Components certification test

🔗 Link to Action run

Commit ref: faf743b

❌ Some certification tests failed

These tests failed:

bindings.azure.eventhubs
pubsub.aws.snssqs
pubsub.gcp.pubsub
pubsub.mqtt3
pubsub.rabbitmq
state.azure.cosmosdb
state.memcached
state.sqlserver

dapr-bot · 2024-12-03T18:41:02Z

Components conformance test

🔗 Link to Action run

Commit ref: faf743b

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
bindings.azure.eventhubs
state.azure.cosmosdb

sicoyle added 2 commits November 15, 2024 16:39

feat(iam roles anywhere): add support for postgres components

fa08cbc

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix(metadata): acct for manipulated fields properly

3f52321

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle requested review from a team as code owners November 15, 2024 23:25

sicoyle changed the title ~~Feat iam roles anywhere postgres~~ feat(postgres): add iam roles anywhere auth profile Nov 15, 2024

sicoyle added 4 commits November 15, 2024 17:45

fix: rm duplicate field on regions in the metadata

6c4f42d

Signed-off-by: Samantha Coyle <sam@diagrid.io>

style: make linter happy

5cffcf1

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix: update close funcs

d867910

Signed-off-by: Samantha Coyle <sam@diagrid.io>

Merge branch 'main' into feat-iam-roles-anywhere-postgres

0e4401c

famarting reviewed Nov 19, 2024

View reviewed changes

yaron2 and others added 6 commits November 26, 2024 08:45

Merge branch 'main' into feat-iam-roles-anywhere-postgres

b6c85f4

fix: address merge conflicts

ad47876

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix: address merge conflicts

5056e3b

Signed-off-by: Samantha Coyle <sam@diagrid.io>

Merge branch 'main' into feat-iam-roles-anywhere-postgres

fb1ee5f

fix: update from changes in main

8bc140b

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix: clean up and add assume role capability

a68577f

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix: make sure metadata bundle is good

97493ea

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle added 3 commits December 3, 2024 11:28

style: make linter happy

309efba

Signed-off-by: Samantha Coyle <sam@diagrid.io>

style: update default values

5b32347

Signed-off-by: Samantha Coyle <sam@diagrid.io>

fix: go mod tidy

faf743b

Signed-off-by: Samantha Coyle <sam@diagrid.io>

sicoyle requested a review from famarting December 3, 2024 20:53

famarting approved these changes Dec 3, 2024

View reviewed changes

yaron2 approved these changes Dec 3, 2024

View reviewed changes

yaron2 merged commit 72c92fb into dapr:main Dec 3, 2024
90 checks passed

sicoyle mentioned this pull request Dec 4, 2024

docs(aws): revamp aws docs + iam role anywhere profile dapr/docs#4456

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(postgres): add iam roles anywhere auth profile #3604

feat(postgres): add iam roles anywhere auth profile #3604

sicoyle commented Nov 15, 2024 •

edited

Loading

sicoyle commented Nov 19, 2024

dapr-bot commented Nov 19, 2024

dapr-bot commented Nov 19, 2024 •

edited

Loading

dapr-bot commented Nov 19, 2024 •

edited

Loading

famarting Nov 19, 2024

famarting commented Nov 19, 2024

sicoyle commented Nov 20, 2024

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

dapr-bot commented Dec 3, 2024

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

feat(postgres): add iam roles anywhere auth profile #3604

feat(postgres): add iam roles anywhere auth profile #3604

Conversation

sicoyle commented Nov 15, 2024 • edited Loading

Description

Issue reference

Checklist

sicoyle commented Nov 19, 2024

dapr-bot commented Nov 19, 2024

Complete Build Matrix

dapr-bot commented Nov 19, 2024 • edited Loading

Components conformance test

❌ Some conformance tests failed

dapr-bot commented Nov 19, 2024 • edited Loading

Components certification test

❌ Some certification tests failed

famarting Nov 19, 2024

Choose a reason for hiding this comment

famarting commented Nov 19, 2024

sicoyle commented Nov 20, 2024

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

dapr-bot commented Dec 3, 2024 • edited Loading

Components conformance test

❌ Some conformance tests failed

dapr-bot commented Dec 3, 2024 • edited Loading

Components certification test

❌ Some certification tests failed

dapr-bot commented Dec 3, 2024

Complete Build Matrix

sicoyle commented Dec 3, 2024

sicoyle commented Dec 3, 2024

dapr-bot commented Dec 3, 2024

Complete Build Matrix

dapr-bot commented Dec 3, 2024 • edited Loading

Components certification test

❌ Some certification tests failed

dapr-bot commented Dec 3, 2024 • edited Loading

Components conformance test

❌ Some conformance tests failed

sicoyle commented Nov 15, 2024 •

edited

Loading

dapr-bot commented Nov 19, 2024 •

edited

Loading

dapr-bot commented Nov 19, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading