Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rds): subnet selection not respected for multi user secret rotation #19237

Merged
merged 8 commits into from
Mar 8, 2022

Conversation

jogold
Copy link
Contributor

@jogold jogold commented Mar 4, 2022

The subnet selection was always overriden by the subnet selection of the
instance/cluster.

Avoid these kinds of errors by explicitely defining rotation options and
their defaults.

Closes #19233


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

The subnet selection was always overriden by the subnet selection of the
instance/cluster.

Avoid these kinds of errors by explicitely defining rotation options and
their defaults.

Closes aws#19233
@gitpod-io
Copy link

gitpod-io bot commented Mar 4, 2022

@github-actions github-actions bot added the @aws-cdk/aws-rds Related to Amazon Relational Database label Mar 4, 2022
@rix0rrr rix0rrr added bug This issue is a bug. p1 and removed bug This issue is a bug. p1 @aws-cdk/aws-rds Related to Amazon Relational Database labels Mar 4, 2022
Copy link
Contributor

@skinny85 skinny85 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for the fix @jogold! A few minor comments.

Comment on lines 959 to 965
const rotationOptions: Complete<RotationMultiUserOptions> = {
automaticallyAfter: options.automaticallyAfter,
endpoint: options.endpoint,
excludeCharacters: options.excludeCharacters ?? DEFAULT_PASSWORD_EXCLUDE_CHARS,
secret: options.secret,
vpcSubnets: options.vpcSubnets ?? this.vpcPlacement,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block is repeated 4 times (yes, I know secret is not always there 😛). Can we get rid of this duplication?

Comment on lines 590 to 596
const rotationOptions: Complete<RotationMultiUserOptions> = {
automaticallyAfter: options.automaticallyAfter,
endpoint: options.endpoint,
excludeCharacters: options.excludeCharacters ?? DEFAULT_PASSWORD_EXCLUDE_CHARS,
secret: options.secret,
vpcSubnets: options.vpcSubnets ?? this.vpcSubnets,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block is repeated 6 times (yes, I know secret is not always there 😛). Can we get rid of this duplication?

@@ -954,6 +954,76 @@ describe('cluster', () => {
});
});

test('addRotationMultiUser() with options', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we write a better description of what this test is checking?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are testing here that those rotation options are correctly passed: automaticallyAfter, excludeCharacters and vpcSubnets.

Same as here:

test('addRotationSingleUser() with options', () => {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, can we write something like that? 🙂

masterSecretArn: {
Ref: 'DatabaseSecretAttachmentE5D1B020',
},
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to say - I don't understand this assertion...

  1. Do we need all of these properties here? Are we not worried only about the vpcSubnetIds?

  2.       'Fn::Join': ['', [
            { Ref: 'VpcprivateSubnet1SubnetCEAD3716' },
            ',',
            { Ref: 'VpcprivateSubnet2Subnet2DE7549C' },
          ]],
    

    Is there a clearer way we can formulate this assertion? Maybe imported subnets? It is not clear to me that these are the PRIVATE_WITH_NAT that we provided when creating the rotation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Do we need all of these properties here? Are we not worried only about the vpcSubnetIds?

We can remove functionName and vpcSecurityGroupIds. We need the other ones because we are checking that options are correctly passed (and endpoint is also an option).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Is there a clearer way we can formulate this assertion?

See the propostion with stack.resolve(). Do we want to update the same test for single user rotation?

Copy link
Contributor

@skinny85 skinny85 Mar 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of weird resolve() tricks, can't we do this?

    // GIVEN
    vpc = ec2.Vpc.fromVpcAttributes(stack, 'Vpc', {
      vpcId: 'vpc-id',
      availabilityZones: ['az1'],
      publicSubnetIds: ['public-subnet-id-1', 'public-subnet-id-2'],
      publicSubnetNames: ['public-subnet-name-1', 'public-subnet-name-2'],
      privateSubnetIds: ['private-subnet-id-1', 'private-subnet-id-2'],
      privateSubnetNames: ['private-subnet-name-1', 'private-subnet-name-2'],
      isolatedSubnetIds: ['isolated-subnet-id-1', 'isolated-subnet-id-2'],
      isolatedSubnetNames: ['isolated-subnet-name-1', 'isolated-subnet-name-2'],
    });
    const instance = new rds.DatabaseInstance(stack, 'Instance', {
      engine: rds.DatabaseInstanceEngine.MYSQL,
      vpc: vpc,
      vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
    });

    // WHEN
    instance.addRotationMultiUser('user', {
      secret: new rds.DatabaseSecret(stack, 'DatabaseSecret', {
        username: 'user',
      }),
      vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_NAT },
    });

    // THEN
    Template.fromStack(stack).hasResourceProperties('AWS::Serverless::Application', {
      Parameters: {
        vpcSubnetIds: 'private-subnet-id-1,private-subnet-id-2',
      },
    });

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it!

},
},
});
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here as for the test above.

@mergify mergify bot dismissed skinny85’s stale review March 5, 2022 10:09

Pull request has been modified.

@jogold jogold requested a review from skinny85 March 5, 2022 10:35
Copy link
Contributor

@skinny85 skinny85 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there! 🙂

@@ -456,7 +456,7 @@ export abstract class SnapshotCredentials {
/**
* Properties common to single-user and multi-user rotation options.
*/
interface CommonRotationUserOptions {
export interface CommonRotationUserOptions {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change...? I guess because you need this type in applyDefaultRotationOptions()...?

How about creating a new file in lib/private, something like rotation-utils.ts, that contains this type, and the applyDefaultRotationOptions() function?

Any chance we can move this to a private file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not possible to have a public interface that extends an interface from a non exported file with jsii...

error JSII9002: Unable to resolve type "@aws-cdk/aws-rds.CommonRotationUserOptions". It may be @internal or not exported from the module's entry point (as configured in "package.json" as "main").

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's weird, because this interface used to be not exported, but JSII is gonna JSII...😉. Let's leave it as-is.

* Transforms optional properties to required properties that may be undefined
*/
type Complete<T> = {
[P in keyof Required<T>]: Pick<T, P> extends Required<Pick<T, P>> ? T[P] : (T[P] | undefined);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't even pretend I understand what this is doing 😃.

Do we really need this?

@@ -954,6 +954,76 @@ describe('cluster', () => {
});
});

test('addRotationMultiUser() with options', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, can we write something like that? 🙂

masterSecretArn: {
Ref: 'DatabaseSecretAttachmentE5D1B020',
},
},
Copy link
Contributor

@skinny85 skinny85 Mar 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of weird resolve() tricks, can't we do this?

    // GIVEN
    vpc = ec2.Vpc.fromVpcAttributes(stack, 'Vpc', {
      vpcId: 'vpc-id',
      availabilityZones: ['az1'],
      publicSubnetIds: ['public-subnet-id-1', 'public-subnet-id-2'],
      publicSubnetNames: ['public-subnet-name-1', 'public-subnet-name-2'],
      privateSubnetIds: ['private-subnet-id-1', 'private-subnet-id-2'],
      privateSubnetNames: ['private-subnet-name-1', 'private-subnet-name-2'],
      isolatedSubnetIds: ['isolated-subnet-id-1', 'isolated-subnet-id-2'],
      isolatedSubnetNames: ['isolated-subnet-name-1', 'isolated-subnet-name-2'],
    });
    const instance = new rds.DatabaseInstance(stack, 'Instance', {
      engine: rds.DatabaseInstanceEngine.MYSQL,
      vpc: vpc,
      vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
    });

    // WHEN
    instance.addRotationMultiUser('user', {
      secret: new rds.DatabaseSecret(stack, 'DatabaseSecret', {
        username: 'user',
      }),
      vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_NAT },
    });

    // THEN
    Template.fromStack(stack).hasResourceProperties('AWS::Serverless::Application', {
      Parameters: {
        vpcSubnetIds: 'private-subnet-id-1,private-subnet-id-2',
      },
    });

?

@mergify mergify bot dismissed skinny85’s stale review March 8, 2022 09:00

Pull request has been modified.

@jogold jogold requested a review from skinny85 March 8, 2022 09:07
Copy link
Contributor

@skinny85 skinny85 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it, thanks @jogold!

/**
* Applies defaults for rotation options
*/
export function applyDefaultRotationOptions(options: CommonRotationUserOptions, defaultvpcSubnets?: ec2.SubnetSelection): CommonRotationUserOptions {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So much simpler, love it!

@@ -456,7 +456,7 @@ export abstract class SnapshotCredentials {
/**
* Properties common to single-user and multi-user rotation options.
*/
interface CommonRotationUserOptions {
export interface CommonRotationUserOptions {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's weird, because this interface used to be not exported, but JSII is gonna JSII...😉. Let's leave it as-is.

masterSecretArn: {
Ref: 'DatabaseSecretAttachmentE5D1B020',
},
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it!

@mergify
Copy link
Contributor

mergify bot commented Mar 8, 2022

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Comment on lines +888 to +890
vpcWithIsolated.selectSubnets({
subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
}).subnetIds;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can kill this safely:

Suggested change
vpcWithIsolated.selectSubnets({
subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
}).subnetIds;

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
  • Commit ID: 7a447a6
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit dc7a17c into aws:master Mar 8, 2022
@mergify
Copy link
Contributor

mergify bot commented Mar 8, 2022

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

jogold added a commit to jogold/aws-cdk that referenced this pull request Mar 8, 2022
mergify bot pushed a commit that referenced this pull request Mar 8, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Forgot to clean this up in #19237


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TheRealAmazonKendra pushed a commit to TheRealAmazonKendra/aws-cdk that referenced this pull request Mar 11, 2022
…on (aws#19237)

The subnet selection was always overriden by the subnet selection of the
instance/cluster.

Avoid these kinds of errors by explicitely defining rotation options and
their defaults.

Closes aws#19233


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TheRealAmazonKendra pushed a commit to TheRealAmazonKendra/aws-cdk that referenced this pull request Mar 11, 2022
Forgot to clean this up in aws#19237


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@jogold jogold deleted the rds-multi-rotation-subnet branch March 12, 2022 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws-rds: Subnet selection doesn't work for cluster.addRotationMultiUser
4 participants