Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to add secret rotation to an RDS database #7885

Closed
wbprice opened this issue May 8, 2020 · 8 comments · Fixed by #7896
Closed

Unable to add secret rotation to an RDS database #7885

wbprice opened this issue May 8, 2020 · 8 comments · Fixed by #7896
Assignees
@skinny85
Labels
@aws-cdk/aws-secretsmanager Related to AWS Secrets Manager bug This issue is a bug. in-progress This issue is being actively worked on. p1

Comments

@wbprice
Copy link

wbprice commented May 8, 2020

After using .addRotationSingleUser() on a database instance, CDK fails to deploy the rotation application stack successfully.

Reproduction Steps

Code Sample:

import * as cdk from "@aws-cdk/core";
import { Vpc, InstanceClass, InstanceSize, InstanceType } from "@aws-cdk/aws-ec2";
import { DatabaseInstance, DatabaseInstanceEngine } from "@aws-cdk/aws-rds";
import { Duration } from "@aws-cdk/core";

export class AwsCdkDatabaseSecretErrorReplicationStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = new Vpc(this, "TestVPC");

    const database = new DatabaseInstance(this, "testDatbase", {
      instanceIdentifier: "testDatabase",
      engine: DatabaseInstanceEngine.POSTGRES,
      engineVersion: "10.6",
      instanceClass: InstanceType.of(InstanceClass.BURSTABLE2, InstanceSize.MICRO),
      masterUsername: "gladmin",
      vpc,
      backupRetention: Duration.days(30),
      multiAz: true
    });

    database.addRotationSingleUser();
  }
}

Error Log

 25/35 | 6:58:38 PM | CREATE_FAILED        | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F) Embedded stack arn:aws:cloudformation:us-east-1:456718055477:stack/AwsCdkDatabaseSecretErrorReplicationStack-testDatbaseRotationSingleUser544CCB3F-1J7EYPVP6B872/57bcda50-917f-11ea-8713-12cf4a8c2bc2 was not successfully created: The following resource(s) failed to create: [SecretsManagerRDSPostgreSQLRotationSingleUser].
	new SecretRotation (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-secretsmanager/lib/secret-rotation.ts:239:25)
	\_ DatabaseInstance.addRotationSingleUser (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:786:12)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:23:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 26/35 | 6:58:39 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 27/35 | 6:58:39 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 28/35 | 6:58:39 PM | CREATE_FAILED        | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C) Resource creation cancelled
	new DatabaseInstance (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:870:22)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:12:22)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

Environment

  • CLI Version : aws-cli/2.0.1 Python/3.7.4 Darwin/19.4.0 botocore/2.0.0dev5
  • Framework Version: 1.38.0 (build d5fa31f)
  • OS : MacOS Catalina 10.15.4 (19E287)
  • Language : English

Other

  • I created a test case here.
  • I'm deploying this into a personal account using an admin-level user. I don't think it's a permission issue.

This is 🐛 Bug Report
@wbprice wbprice added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels May 8, 2020
@jogold
Copy link
Contributor

jogold commented May 9, 2020 •
edited
Loading

What error messages are you seeing in the nested stack?

@wbprice
Copy link
Author

wbprice commented May 9, 2020

Unfortunately, CDK doesn't seem to log any errors from the nested stack, only reporting that it failed to create.

I'll attach the output with verbose logging turned on in hopes that it's helpful:

CDK toolkit version: 1.38.0 (build d5fa31f)
Command line arguments: {
  _: [ 'deploy' ],
  verbose: true,
  v: true,
  'ignore-errors': false,
  ignoreErrors: false,
  json: false,
  j: false,
  ec2creds: undefined,
  i: undefined,
  'version-reporting': undefined,
  versionReporting: undefined,
  'path-metadata': true,
  pathMetadata: true,
  'asset-metadata': true,
  assetMetadata: true,
  'role-arn': undefined,
  r: undefined,
  roleArn: undefined,
  staging: true,
  'no-color': false,
  noColor: false,
  fail: false,
  'build-exclude': [],
  E: [],
  buildExclude: [],
  ci: false,
  execute: true,
  force: false,
  f: false,
  parameters: [ {} ],
  'previous-parameters': true,
  previousParameters: true,
  '$0': 'cdk'
}
cdk.json: {
  "app": "npx ts-node bin/aws-cdk-database-secret-error-replication.ts",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true"
  }
}
merged settings: {
  versionReporting: true,
  pathMetadata: true,
  output: 'cdk.out',
  app: 'npx ts-node bin/aws-cdk-database-secret-error-replication.ts',
  context: {
    '@aws-cdk/core:enableStackNameDuplicates': 'true',
    'aws-cdk:enableDiffNoFail': 'true'
  },
  tags: [],
  assetMetadata: true,
  toolkitBucket: {},
  staging: true
}
Determining whether we're on an EC2 instance.
Does not look like EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
Resolving default credentials
Retrieved account ID 600917086294 from disk cache
Setting "CDK_DEFAULT_ACCOUNT" environment variable to 600917086294
context: {
  '@aws-cdk/core:enableStackNameDuplicates': 'true',
  'aws-cdk:enableDiffNoFail': 'true',
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true
}
outdir: cdk.out
env: {
  CDK_DEFAULT_REGION: 'us-east-1',
  CDK_DEFAULT_ACCOUNT: '600917086294',
  CDK_CONTEXT_JSON: '{"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '2.0.0',
  CDK_CLI_VERSION: '1.38.0'
}
Reading existing template for stack AwsCdkDatabaseSecretErrorReplicationStack.
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
┌───┬────────────────────────────────────┬────────┬────────────────────────────────────┬────────────────────────────────────┬───────────┐
│   │ Resource                           │ Effect │ Action                             │ Principal                          │ Condition │
├───┼────────────────────────────────────┼────────┼────────────────────────────────────┼────────────────────────────────────┼───────────┤
│ + │ ${testDatbase/Secret/Attachment}   │ Deny   │ secretsmanager:DeleteSecret        │ AWS:arn:${AWS::Partition}:iam::${A │           │
│   │                                    │        │                                    │ WS::AccountId}:root                │           │
└───┴────────────────────────────────────┴────────┴────────────────────────────────────┴────────────────────────────────────┴───────────┘
Security Group Changes
┌───┬─────────────────────────────────────────┬─────┬─────────────────────────────────────────┬─────────────────────────────────────────┐
│   │ Group                                   │ Dir │ Protocol                                │ Peer                                    │
├───┼─────────────────────────────────────────┼─────┼─────────────────────────────────────────┼─────────────────────────────────────────┤
│ + │ ${testDatbase/RotationSingleUser/Securi │ Out │ Everything                              │ Everyone (IPv4)                         │
│   │ tyGroup.GroupId}                        │     │                                         │                                         │
├───┼─────────────────────────────────────────┼─────┼─────────────────────────────────────────┼─────────────────────────────────────────┤
│ + │ ${testDatbase/SecurityGroup.GroupId}    │ In  │ TCP ${testDatbase.Endpoint.Port}        │ ${testDatbase/RotationSingleUser/Securi │
│   │                                         │     │                                         │ tyGroup.GroupId}                        │
│ + │ ${testDatbase/SecurityGroup.GroupId}    │ Out │ Everything                              │ Everyone (IPv4)                         │
└───┴─────────────────────────────────────────┴─────┴─────────────────────────────────────────┴─────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Do you wish to deploy these changes (y/n)? y
AwsCdkDatabaseSecretErrorReplicationStack: deploying...
Waiting for stack CDKToolkit to finish creating or updating...
AwsCdkDatabaseSecretErrorReplicationStack: checking if we can skip deploy
AwsCdkDatabaseSecretErrorReplicationStack: no existing stack
AwsCdkDatabaseSecretErrorReplicationStack: deploying...
Attempting to create ChangeSet CDK-7028fa29-8974-48a1-a030-f475fec38d3d to create stack AwsCdkDatabaseSecretErrorReplicationStack
AwsCdkDatabaseSecretErrorReplicationStack: creating CloudFormation changeset...
Initiated creation of changeset: arn:aws:cloudformation:us-east-1:600917086294:changeSet/CDK-7028fa29-8974-48a1-a030-f475fec38d3d/19495dec-a795-45a8-8c06-6f6e418c0fda; waiting for it to finish creating...
Waiting for changeset CDK-7028fa29-8974-48a1-a030-f475fec38d3d on stack AwsCdkDatabaseSecretErrorReplicationStack to finish creating...
Changeset CDK-7028fa29-8974-48a1-a030-f475fec38d3d on stack AwsCdkDatabaseSecretErrorReplicationStack is still creating
Changeset CDK-7028fa29-8974-48a1-a030-f475fec38d3d on stack AwsCdkDatabaseSecretErrorReplicationStack is still creating
Initiating execution of changeset CDK-7028fa29-8974-48a1-a030-f475fec38d3d on stack AwsCdkDatabaseSecretErrorReplicationStack
Execution of changeset CDK-7028fa29-8974-48a1-a030-f475fec38d3d on stack AwsCdkDatabaseSecretErrorReplicationStack has started; waiting for the update to complete...
Waiting for stack AwsCdkDatabaseSecretErrorReplicationStack to finish creating or updating...
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS (User Initiated))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                          | CDKMetadata
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet1/EIP (TestVPCPublicSubnet1EIPB686BDA0)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::VPC                               | TestVPC (TestVPCD26570D8)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::InternetGateway                   | TestVPC/IGW (TestVPCIGW5175BCF9)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet2/EIP (TestVPCPublicSubnet2EIP59BF172B)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::SecretsManager::Secret                 | testDatbase/Secret (testDatbaseSecret6C76D85B)
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet1/EIP (TestVPCPublicSubnet1EIPB686BDA0) Resource creation Initiated
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::VPC                               | TestVPC (TestVPCD26570D8) Resource creation Initiated
  0/35 | 2:16:19 PM | CREATE_IN_PROGRESS   | AWS::EC2::InternetGateway                   | TestVPC/IGW (TestVPCIGW5175BCF9) Resource creation Initiated
  0/35 | 2:16:20 PM | CREATE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet2/EIP (TestVPCPublicSubnet2EIP59BF172B) Resource creation Initiated
  0/35 | 2:16:21 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                          | CDKMetadata Resource creation Initiated
  1/35 | 2:16:21 PM | CREATE_COMPLETE      | AWS::CDK::Metadata                          | CDKMetadata
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
  1/35 | 2:16:21 PM | CREATE_IN_PROGRESS   | AWS::SecretsManager::Secret                 | testDatbase/Secret (testDatbaseSecret6C76D85B) Resource creation Initiated
  2/35 | 2:16:21 PM | CREATE_COMPLETE      | AWS::SecretsManager::Secret                 | testDatbase/Secret (testDatbaseSecret6C76D85B)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
  3/35 | 2:16:35 PM | CREATE_COMPLETE      | AWS::EC2::InternetGateway                   | TestVPC/IGW (TestVPCIGW5175BCF9)
  4/35 | 2:16:36 PM | CREATE_COMPLETE      | AWS::EC2::EIP                               | TestVPC/PublicSubnet2/EIP (TestVPCPublicSubnet2EIP59BF172B)
  5/35 | 2:16:36 PM | CREATE_COMPLETE      | AWS::EC2::VPC                               | TestVPC (TestVPCD26570D8)
  6/35 | 2:16:36 PM | CREATE_COMPLETE      | AWS::EC2::EIP                               | TestVPC/PublicSubnet1/EIP (TestVPCPublicSubnet1EIPB686BDA0)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::VPCGatewayAttachment              | TestVPC/VPCGW (TestVPCVPCGWF063BA4F)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet1/RouteTable (TestVPCPrivateSubnet1RouteTable1BE239FB)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet1/RouteTable (TestVPCPublicSubnet1RouteTable959CA972)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/RotationSingleUser/SecurityGroup (testDatbaseRotationSingleUserSecurityGroupF2499C61)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet2/RouteTable (TestVPCPublicSubnet2RouteTable9C86C355)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet1/Subnet (TestVPCPrivateSubnet1Subnet005D5ABE)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/SecurityGroup (testDatbaseSecurityGroup5F7F3ECD)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::VPCGatewayAttachment              | TestVPC/VPCGW (TestVPCVPCGWF063BA4F) Resource creation Initiated
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet1/Subnet (TestVPCPublicSubnet1SubnetAE333572)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet1/RouteTable (TestVPCPrivateSubnet1RouteTable1BE239FB) Resource creation Initiated
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet2/RouteTable (TestVPCPrivateSubnet2RouteTable9B691DB2)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet1/RouteTable (TestVPCPublicSubnet1RouteTable959CA972) Resource creation Initiated
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet2/Subnet (TestVPCPrivateSubnet2Subnet45EC7DE3)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet2/Subnet (TestVPCPublicSubnet2SubnetF2FC6943)
  6/35 | 2:16:38 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet2/RouteTable (TestVPCPublicSubnet2RouteTable9C86C355) Resource creation Initiated
  6/35 | 2:16:39 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet1/Subnet (TestVPCPrivateSubnet1Subnet005D5ABE) Resource creation Initiated
  6/35 | 2:16:39 PM | CREATE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet2/RouteTable (TestVPCPrivateSubnet2RouteTable9B691DB2) Resource creation Initiated
  6/35 | 2:16:39 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet1/Subnet (TestVPCPublicSubnet1SubnetAE333572) Resource creation Initiated
  6/35 | 2:16:39 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet2/Subnet (TestVPCPrivateSubnet2Subnet45EC7DE3) Resource creation Initiated
  6/35 | 2:16:39 PM | CREATE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet2/Subnet (TestVPCPublicSubnet2SubnetF2FC6943) Resource creation Initiated
  7/35 | 2:16:39 PM | CREATE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet1/RouteTable (TestVPCPrivateSubnet1RouteTable1BE239FB)
  8/35 | 2:16:39 PM | CREATE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet1/RouteTable (TestVPCPublicSubnet1RouteTable959CA972)
  9/35 | 2:16:40 PM | CREATE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet2/RouteTable (TestVPCPrivateSubnet2RouteTable9B691DB2)
 10/35 | 2:16:41 PM | CREATE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet2/RouteTable (TestVPCPublicSubnet2RouteTable9C86C355)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 10/35 | 2:16:43 PM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/RotationSingleUser/SecurityGroup (testDatbaseRotationSingleUserSecurityGroupF2499C61) Resource creation Initiated
 10/35 | 2:16:43 PM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/SecurityGroup (testDatbaseSecurityGroup5F7F3ECD) Resource creation Initiated
 11/35 | 2:16:45 PM | CREATE_COMPLETE      | AWS::EC2::SecurityGroup                     | testDatbase/SecurityGroup (testDatbaseSecurityGroup5F7F3ECD)
 12/35 | 2:16:45 PM | CREATE_COMPLETE      | AWS::EC2::SecurityGroup                     | testDatbase/RotationSingleUser/SecurityGroup (testDatbaseRotationSingleUserSecurityGroupF2499C61)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 13/35 | 2:16:54 PM | CREATE_COMPLETE      | AWS::EC2::VPCGatewayAttachment              | TestVPC/VPCGW (TestVPCVPCGWF063BA4F)
 14/35 | 2:16:55 PM | CREATE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet1/Subnet (TestVPCPrivateSubnet1Subnet005D5ABE)
 15/35 | 2:16:55 PM | CREATE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PublicSubnet1/Subnet (TestVPCPublicSubnet1SubnetAE333572)
 16/35 | 2:16:55 PM | CREATE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PrivateSubnet2/Subnet (TestVPCPrivateSubnet2Subnet45EC7DE3)
 17/35 | 2:16:55 PM | CREATE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PublicSubnet2/Subnet (TestVPCPublicSubnet2SubnetF2FC6943)
 17/35 | 2:16:56 PM | CREATE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet1/DefaultRoute (TestVPCPublicSubnet1DefaultRoute11545FC4)
 17/35 | 2:16:56 PM | CREATE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet2/DefaultRoute (TestVPCPublicSubnet2DefaultRouteDCA67168)
 17/35 | 2:16:56 PM | CREATE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet1/DefaultRoute (TestVPCPublicSubnet1DefaultRoute11545FC4) Resource creation Initiated
 17/35 | 2:16:56 PM | CREATE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet2/DefaultRoute (TestVPCPublicSubnet2DefaultRouteDCA67168) Resource creation Initiated
 17/35 | 2:16:57 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet1/RouteTableAssociation (TestVPCPrivateSubnet1RouteTableAssociation3FFF492E)
 17/35 | 2:16:57 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet1/RouteTableAssociation (TestVPCPublicSubnet1RouteTableAssociationD7253C7B)
 17/35 | 2:16:57 PM | CREATE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | testDatbase/SubnetGroup (testDatbaseSubnetGroupC801044A)
 17/35 | 2:16:57 PM | CREATE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 17/35 | 2:16:57 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F)
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet2/RouteTableAssociation (TestVPCPrivateSubnet2RouteTableAssociation08DD625F)
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet2/RouteTableAssociation (TestVPCPublicSubnet2RouteTableAssociationBD2B01F3)
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet1/RouteTableAssociation (TestVPCPrivateSubnet1RouteTableAssociation3FFF492E) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet1/RouteTableAssociation (TestVPCPublicSubnet1RouteTableAssociationD7253C7B) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22)
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | testDatbase/SubnetGroup (testDatbaseSubnetGroupC801044A) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet2/RouteTableAssociation (TestVPCPublicSubnet2RouteTableAssociationBD2B01F3) Resource creation Initiated
 17/35 | 2:16:58 PM | CREATE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet2/RouteTableAssociation (TestVPCPrivateSubnet2RouteTableAssociation08DD625F) Resource creation Initiated
 18/35 | 2:16:58 PM | CREATE_COMPLETE      | AWS::RDS::DBSubnetGroup                     | testDatbase/SubnetGroup (testDatbaseSubnetGroupC801044A)
 18/35 | 2:16:59 PM | CREATE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22) Resource creation Initiated
 18/35 | 2:17:01 PM | CREATE_IN_PROGRESS   | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 18/35 | 2:17:03 PM | CREATE_IN_PROGRESS   | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C) Resource creation Initiated
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 19/35 | 2:17:12 PM | CREATE_COMPLETE      | AWS::EC2::Route                             | TestVPC/PublicSubnet1/DefaultRoute (TestVPCPublicSubnet1DefaultRoute11545FC4)
 20/35 | 2:17:12 PM | CREATE_COMPLETE      | AWS::EC2::Route                             | TestVPC/PublicSubnet2/DefaultRoute (TestVPCPublicSubnet2DefaultRouteDCA67168)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
 21/35 | 2:17:13 PM | CREATE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet1/RouteTableAssociation (TestVPCPrivateSubnet1RouteTableAssociation3FFF492E)
 22/35 | 2:17:13 PM | CREATE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet1/RouteTableAssociation (TestVPCPublicSubnet1RouteTableAssociationD7253C7B)
 23/35 | 2:17:14 PM | CREATE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet2/RouteTableAssociation (TestVPCPrivateSubnet2RouteTableAssociation08DD625F)
 24/35 | 2:17:14 PM | CREATE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet2/RouteTableAssociation (TestVPCPublicSubnet2RouteTableAssociationBD2B01F3)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (CREATE_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
 25/35 | 2:17:46 PM | CREATE_FAILED        | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F) Embedded stack arn:aws:cloudformation:us-east-1:600917086294:stack/AwsCdkDatabaseSecretErrorReplicationStack-testDatbaseRotationSingleUser544CCB3F-1FPEF5MTQ6TJ5/45592e70-9221-11ea-b793-0a35f6ddac11 was not successfully created: The following resource(s) failed to create: [SecretsManagerRDSPostgreSQLRotationSingleUser].
	new SecretRotation (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-secretsmanager/lib/secret-rotation.ts:239:25)
	\_ DatabaseInstance.addRotationSingleUser (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:786:12)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:23:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 26/35 | 2:17:46 PM | CREATE_FAILED        | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C) Resource creation cancelled
	new DatabaseInstance (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:870:22)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:12:22)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 27/35 | 2:17:46 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 28/35 | 2:17:46 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 28/35 | 2:17:47 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack                  | AwsCdkDatabaseSecretErrorReplicationStack The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [testDatbase2F413D3C, testDatbaseRotationSingleUser544CCB3F, TestVPCPublicSubnet2NATGatewayBE12FD22, TestVPCPublicSubnet1NATGateway6A40FA74]. . Rollback requested by user.))
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet2/RouteTableAssociation (TestVPCPublicSubnet2RouteTableAssociationBD2B01F3)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet1/DefaultRoute (TestVPCPublicSubnet1DefaultRoute11545FC4)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::CDK::Metadata                          | CDKMetadata
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet1/RouteTableAssociation (TestVPCPublicSubnet1RouteTableAssociationD7253C7B)
 28/35 | 2:18:18 PM | DELETE_IN_PROGRESS   | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22)
 28/35 | 2:18:19 PM | DELETE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet2/RouteTableAssociation (TestVPCPrivateSubnet2RouteTableAssociation08DD625F)
 28/35 | 2:18:19 PM | DELETE_IN_PROGRESS   | AWS::EC2::Route                             | TestVPC/PublicSubnet2/DefaultRoute (TestVPCPublicSubnet2DefaultRouteDCA67168)
 28/35 | 2:18:19 PM | DELETE_IN_PROGRESS   | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet1/RouteTableAssociation (TestVPCPrivateSubnet1RouteTableAssociation3FFF492E)
 28/35 | 2:18:19 PM | DELETE_SKIPPED       | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C)
 29/35 | 2:18:20 PM | DELETE_COMPLETE      | AWS::CDK::Metadata                          | CDKMetadata
 29/35 | 2:18:20 PM | DELETE_IN_PROGRESS   | AWS::SecretsManager::Secret                 | testDatbase/Secret (testDatbaseSecret6C76D85B)
 29/35 | 2:18:20 PM | DELETE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/SecurityGroup (testDatbaseSecurityGroup5F7F3ECD)
 29/35 | 2:18:20 PM | DELETE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | testDatbase/SubnetGroup (testDatbaseSubnetGroupC801044A)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 30/35 | 2:18:21 PM | DELETE_COMPLETE      | AWS::EC2::SecurityGroup                     | testDatbase/SecurityGroup (testDatbaseSecurityGroup5F7F3ECD)
 31/35 | 2:18:21 PM | DELETE_COMPLETE      | AWS::SecretsManager::Secret                 | testDatbase/Secret (testDatbaseSecret6C76D85B)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 32/35 | 2:18:30 PM | DELETE_COMPLETE      | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F)
 32/35 | 2:18:30 PM | DELETE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | testDatbase/RotationSingleUser/SecurityGroup (testDatbaseRotationSingleUserSecurityGroupF2499C61)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 33/35 | 2:18:32 PM | DELETE_COMPLETE      | AWS::EC2::SecurityGroup                     | testDatbase/RotationSingleUser/SecurityGroup (testDatbaseRotationSingleUserSecurityGroupF2499C61)
 34/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet2/RouteTableAssociation (TestVPCPublicSubnet2RouteTableAssociationBD2B01F3)
 35/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet2/RouteTableAssociation (TestVPCPrivateSubnet2RouteTableAssociation08DD625F)
 36/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::Route                             | TestVPC/PublicSubnet2/DefaultRoute (TestVPCPublicSubnet2DefaultRouteDCA67168)
 37/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PrivateSubnet1/RouteTableAssociation (TestVPCPrivateSubnet1RouteTableAssociation3FFF492E)
 38/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::SubnetRouteTableAssociation       | TestVPC/PublicSubnet1/RouteTableAssociation (TestVPCPublicSubnet1RouteTableAssociationD7253C7B)
 39/35 | 2:18:34 PM | DELETE_COMPLETE      | AWS::EC2::Route                             | TestVPC/PublicSubnet1/DefaultRoute (TestVPCPublicSubnet1DefaultRoute11545FC4)
 39/35 | 2:18:35 PM | DELETE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet2/RouteTable (TestVPCPrivateSubnet2RouteTable9B691DB2)
 39/35 | 2:18:35 PM | DELETE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet2/RouteTable (TestVPCPublicSubnet2RouteTable9C86C355)
 39/35 | 2:18:35 PM | DELETE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet1/RouteTable (TestVPCPublicSubnet1RouteTable959CA972)
 39/35 | 2:18:35 PM | DELETE_IN_PROGRESS   | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet1/RouteTable (TestVPCPrivateSubnet1RouteTable1BE239FB)
 39/35 | 2:18:35 PM | DELETE_IN_PROGRESS   | AWS::EC2::VPCGatewayAttachment              | TestVPC/VPCGW (TestVPCVPCGWF063BA4F)
 40/35 | 2:18:36 PM | DELETE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet2/RouteTable (TestVPCPrivateSubnet2RouteTable9B691DB2)
 41/35 | 2:18:36 PM | DELETE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet2/RouteTable (TestVPCPublicSubnet2RouteTable9C86C355)
 42/35 | 2:18:36 PM | DELETE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PrivateSubnet1/RouteTable (TestVPCPrivateSubnet1RouteTable1BE239FB)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 43/35 | 2:19:07 PM | DELETE_COMPLETE      | AWS::EC2::RouteTable                        | TestVPC/PublicSubnet1/RouteTable (TestVPCPublicSubnet1RouteTable959CA972)
 44/35 | 2:19:08 PM | DELETE_COMPLETE      | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 44/35 | 2:19:09 PM | DELETE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet1/Subnet (TestVPCPublicSubnet1SubnetAE333572)
 44/35 | 2:19:09 PM | DELETE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet1/EIP (TestVPCPublicSubnet1EIPB686BDA0)
 45/35 | 2:19:10 PM | DELETE_COMPLETE      | AWS::EC2::EIP                               | TestVPC/PublicSubnet1/EIP (TestVPCPublicSubnet1EIPB686BDA0)
 46/35 | 2:19:13 PM | DELETE_COMPLETE      | AWS::EC2::VPCGatewayAttachment              | TestVPC/VPCGW (TestVPCVPCGWF063BA4F)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 46/35 | 2:19:14 PM | DELETE_IN_PROGRESS   | AWS::EC2::InternetGateway                   | TestVPC/IGW (TestVPCIGW5175BCF9)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 47/35 | 2:19:25 PM | DELETE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PublicSubnet1/Subnet (TestVPCPublicSubnet1SubnetAE333572)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 48/35 | 2:19:30 PM | DELETE_COMPLETE      | AWS::EC2::InternetGateway                   | TestVPC/IGW (TestVPCIGW5175BCF9)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 49/35 | 2:19:35 PM | DELETE_COMPLETE      | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22)
 49/35 | 2:19:36 PM | DELETE_IN_PROGRESS   | AWS::EC2::EIP                               | TestVPC/PublicSubnet2/EIP (TestVPCPublicSubnet2EIP59BF172B)
 49/35 | 2:19:36 PM | DELETE_IN_PROGRESS   | AWS::EC2::Subnet                            | TestVPC/PublicSubnet2/Subnet (TestVPCPublicSubnet2SubnetF2FC6943)
 50/35 | 2:19:37 PM | DELETE_COMPLETE      | AWS::EC2::EIP                               | TestVPC/PublicSubnet2/EIP (TestVPCPublicSubnet2EIP59BF172B)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
 51/35 | 2:19:53 PM | DELETE_COMPLETE      | AWS::EC2::Subnet                            | TestVPC/PublicSubnet2/Subnet (TestVPCPublicSubnet2SubnetF2FC6943)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)
51/35 Currently in progress: AwsCdkDatabaseSecretErrorReplicationStack, testDatbaseSubnetGroupC801044A
Stack AwsCdkDatabaseSecretErrorReplicationStack is still not stable (ROLLBACK_IN_PROGRESS)

@jogold
Copy link
Contributor

jogold commented May 9, 2020

Can you look in the console while deploying?

@wbprice
Copy link
Author

wbprice commented May 9, 2020 •
edited
Loading

Are you talking about https://console.aws.amazon.com/ or my terminal? Here's what I can see in Cloudformation, console output from cdk deploy is above.

Screen Shot 2020-05-09 at 3 25 42 PM

@jogold
Copy link
Contributor

jogold commented May 9, 2020

Yes in Cloudformation but for the nested stack that is creating the secret rotation.

@wbprice
Copy link
Author

wbprice commented May 9, 2020 •
edited
Loading

OK, cool. I didn't know nested stacks appeared in Cloudformation's stack list.

image

The error suggests that the lambda name AwsCdkDatabaseSecretErrorReplicationStacktestDatbaseRotationSingleUser90948986 is too long. I'll try again with a shorter stack name.

@wbprice
Copy link
Author

wbprice commented May 10, 2020

Yep, that was it. Thanks for your help!

Screen Shot 2020-05-09 at 8 21 37 PM

@wbprice wbprice closed this as completed May 10, 2020
@jogold
Copy link
Contributor

jogold commented May 10, 2020

You can leave this open, we can fix this in the CDK here:

aws-cdk/packages/@aws-cdk/aws-secretsmanager/lib/secret-rotation.ts

Line 213 in 685a4bf

const rotationFunctionName = this.node.uniqueId;

jogold added a commit to jogold/aws-cdk that referenced this issue May 10, 2020
@jogold
fix(secretsmanager): rotation function name exceeds 64 chars
0757125 
Get the last 64 chars of the `uniqueId`.

See aws#7885 (comment).

Closes aws#7885
@jogold jogold mentioned this issue May 10, 2020
Merged
@wbprice wbprice reopened this May 10, 2020
@SomayaB SomayaB added in-progress This issue is being actively worked on. @aws-cdk/aws-secretsmanager Related to AWS Secrets Manager labels May 11, 2020
@SomayaB SomayaB removed the needs-triage This issue or PR still needs to be triaged. label May 11, 2020
@skinny85 skinny85 added the p1 label May 29, 2020
ahammond pushed a commit to ahammond/aws-cdk that referenced this issue Jun 8, 2020
@jogold
fix(secretsmanager): rotation function name exceeds 64 chars
5db4871 
Get the last 64 chars of the `uniqueId`.

See aws#7885 (comment).

Closes aws#7885
@ahammond ahammond mentioned this issue Jun 8, 2020
Closed
@mergify mergify bot closed this as completed in #7896 Jun 10, 2020
mergify bot pushed a commit that referenced this issue Jun 10, 2020
@jogold
fix(secretsmanager): rotation function name can exceed 64 chars (#7896)
24e474b 
Get the last 64 chars of the `uniqueId`.

See #7885 (comment).

Closes #7885
Closes #8442

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@wchaws wchaws mentioned this issue Aug 18, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@skinny85 skinny85

Labels
@aws-cdk/aws-secretsmanager Related to AWS Secrets Manager bug This issue is a bug. in-progress This issue is being actively worked on. p1
Projects
None yet
Milestone
No milestone
4 participants
@skinny85 @wbprice @jogold @SomayaB