cdk bootstrap fails with AccessDenied

[metcalfc]

What is the problem?

A bootstrap fails. My IAM user has Cloud Formation Full access.

⏳  Bootstrapping environment aws://123/us-west-2...
❌  Environment aws://123/us-west-2 failed bootstrapping: AccessDenied: User: arn:aws:iam::123:user/metcalfc is not authorized to perform: cloudformation:DescribeStacks on resource: arn:aws:cloudformation:us-west-2:123:stack/CDKToolkit/* with an explicit deny in an identity-based policy
    at Request.extractError (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/request.js:690:12)
    at Request.callListeners (/usr/local/share/.config/yarn/global/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'AccessDenied',
  time: 2021-11-08T21:37:40.255Z,
  requestId: 'a1f21f21-ad16-48aa-a63d-fed7f1eed140',
  statusCode: 403,
  retryable: false,
  retryDelay: 626.886871840612
}
User: arn:aws:iam::123:user/metcalfc is not authorized to perform: cloudformation:DescribeStacks on resource: arn:aws:cloudformation:us-west-2:123:stack/CDKToolkit/* with an explicit deny in an identity-based policy

Reproduction Steps

It happens every bootstrap.

 cdk bootstrap aws://123/us-west-2

I've tried with the legacy and the modern strategy.

ℹ️ CDK Version: 1.131.0 (build 7560c79)
ℹ️ AWS environment variables:
  - AWS_STS_REGIONAL_ENDPOINTS = regional
  - AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  - AWS_SDK_LOAD_CONFIG = 1
ℹ️ CDK environment variables:
  - CDK_NEW_BOOTSTRAP = 1

What did you expect to happen?

I expected a successful bootstrap.

What actually happened?

The bootstrap failed.

CDK CLI Version

1.131.0 (build 7560c79)

Framework Version

No response

Node.js Version

v14.18.1

OS

Ubuntu 20.04

Language

Typescript

Language Version

No response

Other information

No response

[metcalfc]

I ran aws-nuke and it found an S3 CF bucket left over from an ancient deep racer workshop I did at an AWS loft. After deleting that bucket it started working. I'm not sure if its possible to do a better check and at least tell folks to check S3 for other CF buckets.

If it is possible to add a check that would be awesome. I spent I long time and got fairly desperate with aws-nuke. Otherwise, please feel free to close this.

[ryparker]

Hey @metcalfc 👋🏻

I'm glad you were able to get it working. I'm not sure why an existing S3 bucket (that's unrelated to CDK) would cause an Access Denied during bootstrap. Do you have any further information about the S3 bucket's policies or how it was setup?

[metcalfc]

It seems odd to me as well. I didn't clean up any policies. I restricted aws-nuke to resources in us-west-2. This is what it reported about the bucket in question:

us-west-2 - S3Bucket - s3://cf-templates-l9s2byolwgsf-us-west-2 - [Name: "cf-templates-l9s2byolwgsf-us-west-2"] - waiting
us-west-2 - S3Object - s3://cf-templates-l9s2byolwgsf-us-west-2/2021312uYN-bootstrap-template.yaml - [Bucket: "cf-templates-l9s2byolwgsf-us-west-2", IsLatest: "true", Key: "2021312uYN-bootstrap-template.yaml", VersionID: "null"] - removed

That might have been an old CDK bootstrap? But I don't understand why it would care about an old bucket, when it was trying to go into a different bucket name.

[ryparker]

Thanks for the response. It looks like this may be related to: #15307

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.