CDK CLI: assuming a doubly-chained role fails since 2.167.0

[Joe-Zer0]

Describe the bug

This may be related to #32120, but it's different enough I decided to create a separate issue.
2.166.0 works. 2.167.0 and 2.167.2 do not work.
Credentials File

[role0]
aws_access_key_id        = XXXXX
aws_secret_access_key    = XXXXX
aws_session_token        = XXXXX
aws_security_token       = XXXXX
x_principal_arn          = XXXXX
x_security_token_expires = XXXXX

[role1]
source_profile = role0
role_arn       = arn:aws:iam::12345:role/Role1

[role2]
source_profile = role1
role_arn       = arn:aws:iam::12345:role/Role2

Not sure if it's relevant, but role0 is for Account A and role1 and role2 are for Account B.
Using version 2.167.0, cdk synth --profile role1 works correctly, the CDK_DEFAULT_ACCOUNT environment variable is populated. The issue happens when running cdk synth --profile role2, CDK_DEFAULT_ACCOUNT is not populated. But CDK_DEFAULT_REGION is still populated correctly.

Please let me know if there is any additional information I can provide.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.166.0

Expected Behavior

I would expect cdk synth --profile role2 to populate CDK_DEFAULT_ACCOUNT with the account number.

Current Behavior

cdk synth --profile role2 does not populate CDK_DEFAULT_ACCOUNT with the account number.

Reproduction Steps

Run cdk synth on any stack with a profile that is "double assumed".

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.167.2

Framework Version

No response

Node.js Version

v20.17.0

OS

Windows 10

Language

Python

Language Version

No response

Other information

No response

[pahud]

but role1 is for Account A and role1 and role2 are for Account B.

I presume role0 for Account A and both role1,2 for Account B right?

[Joe-Zer0]

Apologies, you are correct. I'll update the initial comment as well.

[pahud]

This is my test

[profile role1]
role_arn = arn:aws:iam::ACCOUNT_ID:role/AdminRole4Switch
source_profile = default

[profile role2]
role_arn = arn:aws:iam::ACCOUNT_ID:role/AdminRole4Switch2
source_profile = role1

CLI Test

I can run the following commands using AWS CLI correctly

% aws --profile role1 sts get-caller-identity
% aws --profile role2 sts get-caller-identity

CDK Test

const app = new App();

const env = { region: process.env.CDK_DEFAULT_REGION, account: process.env.CDK_DEFAULT_ACCOUNT };

const stack = new Stack(app, 'issue-triage-stack', { env } )

new CfnOutput(stack, 'cdk_default_account', { value: process.env.CDK_DEFAULT_ACCOUNT || 'not found'  });
new CfnOutput(stack, 'cdk_default_region', { value: process.env.CDK_DEFAULT_REGION || 'not found'  });

I can run this correctly

% npx cdk --profile role1 diff

And see the Outputs

Outputs
[+] Output cdk_default_account cdkdefaultaccount: {"Value":"ACCOUNT"}
[+] Output cdk_default_region cdkdefaultregion: {"Value":"us-east-1"}

But it won't resolve using role2 profile in 1.166.0

% npx cdk --profile role2 diff               
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment

I don't think CDK 2.166.0 supports this. But if I change the config of role2 using role0 as the source

[profile role2]
role_arn = arn:aws:iam::ACCOUNT:role/AdminRole4Switch2
source_profile = default

cdk diff works pretty great!

Can you provide a minimal sample as above that works in 166 but not in 167?

[pahud]

by the way, we don't specify these in ~/.aws/config

are you sure they are configured in ~/.aws/config ?

aws_access_key_id        = XXXXX
aws_secret_access_key    = XXXXX

[Joe-Zer0]

Those are specified in ~/.aws/credentials. All 3 roles are in ~/.aws/credentials. Although I believe it is possible to move role1 and role2 to ~/.aws/config.

[Joe-Zer0]

Here's some more detailed reproduction steps, similar to what you have.
Using the same credentials file as I specified above:

⟫ aws sts get-caller-identity --profile role1
{
    "UserId": "XXXXX",
    "Account": "XXXXX",
    "Arn": "arn:aws:sts::XXXXX:assumed-role/Role1/botocore-session-XXXXX"
}

⟫ aws sts get-caller-identity --profile role2
{
    "UserId": "XXXXX",
    "Account": "XXXXX",
    "Arn": "arn:aws:sts::XXXXX:assumed-role/Role2/botocore-session-XXXXX"
}

cdk.json

{
  "app": "python app.py"
}

app.py

from aws_cdk import App
import os

print(f"CDK_DEFAULT_ACCOUNT = {os.environ.get('CDK_DEFAULT_ACCOUNT')}")
print(f"CDK_DEFAULT_REGION = {os.environ.get('CDK_DEFAULT_REGION')}")

app = App()
app.synth()

⟫ npm install aws-cdk@2.166.0 -g

⟫ cdk synth --profile role1
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ cdk synth --profile role2
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ npm install aws-cdk@2.167.0 -g

⟫ cdk synth --profile role1
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ cdk synth --profile role2
CDK_DEFAULT_ACCOUNT = None          <-- This is the issue
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

Hopefully that helps. Let me know if there's any other info I can provide.

[pahud]

OK so you are actually using CDK in Python but you mentioned TypeScript in your original description?

[Joe-Zer0]

Sorry about that, I must've misclicked. Fixed!

[pahud]

No worries.

Unfortunately, I still can't reproduce that in 2.166.0

% aws sts get-caller-identity --profile role1
{
    "UserId": "<***>",
    "Account": "<***>",
    "Arn": "arn:aws:sts::<***>:assumed-role/<***>"
}

% aws sts get-caller-identity --profile role2
{
    "UserId": "<***>",
    "Account": "<***>",
    "Arn": "arn:aws:sts::<***>:assumed-role/<***>"
}

 % cdk synth --profile role1
CDK_DEFAULT_ACCOUNT = ACCOUNT
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks
% cdk synth --profile role2
CDK_DEFAULT_ACCOUNT = None <--------
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks
% cdk version                      
2.166.0 (build 7bb9203)
% grep aws-cdk-lib requirements.txt
aws-cdk-lib==2.166.0

app.py

#!/usr/bin/env python3
import os
from aws_cdk import App

print(f"CDK_DEFAULT_ACCOUNT = {os.environ.get('CDK_DEFAULT_ACCOUNT')}")
print(f"CDK_DEFAULT_REGION = {os.environ.get('CDK_DEFAULT_REGION')}")

app = App()
app.synth()

Are you running on Windows 10?

[Joe-Zer0]

The issue is with the cdk cli version 1.167.0. Not with aws-cdk-lib 2.166.0.

[pahud]

Yes I know but from my testing even 2.166.0(both CLI and aws-cdk-lib) does not support that.

I wil reach out to someone else on my team to have a second look.

[Joe-Zer0]

Yeah, I'm running on Windows 10. I had someone try this on a Mac, and they're having the same issue. I tried it with javascript, and I'm seeing the same issue as well.

const cdk = require('aws-cdk-lib');

console.log(`CDK_DEFAULT_ACCOUNT = ${process.env.CDK_DEFAULT_ACCOUNT}`);
console.log(`CDK_DEFAULT_REGION = ${process.env.CDK_DEFAULT_REGION}`);

const app = new cdk.App();

⟫ npm install aws-cdk@2.166.0 -g

⟫ cdk synth --profile role1
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ cdk synth --profile role2
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ npm install aws-cdk@2.167.0 -g

⟫ cdk synth --profile role1
CDK_DEFAULT_ACCOUNT = XXXXX
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

⟫ cdk synth --profile role2
CDK_DEFAULT_ACCOUNT = None          <-- This is the issue
CDK_DEFAULT_REGION = us-east-1
This app contains no stacks

[iliapolo]

Related: #25870. I am also unable to reproduce. That is, i'm seeing the same behavior on 2.166.0. This is my ~/.aws/credentials file:

[role0]
aws_access_key_id        = XXXX
aws_secret_access_key    = XXXX
aws_session_token        = XXXX

[role1]
source_profile = role0
role_arn       = arn:aws:iam::ACCOUNT_B:role/TestIssue32195

[role2]
source_profile = role1
role_arn       = arn:aws:iam::ACCOUNT_B:role/TestIssue32195-2

• role0: ACCOUNT_A
• role1: ACCOUNT_B
• role2: ACCOUNT_B

Both role1 and role2 allow ACCOUNT_A in its trust policy.

This is what i'm seeing in 2.166.0:

cdk synth --profile role2                                                                                           
CDK_DEFAULT_ACCOUNT: undefined.  <-- Getting undefined in 2.166.0 as well
CDK_DEFAULT_REGION: us-east-1
This app contains no stacks

cdk synth --profile role1                                                                                                                               
CDK_DEFAULT_ACCOUNT: ACCOUNT_B
CDK_DEFAULT_REGION: us-east-1
This app contains no stacks

dk synth --profile role0                                                                                          
CDK_DEFAULT_ACCOUNT: ACCOUNT_A
CDK_DEFAULT_REGION: us-east-1
This app contains no stacks

[iliapolo]

Debug logs show Unable to determine the default AWS account (ProcessCredentialsProviderFailure): Profile role2 did not include credential process

[iliapolo]

Hmm. Im getting:

aws sts get-caller-identity --profile role2                                                                 

An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::ACCOUNT_B:assumed-role/TestIssue32195/botocore-session-1732089642 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::ACCOUNT_B:role/TestIssue32195-2

@Joe-Zer0 Can you confirm you aren't seeing this error from sts? Can you share how Role1 allows Role2 to assume it?

[iliapolo]

Ok I managed to reproduce. I was just missing the right policies for Role1 to be able to assume Role2. Investigating what happened.

 ❯ npx aws-cdk@2.167.2 synth --profile role2                                                                                                               [10:12:08]
CDK_DEFAULT_ACCOUNT: undefined
CDK_DEFAULT_REGION: us-east-1

❯ npx aws-cdk@2.166.0 synth --profile role2                                                                                                               [10:12:26]
CDK_DEFAULT_ACCOUNT: ACCOUNT_B
CDK_DEFAULT_REGION: us-east-1
This app contains no stacks

[rix0rrr]

We have tracked the issue to the JavaScript AWS SDKv3. Doubly-chained role assumption does not seem to work there.

[iliapolo]

aws/aws-sdk-js-v3#6681

[zshzbh]

For JS SDK issue - it could be resolved by updating the package version to v3.651.1 or higher

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.