cli: --profile flag has weird behavior on diff

[wtibbitts]

Describe the bug

I have a CDK stack that creates an EKS cluster in typescript. It uses a build script to do the cdk synth and a deploy script to do the cdk deployment passing in --profile flag to specify the account and then, for the deploy script, gets the kubeconfig and runs some additional steps after. To adjust for the change that removed the default master role (which was used in the kubeconfig), I am now looking up the current user (using the sdk get-caller-identity) and adding them to the aws-auth config. This all works fine, however, I have noticed that when I run cdk diff --profile <profile> after the build script, I do not get the correct user. It instead gets my default profile user. If I set AWS_PROFILE before cdk diff instead of using the profile flag, I get the correct user.

What makes this more confusing is that the diff still seems to be generated against the correct account. I don't understand how part of the script would not use the credentials.

Expected Behavior

Using the --profile flag with cdk diff, should get the correct user.

Current Behavior

Cdk diff --profile <profile> seems to not set credentials for code run in the script.

Reproduction Steps

1. Generate a cdk script that uses current user information.
2. Build & deploy it to non default account using ---profile flag
3. Make some change
4. Build
5. Check diff using cdk diff --profile

Possible Solution

No response

Additional Information/Context

Log Output from build/deploy:

current user: {
'$metadata': {
httpStatusCode: 200,
requestId: 'ba49b24d-e8b0-45fa-ad0a-78a996fd8fe7',
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
UserId: 'AIDAI74##########K',
Account: '2733#######',
Arn: 'arn:aws:iam::2733#######:user/wes'
}

Diff output:

current user: {
'$metadata': {
httpStatusCode: 200,
requestId: '7c8a0cfd-d2ae-4a6b-baa7-5525ac9a4532',
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
UserId: 'AIDAJ#########',
Account: '557#######',
Arn: 'arn:aws:iam::5577#########:user/wes'
}

Stack testDummyInfrastructureStack
Resources
[] Custom::AWSCDK-EKS-KubernetesResource DummyCluster/Cluster/AwsAuth/manifest/Resource DummyClusterAwsAuthmanifest9215E465
└─ [] Manifest
└─ [~] .Fn::Join:
└─ @@ -29,6 +29,6 @@
[ ] "Arn"
[ ] ]
[ ] },
[-] "\",\"username\":\"system:node:{{EC2PrivateDNSName}}\",\"groups\":[\"system:bootstrappers\",\"system:nodes\"]}]","mapUsers":"[{\"userarn\":\"arn:aws:iam::5577#############:user/wes\",\"username\":\"arn:aws:iam::5577#############:user/wes\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::5577#############:user/mike\",\"username\":\"arn:aws:iam::5577#############:user/mike\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::5577#############:user/michael\",\"username\":\"arn:aws:iam::5577#############:user/michael\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::273363790070:user/wes\",\"username\":\"arn:aws:iam::273363790070:user/wes\",\"groups\":[\"system:masters\"]}]","mapAccounts":"[\"5577#############\"]"}}]"
[+] "\",\"username\":\"system:node:{{EC2PrivateDNSName}}\",\"groups\":[\"system:bootstrappers\",\"system:nodes\"]}]","mapUsers":"[{\"userarn\":\"arn:aws:iam::5577#############:user/wes\",\"username\":\"arn:aws:iam::5577#############:user/wes\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::5577#############:user/mike\",\"username\":\"arn:aws:iam::5577#############:user/mike\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::5577#############:user/michael\",\"username\":\"arn:aws:iam::5577#############:user/michael\",\"groups\":[\"system:masters\"]},{\"userarn\":\"arn:aws:iam::5577#############:user/wes\",\"username\":\"arn:aws:iam::5577#############:user/wes\",\"groups\":[\"system:masters\"]}]","mapAccounts":"[\"5577#############\"]"}}]"

CDK CLI Version

2.92.0

Framework Version

No response

Node.js Version

v18.7.1

OS

macOS ventura 13.4

Language

Typescript

Language Version

4.9.4

Other information

No response

[peterwoodworth]

Could you please provide specific reproduction code which we can copy+paste?

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[wtibbitts]

I'll work on getting a minimal project together for it.

[wtibbitts]

After setting up a simple project for this it became clear, that the difference is not tied to diff, but all CDK commands and revolves around how the AWS credentials profile is set. If the flag --profile is used, sdk code run in the script will still use default credentials. If AWS_PROFILE is used, sdk code will use the correct user.

Here is a simple project to replicate. The cluster is even unnecessary.

Just run cdk synth --profile <non-default profile> and compare with running AWS_PROFILE=<non-default profile> cdk synth. I would expect these to have the same user credentials for all code in run in the project.

[ozeebee]

I confirm this behavior.
The "--profile" command line option seems to be ignored whereas the "AWS_PROFILE" env variable works.
We have this problem using SSO credentials/profiles.

[juweeks]

related? #20956

[pahud]

#20956 only happens when both --profile and credential-process exist. Do you use both of them in your aws config?

[wtibbitts]

I do not set credential-process at all.

[TheRealAmazonKendra]

To further troubleshoot this we would need to know the contents of your ~/.aws/config and ~/.aws/credentials files. Can you please provide that? It's likely that you're using a setup that is not compatible with the limitations in the AWS SDK v2 and that this will be solved when #31702 is merged.