Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: CDK does not seem to be compatible with config generated from aws configure sso #27265

Open
marcusirgens opened this issue Sep 24, 2023 · 19 comments
Assignees
Labels
bug This issue is a bug. cli Issues related to the CDK CLI p1 package/tools Related to AWS CDK Tools or CLI

Comments

@marcusirgens
Copy link

marcusirgens commented Sep 24, 2023

Describe the bug

When setting up a project from scratch with SSO credentials, CDK fails to authorize.

The steps I'm following are as follows:

  1. Configure my credentials using aws configure sso
  2. aws sso login --profile my-org-dev
  3. I set the account and region in bin/{stack}.ts
  4. AWS_PROFILE=my-org-dev cdk deploy

I receive the following error message:

$ AWS_PROFILE=my-org-dev cdk deploy

# Output:
# ✨  Synthesis time: 3.42s
#
#
# ❌ Deployment failed: Error: Need to perform AWS calls for account 123456789012, but no credentials have been configured

Expected Behavior

I expect CDK to be compatible with credentials generated with aws configure sso and aws sso login.

Current Behavior

When deploying, I get the following output:

$ AWS_PROFILE=my-org-dev cdk deploy -vvv

# Output:
# [18:26:42] CDK toolkit version: 2.97.0 (build d7cf3be)
# [18:26:42] Command line arguments: {
#   _: [ 'deploy' ],
#   v: 3,
#   verbose: 3,
#   lookups: true,
#   'ignore-errors': false,
#   ignoreErrors: false,
#   json: false,
#   j: false,
#   debug: false,
#   ec2creds: undefined,
#   i: undefined,
#   'version-reporting': undefined,
#   versionReporting: undefined,
#   'path-metadata': undefined,
#   pathMetadata: undefined,
#   'asset-metadata': undefined,
#   assetMetadata: undefined,
#   'role-arn': undefined,
#   r: undefined,
#   roleArn: undefined,
#   staging: true,
#   'no-color': false,
#   noColor: false,
#   ci: false,
#   all: false,
#   'build-exclude': [],
#   E: [],
#   buildExclude: [],
#   force: false,
#   f: false,
#   parameters: [ {} ],
#   'previous-parameters': true,
#   previousParameters: true,
#   logs: true,
#   concurrency: 1,
#   'asset-prebuild': true,
#   assetPrebuild: true,
#   '$0': '/Users/buggs/node/n/bin/cdk'
# }
# [18:26:42] cdk.json: {
#   "app": "npx ts-node --prefer-ts-exts bin/tmp.3f08_ww_ph.ts",
#   "watch": {
#     "include": [
#       "**"
#     ],
#     "exclude": [
#       "README.md",
#       "cdk*.json",
#       "**/*.d.ts",
#       "**/*.js",
#       "tsconfig.json",
#       "package*.json",
#       "yarn.lock",
#       "node_modules",
#       "test"
#     ]
#   },
#   "context": {
#     "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
#     "@aws-cdk/core:checkSecretUsage": true,
#     "@aws-cdk/core:target-partitions": [
#       "aws",
#       "aws-cn"
#     ],
#     "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
#     "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
#     "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
#     "@aws-cdk/aws-iam:minimizePolicies": true,
#     "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
#     "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
#     "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
#     "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
#     "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
#     "@aws-cdk/core:enablePartitionLiterals": true,
#     "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
#     "@aws-cdk/aws-iam:standardizedServicePrincipals": true,
#     "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
#     "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
#     "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
#     "@aws-cdk/aws-route53-patters:useCertificate": true,
#     "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
#     "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
#     "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
#     "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
#     "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
#     "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
#     "@aws-cdk/aws-redshift:columnId": true,
#     "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
#     "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
#     "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
#     "@aws-cdk/aws-kms:aliasNameRef": true,
#     "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
#     "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
#     "@aws-cdk/aws-efs:denyAnonymousAccess": true,
#     "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
#     "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
#     "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
#     "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
#     "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true
#   }
# }
# [18:26:42] merged settings: {
#   versionReporting: true,
#   assetMetadata: true,
#   pathMetadata: true,
#   output: 'cdk.out',
#   app: 'npx ts-node --prefer-ts-exts bin/tmp.3f08_ww_ph.ts',
#   watch: {
#     include: [ '**' ],
#     exclude: [
#       'README.md',
#       'cdk*.json',
#       '**/*.d.ts',
#       '**/*.js',
#       'tsconfig.json',
#       'package*.json',
#       'yarn.lock',
#       'node_modules',
#       'test'
#     ]
#   },
#   context: {
#     '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
#     '@aws-cdk/core:checkSecretUsage': true,
#     '@aws-cdk/core:target-partitions': [ 'aws', 'aws-cn' ],
#     '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
#     '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
#     '@aws-cdk/aws-ecs:arnFormatIncludesClusterName': true,
#     '@aws-cdk/aws-iam:minimizePolicies': true,
#     '@aws-cdk/core:validateSnapshotRemovalPolicy': true,
#     '@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
#     '@aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
#     '@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true,
#     '@aws-cdk/aws-apigateway:disableCloudWatchRole': true,
#     '@aws-cdk/core:enablePartitionLiterals': true,
#     '@aws-cdk/aws-events:eventsTargetQueueSameAccount': true,
#     '@aws-cdk/aws-iam:standardizedServicePrincipals': true,
#     '@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker': true,
#     '@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName': true,
#     '@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy': true,
#     '@aws-cdk/aws-route53-patters:useCertificate': true,
#     '@aws-cdk/customresources:installLatestAwsSdkDefault': false,
#     '@aws-cdk/aws-rds:databaseProxyUniqueResourceName': true,
#     '@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup': true,
#     '@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId': true,
#     '@aws-cdk/aws-ec2:launchTemplateDefaultUserData': true,
#     '@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments': true,
#     '@aws-cdk/aws-redshift:columnId': true,
#     '@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2': true,
#     '@aws-cdk/aws-ec2:restrictDefaultSecurityGroup': true,
#     '@aws-cdk/aws-apigateway:requestValidatorUniqueId': true,
#     '@aws-cdk/aws-kms:aliasNameRef': true,
#     '@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig': true,
#     '@aws-cdk/core:includePrefixInUniqueNameGeneration': true,
#     '@aws-cdk/aws-efs:denyAnonymousAccess': true,
#     '@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby': true,
#     '@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion': true,
#     '@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId': true,
#     '@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters': true,
#     '@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier': true
#   },
#   debug: false,
#   toolkitBucket: {},
#   staging: true,
#   bundlingStacks: [ '**' ],
#   lookups: true,
#   assetPrebuild: true
# }
# [18:26:42] [trace] SdkProvider#withAwsCliCompatibleDefaults()
# [18:26:42] Determining if we're on an EC2 instance.
# [18:26:42] Does not look like an EC2 instance.
# [18:26:42] Reading cached notices from /Users/buggs/.cdk/cache/notices.json
# [18:26:42] Toolkit stack: CDKToolkit
# [18:26:42] Setting "CDK_DEFAULT_REGION" environment variable to eu-north-1
# [18:26:42] [trace] SdkProvider#defaultAccount()
# [18:26:42] [trace]   SdkProvider#defaultCredentials()
# [18:26:42] Resolving default credentials
# [18:26:42] Unable to determine the default AWS account (ProcessCredentialsProviderFailure): Profile my-org-dev did not include credential process
# [18:26:42] context: {
#   '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
#   '@aws-cdk/core:checkSecretUsage': true,
#   '@aws-cdk/core:target-partitions': [ 'aws', 'aws-cn' ],
#   '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
#   '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
#   '@aws-cdk/aws-ecs:arnFormatIncludesClusterName': true,
#   '@aws-cdk/aws-iam:minimizePolicies': true,
#   '@aws-cdk/core:validateSnapshotRemovalPolicy': true,
#   '@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
#   '@aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
#   '@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true,
#   '@aws-cdk/aws-apigateway:disableCloudWatchRole': true,
#   '@aws-cdk/core:enablePartitionLiterals': true,
#   '@aws-cdk/aws-events:eventsTargetQueueSameAccount': true,
#   '@aws-cdk/aws-iam:standardizedServicePrincipals': true,
#   '@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker': true,
#   '@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName': true,
#   '@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy': true,
#   '@aws-cdk/aws-route53-patters:useCertificate': true,
#   '@aws-cdk/customresources:installLatestAwsSdkDefault': false,
#   '@aws-cdk/aws-rds:databaseProxyUniqueResourceName': true,
#   '@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup': true,
#   '@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId': true,
#   '@aws-cdk/aws-ec2:launchTemplateDefaultUserData': true,
#   '@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments': true,
#   '@aws-cdk/aws-redshift:columnId': true,
#   '@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2': true,
#   '@aws-cdk/aws-ec2:restrictDefaultSecurityGroup': true,
#   '@aws-cdk/aws-apigateway:requestValidatorUniqueId': true,
#   '@aws-cdk/aws-kms:aliasNameRef': true,
#   '@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig': true,
#   '@aws-cdk/core:includePrefixInUniqueNameGeneration': true,
#   '@aws-cdk/aws-efs:denyAnonymousAccess': true,
#   '@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby': true,
#   '@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion': true,
#   '@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId': true,
#   '@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters': true,
#   '@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier': true,
#   'aws:cdk:enable-path-metadata': true,
#   'aws:cdk:enable-asset-metadata': true,
#   'aws:cdk:version-reporting': true,
#   'aws:cdk:bundling-stacks': [ '**' ]
# }
# [18:26:42] outdir: cdk.out
# [18:26:42] env: {
#   CDK_DEFAULT_REGION: 'eu-north-1',
#   CDK_OUTDIR: 'cdk.out',
#   CDK_CLI_ASM_VERSION: '34.0.0',
#   CDK_CLI_VERSION: '2.97.0'
# }
# 
# ✨  Synthesis time: 2.13s
# 
# [18:26:44] Checking for previously published assets
# [18:26:44] [trace] SdkProvider#resolveEnvironment()
# [18:26:44] [trace] SdkProvider#baseCredentialsPartition()
# [18:26:44] [trace]   SdkProvider#resolveEnvironment()
# [18:26:44] [trace]   SdkProvider#obtainBaseCredentials()
# [18:26:44] [trace]     SdkProvider#defaultAccount()
# [18:26:44] [trace] SdkProvider#forEnvironment()
# [18:26:44] [trace]   SdkProvider#resolveEnvironment()
# [18:26:44] [trace]   SdkProvider#obtainBaseCredentials()
# [18:26:44] [trace]     SdkProvider#defaultAccount()
# 
#  ❌ Deployment failed: Error: Need to perform AWS calls for account 123456789012, but no credentials have been configured
#     at SdkProvider.forEnvironment (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:367:13075)
#     at async Deployments.cachedSdkForEnvironment (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:12383)
#     at async Deployments.prepareSdkFor (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:7949)
#     at async Deployments.isSingleAssetPublished (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:11602)
#     at async /Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:165302
# [18:26:44] Reading cached notices from /Users/buggs/.cdk/cache/notices.json
# 
# Need to perform AWS calls for account 123456789012, but no credentials have been configured
# [18:26:44] Error: Need to perform AWS calls for account 123456789012, but no credentials have been configured
#     at SdkProvider.forEnvironment (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:367:13075)
#     at async Deployments.cachedSdkForEnvironment (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:12383)
#     at async Deployments.prepareSdkFor (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:7949)
#     at async Deployments.isSingleAssetPublished (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:11602)
#     at async /Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:457:165302

Getting my current session details:

$ AWS_PROFILE=cdk-error-demo aws sts get-caller-identity --output yaml

# Output:
# Account: '123456789012'
# Arn: arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_MyRoleNameWith_alongrandomsuffix/buggs@iterate.no
# UserId: ANACCESSKEYIDORSECRETNEVERTHELESSREDACTED:buggs@iterate.no

This is what my AWS_CONFIG_FILE looks like:

[profile my-org-dev]
sso_session = Admin in My Org
sso_account_id = 123456789012
sso_role_name = AdministratorAccess
region = eu-north-1
[sso-session 'Admin in My Org']
sso_start_url = https://acme.awsapps.com/start#
sso_region = eu-central-1
sso_registration_scopes = sso:account:access

Reproduction Steps

  1. I install aws-cdk and check the version:

    $ cdk --version
    
    # Output:
    # 2.97.0 (build d7cf3be)
  2. I back up my AWS_CONFIG_FILE and delete the existing file:

    if test -n "$AWS_CONFIG_FILE" -a -f "$AWS_CONFIG_FILE"
      cp "$AWS_CONFIG_FILE" "$AWS_CONFIG_FILE.$(date +%s).bak"; and rm "$AWS_CONFIG_FILE"
    end
    if test -f "$HOME/.aws/config"
      cp "$HOME/.aws/config" "$HOME/.aws/config.$(date +%s).bak"; and rm "$HOME/.aws/config"
    end
    test ! -f "$HOME/.aws/config" -a ! -f "$AWS_CONFIG_FILE"; or echo "Files still present, abort!";
  3. I go to the SSO start URL for my organization, https://acme.awsapps.com/start. Under the account I want to use, I click Command line or programmatic access, where I read the following instructions:

    configure the AWS CLI to retrieve them automatically using the aws configure sso command

  4. In my shell, I configure the AWS CLI using aws configure sso:

    $ aws configure sso --profile my-org-dev
    
    # Output (interactive):
    # SSO session name (Recommended): Admin in My Org
    # SSO start URL [None]: https://acme.awsapps.com/start#
    # SSO region [None]: eu-central-1
    # SSO registration scopes [sso:account:access]:
    # Attempting to automatically open the SSO authorization page in your default browser.
    # If the browser does not open or you wish to use a different device to authorize this request, open the following URL:
    #
    # https://device.sso.eu-central-1.amazonaws.com/
    #
    # Then enter the code:
    #
    # BRDX-HVMG
    # There are 5 AWS accounts available to you.
    # Using the account ID 123412341234
    # The only role available to you is: AdministratorAccess
    # Using the role name "AdministratorAccess"
    # CLI default client Region [None]: eu-north-1
    # CLI default output format [None]:
    #
    # To use this profile, specify the profile name using --profile, as shown:
    #
    # aws s3 ls --profile my-org-dev
  5. In my shell, I verify that I have a valid session:

    $ AWS_PROFILE=my-org-dev aws sts get-caller-identity --output yaml
    
    # Output:
    # Account: '048445190004'
    # Arn: arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_alongrandomsuffix/buggs@iterate.no
    # UserId: ANACCESSKEYIDORSECRETNEVERTHELESSREDACTED:buggs@iterate.no
    
    $ AWS_PROFILE=my-org-dev aws cloudformation list-stacks --query StackSummaries
    
    # (This is successful, this role has administrative privileges).
  6. I initialize a CDK project in a new directory:

    $ AWS_PROFILE=my-org-dev cd (mktemp -d); and cdk init app --language=typescript
    
    # Output (partial):
    # # ...
    # ✅ All done!
  7. I attempt to deploy the stack, which fails because CDK cannot determine which account to use:

    $ AWS_PROFILE=my-org-dev cdk deploy
    
    # Output:
    # ✨  Synthesis time: 2.38s
    # 
    # 
    #  ❌ Deployment failed: Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
  8. I follow the instructions in bin/{the application name}.ts, as indicated in the output from the failing cdk deploy, and add:

    env: { account: '123456789012', region: 'eu-north-1' },
  9. I attempt to deploy the stack, which fails because CDK cannot find the credentials:

    $ AWS_PROFILE=my-org-dev cdk deploy
    
    # Output:
    # ✨  Synthesis time: 2.2s
    # 
    # 
    #  ❌ Deployment failed: Error: Need to perform AWS calls for account 048445190004, but no credentials have been configured

If I remove the sso-session block and move some settings up to the my-org-dev profile, I am able to deploy:

  1. I make the following changes to my $AWS_CONFIG_FILE:
    --- /Users/buggs/.aws/config.old	2023-09-24 18:15:43
    +++ /Users/buggs/.aws/config	2023-09-24 18:15:57
    @@ -1,9 +1,6 @@
     [profile my-org-dev]
    -sso_session = Admin in My Org
     sso_account_id = 123412341234
     sso_role_name = AdministratorAccess
     region = eu-north-1
    -[sso-session 'Admin in My Org']
     sso_start_url = https://acme.awsapps.com/start#
     sso_region = eu-central-1
    -sso_registration_scopes = sso:account:access
  2. I attempt to deploy the stack, which now succeeds:
    AWS_PROFILE=my-org-dev cdk deploy
    
    # Output (partial):
    # (...)
    # ✅  Tmp3F08WwPhStack
    # (...)
    # ✨  Total time: 15.91s

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.97.0 (build d7cf3be)

Framework Version

No response

Node.js Version

18.14.0

OS

macOS 13.4 (22F66)

Language

Typescript

Language Version

~5.2.2

Other information

Similar issues

This seems to be somewhat related to a couple of other issues.

@marcusirgens marcusirgens added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 24, 2023
@github-actions github-actions bot added the package/tools Related to AWS CDK Tools or CLI label Sep 24, 2023
@marcusirgens marcusirgens changed the title cli: CDK is not compatible with confg generated from aws configure sso cli: CDK does not seem to be compatible with config generated from aws configure sso Sep 24, 2023
@HannesOberreiter
Copy link

Just want to say thank you, after a long struggle with why --profile XXXX did not work I found your issue just in time. Removing the sso-session block and sso_session params from my configs did resolve it and now I can deploy my multi account pipeline.

Further notes on the topic which may be helpful, I do not have any [default] block in my .aws/config file and no blocks at all in my .aws/credentials file.

Additionally I found out that if I use the flag --profile with npm run cdk synth --profile xxx it does not work at all (it may get eaten by npm ?). If you need to use npm run you need to set the above mentioned AWS_PROFILE otherwise if you use a global installed cdk you can use the --profile flag.

@indrora
Copy link
Contributor

indrora commented Sep 25, 2023

There's a handful of possible issues here.

  • Is the account that the CDK is trying to use (that is, the AWS Account that you've SSO logged in as) bootstrapped?
  • Is the account that you have used for SSO able to AssumeRole against the role that the CDK has created to run deployments?

@indrora indrora added p2 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. cli Issues related to the CDK CLI and removed needs-triage This issue or PR still needs to be triaged. labels Sep 25, 2023
@peterwoodworth
Copy link
Contributor

@indrora were you able to reproduce this?

@marcusirgens
Copy link
Author

marcusirgens commented Sep 25, 2023

Is the account that the CDK is trying to use (that is, the AWS Account that you've SSO logged in as) bootstrapped?

@indrora Do you mean with CDK?

$ AWS_PROFILE=my-org-dev cdk bootstrap
# Output:
#  ⏳  Bootstrapping environment aws://123456789012/eu-north-1...
#  ❌  Environment aws://123456789012/eu-north-1 failed bootstrapping: Error: Need to perform AWS calls for account 123456789012, but no credentials have been configured
#     at SdkProvider.forEnvironment (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:367:13075)
#     at async _BootstrapStack.lookup (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:454:20535)
#     at async Bootstrapper.modernBootstrap (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:455:1084)
#     at async /Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:459:2104
#     at async Promise.all (index 0)
#     at async CdkToolkit.bootstrap (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:459:1949)
#     at async exec4 (/Users/buggs/node/n/lib/node_modules/aws-cdk/lib/index.js:512:53102)
# 
# Need to perform AWS calls for account 123456789012, but no credentials have been configured

Without sso_session in $AWS_CONFIG_FILE (see #27265 (comment)):

$ AWS_PROFILE=my-org-dev cdk bootstrap

# Output:
# ⏳  Bootstrapping environment aws://123456789012/eu-north-1...
#Trusted accounts for deployment: (none)
#Trusted accounts for lookup: (none)
#Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
#
# ✨ hotswap deployment skipped - no changes were detected (use --force to override)
#
# ✅  Environment aws://123456789012/eu-north-1 bootstrapped (no changes).

Is the account that you have used for SSO able to AssumeRole against the role that the CDK has created to run deployments?

Yes, as far as I can tell (the bottom part is a script that assumes all roles with CDK in their names):

$ AWS_PROFILE=my-org-dev aws sts get-caller-identity --output json

# Output:
# {
#     "UserId": "ARARANDOMREDACTED:buggs@iterate.no",
#     "Account": "123456789012",
#     "Arn": "arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_randomsuffix/buggs@iterate.no"
# }

# This script lists all roles with "cdk" in the name, and runs aws sts assume-role for those, and prints the assumed role.
$ AWS_PROFILE=my-org-dev aws iam list-roles --output json | \
  jq '.Roles[] | .Arn | select(. | ascii_downcase | contains("cdk")) | .' --raw-output0 | \
  AWS_PROFILE=my-org-dev xargs -0n1 -I'{}' aws sts assume-role --role-arn '{}' --role-session-name 'demo-github' --output json | \
  jq '.AssumedRoleUser.Arn'
  
# Output:
# # An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_9291ebc2c934fe48/buggs@iterate.no is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456789012:role/cdk-hnb659fds-cfn-exec-role-123456789012-eu-central-1
# 
# An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_9291ebc2c934fe48/buggs@iterate.no is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456789012:role/cdk-hnb659fds-cfn-exec-role-123456789012-eu-north-1
# 
# An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_9291ebc2c934fe48/buggs@iterate.no is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456789012:role/cdk-hnb659fds-cfn-exec-role-123456789012-us-east-1
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-deploy-role-123456789012-eu-central-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-deploy-role-123456789012-eu-north-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-deploy-role-123456789012-us-east-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-file-publishing-role-123456789012-eu-central-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-file-publishing-role-123456789012-eu-north-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-file-publishing-role-123456789012-us-east-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-image-publishing-role-123456789012-eu-central-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-image-publishing-role-123456789012-eu-north-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-image-publishing-role-123456789012-us-east-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-lookup-role-123456789012-eu-central-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-lookup-role-123456789012-eu-north-1/demo-github"
# "arn:aws:sts::123456789012:assumed-role/cdk-hnb659fds-lookup-role-123456789012-us-east-1/demo-github"

The three roles I cannot assume have the following trust policies:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

@marcusirgens
Copy link
Author

To be 100 % clear:

With the following config, I can perform AWS_PROFILE=my-org-dev aws sso login with no issues and I can assume roles, but I cannot use CDK.

[profile my-org-dev]
sso_session = Admin in My Org
sso_account_id = <REDACTED>
sso_role_name = AdministratorAccess
region = eu-north-1

[sso-session 'Admin in My Org']
sso_start_url = https://<REDACTED>.awsapps.com/start#
sso_region = eu-central-1
sso_registration_scopes = sso:account:access

This is my entire AWS_CONFIG_FILE, there are no other sections in it.

I've also removed files from ~/.aws/cli/cache and ~/.aws/sso/cache to double check that I'm not using any "hidden" credentials, which successfully gave me the following output:

Error loading SSO Token: Token for Admin in My Org does not exist

When removing the sso_session parameter, and moving sso_start_url and sso_region up to profile, making this my entire AWS_CONFIG_FILE, I can use CDK normally:

[profile my-org-dev]
# sso_session = Admin in My Org
sso_account_id = <REDACTED>
sso_role_name = AdministratorAccess
region = eu-north-1
sso_start_url = https://<REDACTED>.awsapps.com/start#
sso_region = eu-central-1

# [sso-session 'Admin in My Org']
# sso_registration_scopes = sso:account:access

The output of aws sts get-caller-identity --output text --query Arn is byte-for-byte the same when using either of these configurations. The problem is that the first one is the one generated by aws configure sso.

@indrora
Copy link
Contributor

indrora commented Sep 25, 2023

@peterwoodworth I don't have an account set up with SSO right now, so I wasn't able to do a proper repro.

Some notes, however:

Discussion #21316 might be of interest: If you're behind an SSL crusher like ZScaler so that Corporate IT can do SSL inspection, the AWS client the CDK uses just wanders into the sun because it doesn't support the custom CA bundle. I don't know if that's the case here, but it's a plausible lead.

I have found numerous issues with AWS SSO and CDK discussed by other customers(1) and in our own ticket tracker (#23520), and found one customer who found a... While not great workaround, a workaround(2). There is a known suggestion on re:post to just do it the hard way(3) for some applications.

PR #19454 was supposed to fix this need back in 2022, but I don't know how well that's worked in the long run. There was also work in #23702 to handle an edge case in SSO profiles.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Sep 26, 2023
@jameswyse
Copy link

I had this issue and it turned out I had some old entries in my ~/.aws/config that looked like

[name]
region = ap-southeast-2

Removing all of those (and leaving the newer [profile name] and [sso-session name] entries did the trick

@bissli82
Copy link

bissli82 commented Oct 1, 2023

same issues for me, please fix.
Workaround indeed to change profile name to other than default

@marcusirgens
Copy link
Author

same issues for me, please fix.
Workaround indeed to change profile name to other than default

Clarifying that this is not the issue I've raised here, nor the fix for it - this issue seems to be that CDK does not understand the SSO session field.

The "workaround" seems to be to not enter anything when prompted for

SSO session name (Recommended): 

while running aws configure sso, which sets up SSO in "legacy" mode, which works fine.

@Marakai
Copy link

Marakai commented Oct 25, 2023

Not one of the solutions proposed work for me when trying to quickly deploy CDK stacks into multiple accounts. Whichever one I configure and use, will cause the other to error out with the "need to perform for A, but current creds are for B".

The unacceptable work-around involves a rigmarole of cleaning/purging of credential and auth caches. Then switch to the other, deploy, and of course get the same error when trying to use the other account, except this time with "need to perform for B, but current creds are for A".

Somewhat ironically, this used to work without issues when using different SSO providers (from Okta to Jumpcloud).

@astef
Copy link

astef commented Nov 15, 2023

There's a chance this old issue is related: #24744

@DanielLaberge
Copy link

To add another data point: I'm also using aws sso login, and I will randomly get that error with CDK commands every now and then, but I can immediately run the same command again (cdk deploy) and it will work, so my SSO session was still valid.

Sounds like a bug in the CLI that sometimes randomly fails to return SSO credentials to CDK.

@benkehoe
Copy link

benkehoe commented Jan 26, 2024

Sounds like a bug in the CLI

The CLI does not provide credentials to the CDK, not directly. Running aws sso login gets an Identity Center token and caches it on the file system. The AWS SDKs load that token and exchange it for IAM credentials. If the CLI works after using aws sso login, e.g. aws sts get-caller-identity, the CLI has correctly cached the token. Any further issues would be in the CDK or its dependencies.

The issue is probably somewhere in the JavaScript SDK. As I understand it, different parts of the CDK use v2 and v3 of the JS SDK, and Identity Center support in v2 was very late in coming, so I wouldn't be surprised if it also has issues with the change to session-based profiles and refresh tokens. Potentially this: aws/aws-sdk-js#4441

@DanielLaberge
Copy link

What can I do to help diagnose the source of the bug? Is there a debug log I can turn on? I could post the logs when the problem reproduces itself.

@dvrax
Copy link

dvrax commented Mar 19, 2024

After much messing about, I found out that I could get it to work by removing any spaces in my sso-session name and removing the single quotes around the sso-session name (not certain if this was necessary or not)

So

[profile my-org-dev]
sso_session = Admin in My Org
...
[sso-session 'Admin in My Org']
...

Could be changed to:

[profile my-org-dev]
sso_session = Admin-in-My-Org
...
[sso-session Admin-in-My-Org]
...

Seems like other config readers have run into this too (ruby, go). If I'm following the code right here the correct fix would need to be in https://github.com/smithy-lang/smithy-typescript. Though it seems like the aws cli is actually generating configuration that violates the spec. It should probably prevent using sso session names with spaces in them.

You can use letters, numbers, hyphens ( - ), and underscores ( _ ), but no spaces.

@ChrisLane
Copy link

I'm having this issue even without spaces or quotes in my session name.

[profile test]
sso_session = myorg
sso_account_name = Test
sso_account_id = <redacted>
sso_role_name = Admin
region = eu-west-1
output = json

[sso-session myorg]
sso_region = us-east-1
sso_start_url = https://<redacted>.awsapps.com/start

@MurmursDev
Copy link

same issues for me, please fix.
Workaround indeed to change profile name to other than default

Clarifying that this is not the issue I've raised here, nor the fix for it - this issue seems to be that CDK does not understand the SSO session field.

The "workaround" seems to be to not enter anything when prompted for

SSO session name (Recommended): 

while running aws configure sso, which sets up SSO in "legacy" mode, which works fine.

This works to me

@codenutt
Copy link

I was seeing something similar using bootstrap and the trick for me was to have an "SSO session name" without spaces. Was able to enter a value for it though

@otaviomacedo otaviomacedo self-assigned this Oct 2, 2024
@TheRealAmazonKendra
Copy link
Contributor

I suspect this should have been fixed when we upgraded our AWS SDK to v2.1691.0 because of aws/aws-sdk-js#4443. Can you please upgrade your cdk cli to our most recent version and see if this is still impacting you?

If it is, you should see a resolution when #31702 is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. cli Issues related to the CDK CLI p1 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

No branches or pull requests