[core] v1.60.0 no longer accepts --profile flag #9937

kyler-hyuna · 2020-08-24T10:44:53Z

I manage multiple AWS accounts and for each, I have a profile configured. In versions <1.60.0 of cdk, I could specify a profile using the --profile <profile-name>. For example if I wanted to use the diff command with the work profile, I would type:

yarn cdk --profile work diff

I am no longer able to do this on version 1.60.0.

Reproduction Steps

Create a stack using v1.59.0 and deploy it
Update your version to v1.60.0 and attempt to run diff or deploy command on the same stack

What did you expect to happen?

Upgrading to v1.60.0 would not affect passing credentials.

What actually happened?

I now receive the error:

Need to perform AWS calls for account <number>, but no credentials have been configured.

Environment

CLI Version: 1.60.0
Framework Version: 1.60.0
Node.js Version: v10.22.0
OS : mac-os
Language (Version): TypeScript

Other

As a result of this, my team is stuck on v1.59.0

This is 🐛 Bug Report

The text was updated successfully, but these errors were encountered:

rix0rrr · 2020-08-24T11:56:19Z

This bug report doesn't trivially reproduce.

$ npx cdk@1.60.0 --profile $SOME_PROFILE_NAME deploy

Works for me. What is special about your profile? What authentication methods does it use? Which file did you configure it in?

Only differences I can see that would affect this are:

v1.59.0...v1.60.0#diff-32e1d6d729ef6be343dc5494aa5d2e65
v1.59.0...v1.60.0#diff-69984e5eb59d539989dd8dd4e0f3f0cf

And there's not much there.

kyler-hyuna · 2020-08-24T12:08:03Z

Nothing too special. At work we use cross-account roles, In my credentials file I have a master block:

[master]
aws_access_key_id = xx
aws_secret_access_key = xxx

and then for a specific aws account profile such as work it'd reference it like this:

[work]
role_arn = arn:aws:iam::xxxx:role
source_profile = master

kyler-hyuna · 2020-08-24T12:21:28Z

Here's a super simple reproduction I just encountered and made for you https://github.com/kyler-hyuna/cdk-test.

master branch is 1.59.0.

Reproduce steps:

On master, after npm install/yarn run the local binary to deploy. yarn cdk --profile work deploy / npm run cdk -- --profile work deploy
Switch to 1.60.0 branch and run npm install/yarn
Use the local binary again to diff or deploy yarn cdk --profile work deploy / npm run cdk -- --profile work diff

rix0rrr · 2020-08-24T12:45:56Z

Thanks for putting together the repro. Still doesn't reproduce on my machine:

$ yarn cdk --profile huijbers-admin-role diff
Stack CdkTestStack
There were no differences

I guess try pasting the output of running it with -v?

EDIT: Aha it might be the cross-accountness...

kyler-hyuna · 2020-08-24T12:49:19Z

Finally, 🙌 Any idea what might've caused it? The cross-account role works as expected with the aws-cli

rix0rrr · 2020-08-24T12:50:56Z

It is not the cross-accountness. Tried to reproduce that as well by assuming into a cross-account role and it still works fine for me.

rix0rrr · 2020-08-24T12:51:32Z

There is something else different in your setup than in mine that you haven't told me about.

rix0rrr · 2020-08-24T12:53:16Z

Oh. Got it.

It's the absence of an ~/.aws/config file.

kyler-hyuna · 2020-08-24T12:54:54Z

This is my config:

[profile work]
region = us-east-1
output = json

rix0rrr · 2020-08-24T13:01:31Z

Oh so you DO have a ~/.aws/config file.

Even if I put the profile into my own config file--didn't have one before--and try to mess with the regions (maybe it's the AWS_STS_ENDPOINTS=regional change), still works for me.

rix0rrr · 2020-08-24T13:05:57Z

I'm to need you to do your own narrowing/diagnosis of the issue. Some tips:

Try running both old and new versions with -vvv and see what the differences are.
Try running both old and new versions with mitmproxy to see what kind of requests get done.
Try fiddling with you configuration files to see if you can identify the changes that make it break.

Otherwise, I'm out of remote debugging ideas...

kyler-hyuna · 2020-08-24T13:30:08Z

I used the -vvv flag on 1.59.0, and it correctly pulled my info from config and credentials:

Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
Resolving default credentials
Looking up default account ID from STS
[AWS sts 200 0.448s 0 retries] getCallerIdentity({})
Default account ID: 123
Setting "CDK_DEFAULT_ACCOUNT" environment variable to 123

On 1.60.0 this is the error:

Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
Resolving default credentials
Unable to determine the default AWS account: Error [ConfigError]: Missing region in config
    at Request.optInRegionalEndpoint (/Users/[omitted]/node_modules/aws-sdk/lib/services/sts.js:75:30

MisterBianco · 2020-08-24T16:52:31Z

I am having this issue as well. However, when I run my deploy command with -vvv I get:

Unable to determine the default AWS account: Error [ConfigError]: Missing region in config
    at Request.optInRegionalEndpoint (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/services/sts.js:75:30)
    at Request.callListeners (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at Request.runTo (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:408:15)
    at Request.send (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:372:10)
    at features.constructor.makeRequest (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/service.js:205:27)
    at features.constructor.svc.<computed> [as assumeRole] (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/service.js:677:23) {
  code: 'ConfigError',
  time: 2020-08-24T16:44:29.810Z
}

Which is interesting because my profile in the ~/.aws/config file is:

[profile shared]
role_arn = ROLE_REDACTED
source_profile = master
output = text
region = us-west-2
adfs_config.ssl_verification = False
adfs_config.role_arn = INHERITED_ROLE_REDACTED
adfs_config.session_duration = 3600
adfs_config.provider_id = urn:amazon:webservices
adfs_config.sspi = False
adfs_config.u2f_trigger_default = True

This issue only happens on 1.60.0

jogold · 2020-08-24T19:35:45Z

Are regional STS endpoints activated in the region where you are deploying (IAM console > Account settings)?

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html

kyler-hyuna · 2020-08-25T13:12:03Z

@jogold Yep

mrpackethead · 2020-08-25T20:02:20Z

I am experiencing the same issue.

[ec2-user@ip-10-x-x-x met_cert]$ cdk deploy --profile myprofile.
Need to perform AWS calls for account 6xxxxxx6, but no credentials have been configured.

I have a ~/.aws/config file but i do not have a default profile. ( as i operate across so many accounts it would be dangerous as it would be way to easy to forget where i was working... My profiles use a source profile that has mfa attached.

[profile loginprofile]
output = json
region = ap-southeast-2
mfa_serial = arn:aws:iam::09xxxxxxxx37:mfa/afrazer
source_profile = metservice::source-profile

[profile myprofile]
region = ap-southeast-2
role_arn = arn:aws:iam::28xxxxxxxxx79:role/TAR-ISOPS-Admin
source_profile = loginprofile

In the -vvv output below we see this..

Unable to determine the default AWS account: Error: EC2 Metadata roleName request returned error:

Not sure what to try next.

[ec2-user@ip-x-x-x-x met_cert]$ cdk deploy --profile myprofile -vvv CDK toolkit version: 1.60.0 (build 8e3f53a) Command line arguments: { _: [ 'deploy' ], profile: 'myprofile', v: 3, verbose: 3, 'ignore-errors': false, ignoreErrors: false, json: false, j: false, ec2creds: undefined, i: undefined, 'version-reporting': undefined, versionReporting: undefined, 'path-metadata': true, pathMetadata: true, 'asset-metadata': true, assetMetadata: true, 'role-arn': undefined, r: undefined, roleArn: undefined, staging: true, 'no-color': false, noColor: false, fail: false, 'build-exclude': [], E: [], buildExclude: [], ci: false, execute: true, force: false, f: false, parameters: [ {} ], 'previous-parameters': true, previousParameters: true, '$0': 'cdk' } cdk.json: { "app": "python3 app.py", "context": { "@aws-cdk/core:enableStackNameDuplicates": "true", "aws-cdk:enableDiffNoFail": "true" } } merged settings: { versionReporting: true, pathMetadata: true, output: 'cdk.out', app: 'python3 app.py', context: { '@aws-cdk/core:enableStackNameDuplicates': 'true', 'aws-cdk:enableDiffNoFail': 'true' }, tags: [], assetMetadata: true, profile: 'myprofile', toolkitBucket: {}, staging: true } Determining whether we're on an EC2 instance. Looks like EC2 instance. Toolkit stack: CDKToolkit Setting "CDK_DEFAULT_REGION" environment variable to ap-southeast-2 Resolving default credentials Unable to determine the default AWS account: Error: EC2 Metadata roleName request returned error at IncomingMessage.<anonymous> (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/util.js:895:34) at IncomingMessage.emit (events.js:327:22) at IncomingMessage.EventEmitter.emit (domain.js:483:12) at endReadableNT (_stream_readable.js:1220:12) at processTicksAndRejections (internal/process/task_queues.js:84:21) { statusCode: 404, retryable: false, time: 2020-08-25T19:53:52.593Z } context: { '@aws-cdk/core:enableStackNameDuplicates': 'true', 'aws-cdk:enableDiffNoFail': 'true', 'aws:cdk:enable-path-metadata': true, 'aws:cdk:enable-asset-metadata': true } outdir: cdk.out env: { CDK_DEFAULT_REGION: 'ap-southeast-2', CDK_CONTEXT_JSON: '{"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}', CDK_OUTDIR: 'cdk.out', CDK_CLI_ASM_VERSION: '5.0.0', CDK_CLI_VERSION: '1.60.0' } Reading existing template for stack met-cert. Need to perform AWS calls for account xxxxxx6, but no credentials have been configured. Error: Need to perform AWS calls for account xxxxxxxx6, but no credentials have been configured. at SdkProvider.obtainCredentials (/usr/lib/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:237:11) at processTicksAndRejections (internal/process/task_queues.js:97:5) at SdkProvider.forEnvironment (/usr/lib/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:123:19) at CloudFormationDeployments.prepareSdkFor (/usr/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:214:9) at CloudFormationDeployments.readCurrentTemplate (/usr/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:132:26) at CdkToolkit.deploy (/usr/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:158:33) at main (/usr/lib/node_modules/aws-cdk/bin/cdk.ts:268:16) at initCommandLine (/usr/lib/node_modules/aws-cdk/bin/cdk.ts:188:9) [ec2-user@ip-10-36-3-10 met_cert]$

mrpackethead · 2020-08-25T20:11:50Z

Hi...

It appears that there is a requirement for CDK to need to have a default profile even though i am not using it.

I added this

[profile default]
region = ap-southeast-2
output = json

And it seems to work. SOmething has changed in 1.60 that has changed this.

fennb · 2020-08-26T02:09:15Z

I am also having this issue.

I can confirm that the commit/precise change that causes this is: https://github.com/aws/aws-cdk/pull/9835/files#diff-69984e5eb59d539989dd8dd4e0f3f0cfR18

If you patch CDK to disable AWS_STS_REGIONAL_ENDPOINTS = 'regional', then things work as per before.

fennb · 2020-08-26T02:39:16Z

Some further research in diffing the -vvv output between patched (working) and unpatched (broken) versions yields the following:

$ diff output.working output.broken
105,108c105,118
< Looking up default account ID from STS
< [AWS sts 200 0.259s 0 retries] getCallerIdentity({})
< Default account ID: 887534651732
< Setting "CDK_DEFAULT_ACCOUNT" environment variable to [REDACTED]
---
> Unable to determine the default AWS account: { ConfigError: Missing region in config
>     at Request.optInRegionalEndpoint (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/services/sts.js:75:30)
>     at Request.callListeners (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
>     at Request.emit (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
>     at Request.emit (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:688:14)
>     at Request.transition (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:22:10)
>     at AcceptorStateMachine.runTo (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:14:12)
>     at Request.runTo (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:408:15)
>     at Request.send (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:372:10)
>     at features.constructor.makeRequest (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/service.js:205:27)
>     at features.constructor.svc.(anonymous function) [as assumeRole] (/usr/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/service.js:677:23)
>   message: 'Missing region in config',
>   code: 'ConfigError',
>   time: 2020-08-26T02:27:18.616Z }
133d142
<   CDK_DEFAULT_ACCOUNT: '[REDACTED]',
---snip---
186,203c195,198
< [AWS cloudformation 200 1.042s 0 retries] describeStacks({ StackName: 'REDACTED' })
< [AWS cloudformation 200 2.847s 0 retries] getTemplate({ StackName: 'REDACTED',
<   TemplateStage: 'Original' })
< This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
< Please confirm you intend to make the following modifications:
<
---snip---
> Need to perform AWS calls for account [REDACTED], but no credentials have been configured.
> Error: Need to perform AWS calls for account [REDACTED], but no credentials have been configured.
>     at SdkProvider.obtainCredentials (/usr/lib/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:237:11)
>     at process._tickCallback (internal/process/next_tick.js:68:7)

As a temporary workaround - if you define a region in your profile, it seems to work (and doesn't seem to matter what region it is), eg ~/.aws/config:

[profile development]
role_arn = arn:aws:iam::XXXX:role/OrganizationAccountAccessRole
source_profile = default
region = us-east-1 ; Required workaround for https://github.com/aws/aws-cdk/issues/9937

rix0rrr · 2020-08-26T07:22:29Z

Thanks for figuring this out y'all!

eladb · 2020-08-26T15:15:30Z

@rix0rrr should we flag this as p0?

d1str0 · 2020-08-27T00:06:23Z

@mrpackethead 's solution worked for me.

@fennb 's solution did not work for me. I had to create the default profile in my aws config. I deleted my default as I don't like having a default aws account to force me to specify which I want to use. Now that I've readded it, it works.

I also have a region specified. Did not try with default but without region.

fennb · 2020-08-27T00:54:57Z

@d1str0 Interesting - I should have mentioned that I do have a default profile, so it's possible you may need both, depending on setup.

As some other context, my profiles are for the purposes of cross-account access (not just for the purpose of changing roles), which may/may not impact things.

This works around a bug in the AWS SDK for JS that only surfaced when we switched to `AWS_STS_REGIONAL_ENDPOINTS=regional`, requiring a `[default]` profile with a region for all users. The bug was that the INI-file AssumeRole provider would ignore the region in the profile, and always fall back to the region in: * The profile specified using `$AWS_PROFILE` (which we don't use). * Otherwise the region in the `[default]` profile (which a user may or may not have). Traditionally it didn't really matter whether the STS client got a region or not because it would always connect to `us-east-1` no matter what, but when we switched to `AWS_STS_REGIONAL_ENDPOINTS=regional`, it became illegal to not have a region. Fix the upstream bug by basically replicating the important parts of `SharedIniFileCredentials` of the AWS SDK in our codebase and patching the bug. Reported upstreeam as aws/aws-sdk-js#3418 Fixes #9937 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

kyler-hyuna · 2020-08-28T15:13:12Z

Thanks @rix0rrr. Appreciate it 💯 . Looking forward to when it releases

kyler-hyuna added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Aug 24, 2020

github-actions bot added the @aws-cdk/core Related to core CDK functionality label Aug 24, 2020

github-actions bot assigned rix0rrr Aug 24, 2020

rix0rrr added p1 effort/small Small work item – less than a day of effort labels Aug 26, 2020

rix0rrr mentioned this issue Aug 27, 2020

SharedIniFileCredentials/AWS_STS_REGIONAL_ENDPOINTS=regional fails if there's no '[default]' profile aws/aws-sdk-js#3418

Closed

rix0rrr mentioned this issue Aug 28, 2020

fix(cli): AssumeRole profiles require a [default] profile #10032

Merged

mergify bot closed this as completed in #10032 Aug 28, 2020

yamatatsu mentioned this issue Sep 16, 2021

(core): WebIdentityCredentials does not work #16513

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core] v1.60.0 no longer accepts --profile flag #9937

[core] v1.60.0 no longer accepts --profile flag #9937

kyler-hyuna commented Aug 24, 2020

rix0rrr commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020 •

edited

Loading

kyler-hyuna commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020

rix0rrr commented Aug 24, 2020

rix0rrr commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020

rix0rrr commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020 •

edited

Loading

kyler-hyuna commented Aug 24, 2020

MisterBianco commented Aug 24, 2020

jogold commented Aug 24, 2020 •

edited

Loading

kyler-hyuna commented Aug 25, 2020

mrpackethead commented Aug 25, 2020

mrpackethead commented Aug 25, 2020

fennb commented Aug 26, 2020

fennb commented Aug 26, 2020 •

edited

Loading

rix0rrr commented Aug 26, 2020 •

edited

Loading

eladb commented Aug 26, 2020

d1str0 commented Aug 27, 2020 •

edited

Loading

fennb commented Aug 27, 2020 •

edited

Loading

kyler-hyuna commented Aug 28, 2020 •

edited

Loading

[core] v1.60.0 no longer accepts --profile flag #9937

[core] v1.60.0 no longer accepts --profile flag #9937

Comments

kyler-hyuna commented Aug 24, 2020

Reproduction Steps

What did you expect to happen?

What actually happened?

Environment

Other

rix0rrr commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020 • edited Loading

rix0rrr commented Aug 24, 2020 • edited Loading

kyler-hyuna commented Aug 24, 2020 • edited Loading

rix0rrr commented Aug 24, 2020

rix0rrr commented Aug 24, 2020

rix0rrr commented Aug 24, 2020

kyler-hyuna commented Aug 24, 2020

rix0rrr commented Aug 24, 2020 • edited Loading

rix0rrr commented Aug 24, 2020 • edited Loading

kyler-hyuna commented Aug 24, 2020

MisterBianco commented Aug 24, 2020

jogold commented Aug 24, 2020 • edited Loading

kyler-hyuna commented Aug 25, 2020

mrpackethead commented Aug 25, 2020

mrpackethead commented Aug 25, 2020

fennb commented Aug 26, 2020

fennb commented Aug 26, 2020 • edited Loading

rix0rrr commented Aug 26, 2020 • edited Loading

eladb commented Aug 26, 2020

d1str0 commented Aug 27, 2020 • edited Loading

fennb commented Aug 27, 2020 • edited Loading

kyler-hyuna commented Aug 28, 2020 • edited Loading

kyler-hyuna commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020 •

edited

Loading

kyler-hyuna commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020 •

edited

Loading

rix0rrr commented Aug 24, 2020 •

edited

Loading

jogold commented Aug 24, 2020 •

edited

Loading

fennb commented Aug 26, 2020 •

edited

Loading

rix0rrr commented Aug 26, 2020 •

edited

Loading

d1str0 commented Aug 27, 2020 •

edited

Loading

fennb commented Aug 27, 2020 •

edited

Loading

kyler-hyuna commented Aug 28, 2020 •

edited

Loading