sts assume-role doesn't work with non-admin permissions

[rantoniuk]

Scenario:

• cross-IAM account environment
• two roles on the target IAM account: Admin role and ReadOnly role
• ~/.aws/config profiles and ~/.aws/credentials configured, MFA also

Now:

• assuming roles works fine in AWS console
• assuming role does work fine in CLI for the Admin role
• assuming role does NOT work in CLI for the ViewOnly role

Details:

This works:

$ AWS_PROFILE=ADMINROLE aws sts assume-role --role-arn arn:aws:iam::XXX:role/AdminRole --role-session-name "SESSION" --query 'Credentials'

The exact same command for ReadOnly role doesn't work:

 AWS_PROFILE=VIEWONLY aws sts assume-role --role-arn arn:aws:iam::XXX:role/ViewOnly --role-session-name "SESSION" --query 'Credentials'

An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::XXX:assumed-role/ViewOnly/botocore-session-1612318233 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::XXX:role/ViewOnly

It looks like the role assumption went through but for some reason the CLI tries to re-assume from the botocore session to the actual session, since this works fine for the role with Full permissions, do I need specific IAM permissions for STS, but as the documentation states it should not. It also works in AWS Console UI...
This looks awfully similar to #2279.

Edit:
Adding AdministratorAccess role to the target account makes it work... definitely it looks like a permission issue that is not mentioned anywhere I looked.

SDK version number

aws-cli/2.1.27 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off

[rantoniuk]

I found a workaround, but I still think it's a weird behavior/requirement for the CLI, since this works without this in the Console.
What I needed to do is to edit the target account role and include the following trust relationship in IAM:

{
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::ACCOUNT_ID:role/ViewOnly"
      },
      "Action": "sts:AssumeRole"
    }

where ACCOUNT_ID is the exact same account number of the account I am in (so the target role IAM account).

[stobrien89]

Hi @warden,

Thanks for writing in! I definitely agree that the initial configuration of assume-role can be a little tricky, but since this is a cross-account MFA scenario, you are required to configure the trust policies as you did in your previous comment, per the AssumeRole documentation.

As for why it worked in the console and not the CLI— I think it's possible that enabling cross-account access may automatically allow for you to assume roles of connected accounts, but it's hard to say whether or not that is the case. Please let me know if you need any more assistance on this!

[rantoniuk]

@stobrien89 I have seen the page you referenced indeed but I don't see anything that would help in this (quite standard for cross-IAM approach I'd say) use case. I think I wasn't clear enough with the IAM trust relationship, so let me rephrase:

• I had the IAM trust relationship configured correctly for MFA with the policy below:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::ROOT_ACCOUNT_ID:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "Bool": {
          "aws:MultiFactorAuthPresent": "true"
        }
      }
    }
  ]
}

This worked fine on AWS Console to allow me to switch from a user in the root IAM account to the ViewOnly role on the target account. However, it did not work with aws-cli.

It seems that aws-cli does a double assume-role call (see detailed error message above) and to get it worked I needed to add an additional trust relationship mentioned above which I believe is quite weird as the initial role assumption worked, as stated by the error message.

[rantoniuk]

Notice that in the added Trust Relationship clause also I needed to remove MFA condition, with it in place it also does not work. So it seems that behind the scenes:

• cli does another assume-role (first problem)
• and does not pass correctly the MFA token.

I think this is a bug, especially that as I stated above, with admin permissions on the role it all works fine without adding the additional trust relationship.

[stobrien89]

Hi @warden,

My apologies for the misunderstanding on your initial post. I do agree that this could potentially be a bug, but I think there may be an explanation for why the double assume-role call is happening. Would you be able to share the contents of your config file (just the affected profiles)? Please obscure any sensitive information. Thanks!

[rantoniuk]

Sure, there you go:

[default]
region = eu-west-1
output = json

[profile AWS-RO]
region = eu-west-1
account = arn:aws:iam::XXX:user/user
role_arn = arn:aws:iam::YYY:role/ViewOnly
mfa_serial = arn:aws:iam::XXX:mfa/user
source_profile = default

[stobrien89]

Hi @warden,

Thank you for the additional information. Is your "admin" profile configured in a similar fashion? We could also get a better handle on what's happening by taking a look at the debug logs.

From what I can see, the two assume-role calls are happening because of the way the AWS-RO profile is configured— the AWS-RO profile implicitly assumes the ViewOnly role and then sts assume-role is explicitly called on the same role. So essentially you could use just the profile configuration without making the explicit sts assume-role call, or just make the explicit call, but doing both is what appears to be causing the odd behavior.

[rantoniuk]

Yes, the only difference for the Admin profile is that the assumed role is having AdministratorAccess role.

From what I can see, the two assume-role calls are happening because of the way the AWS-RO profile is configured— the AWS-RO profile implicitly assumes the ViewOnly role and then sts assume-role is explicitly called on the same role. So essentially you could use just the profile configuration without making the explicit sts assume-role call, or just make the explicit call, but doing both is what appears to be causing the odd behavior.

Indeed, that is the case and your idea is good, but... it is not supported in the workflow with YubiKey ;-)
So, here is what I am doing:

otp=$(ykman oath code $MFA_ARN -s)
credentials=$(aws sts assume-role --role-arn $ASSUMED_ROLE --role-session-name "SESSION" --query 'Credentials' --serial-number $MFA_ARN --token-code $otp)

If I would just use aws whatever --profile XXX - then it would ask for OTP code and there is no way to feed it from external program.

What is even more frustrating, is that I need to have the $ASSUMED_ROLE and $MFA_ARN things hardcoded in the role assumption script as I cannot get them easily from the config (this is unrelated/somehow related to this bug, but just for you to have the full picture).

The script currently looks like this:

unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN
unset AWS_PROFILE

MFA_ARN="XXXX"
ASSUMED_ROLE="YYYY"

otp=$(ykman oath code $MFA_ARN -s)
credentials=$(aws sts assume-role --role-arn $ASSUMED_ROLE --role-session-name "SESSION" --query 'Credentials' --serial-number $MFA_ARN --token-code $otp)
export AWS_ACCESS_KEY_ID=$(echo $credentials | jq --raw-output '.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(echo $credentials | jq --raw-output '.SecretAccessKey')
export AWS_SESSION_TOKEN=$(echo $credentials | jq --raw-output '.SessionToken')
export AWS_PROFILE=XXX

Now, to get rid of the hardcoded MFA_ARN and ASSUMED_ROLE I could use aws sts get-caller-identity, but... guess what:

• aws sts get-caller-identity does not display MFA_ARN since it is not in the default profile
• aws --profile X sts get-caller-identity will ask you for OTP code because MFA is defined in the profile...

and we're stuck in an endless loop ;-)

[stobrien89]

Hi @warden,

Thanks again for the additional information. I was going to ask more about your workflow, but this definitely clears things up!

I realize your current workflow is far from ideal, but given the current CLI functionality and lack of U2F support, I'm not seeing any other ways we could optimize this. I'll definitely run this by the development team on Monday to see what their thoughts are.

[stobrien89]

Hi @warden,

Thank you for your patience while I reviewed with the team. They couldn't find any aspects of your workflow that could be optimized without U2F support— We do have an open feature request for this, but it has been open for quite some time, so it's anyone's guess as to when/if this would be implemented.

Closing for now, but please let me know if you need any additional support!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.