FR: support credential_source = Ec2InstanceMetadata in ~/.aws/config

[ericphanson]

I've an EC2 instance with a role (say role X), which is allowed to assume a role Y. I have a ~/.aws/config that looks like

[profile role-Y]
role_arn = arn:aws:iam::NUMBER:role/Y
credential_source = Ec2InstanceMetadata

I can run commands using the AWS CLI within the EC2 instance like

aws --profile role-Y s3 cp ...

that work fine (and correctly give access denied without the --profile flag). However, if I do

using AWS
role_Y_config = AWSConfig(profile="role-Y")
write(S3Path(...; config=role_Y_config), ...)

then I get access denied. The issue seems to be that AWSConfig does not detect the credentials in the profile in ~/.aws/config and instead falls back to the overall EC2 creds; in particular, I suspect if I had the creds in that ~/.aws/config directly instead of
credential_source = Ec2InstanceMetadata it might work.

[mattBrzezinski]

I think is would be a good feature to have. Looking at the documentation there are three parameters which credential_source can be set to:

• Environment – Specifies that the AWS CLI is to retrieve source credentials from environment variables.
• Ec2InstanceMetadata – Specifies that the AWS CLI is to use the IAM role attached to the EC2 instance profile to get source credentials.
• EcsContainer – Specifies that the AWS CLI is to use the IAM role attached to the ECS container as source credentials.

We would just need a mapping the values to their functions, and then if credential_source is set, parsing it and setting it to the renew field:

credential_source_mapping = Dict(
  "Environment" => env_var_credentials,
  "Ec2InstanceMetadata" => ec2_instance_credentials,
  "EcsContainer" => ecs_instance_credentials
)

[ericphanson]

Cool, thanks for taking a look! I think that's not quite enough here, because for me AWS.ecs_instance_credentials() returns the credentials associated to the role assigned to the EC2 instance ("role X"), but not the credentials associated to the role that I want to assume using the profile ("role Y").

In case that doesn't make sense, going back to the config,

[profile role-Y]
role_arn = arn:aws:iam::NUMBER:role/Y
credential_source = Ec2InstanceMetadata

here role-Y is not the IAM role of the instance, but is a role that the instance is permitted to assume. So AFAIK this profile is saying "please assume the role with ARN role_arn, and you can find the creds in the Ec2InstanceMetadata".

[mattBrzezinski]

Ah okay, I think I understand now. You want to use the credential_source = Ec2InstanceMetadata credentials to then assume the role_arn = arn:aws:iam:XYZ:role/Y role.

This is probably a bit more of a complex assumption process. It's definitely do-able, but would require some more time spent thinking of a solution. Supporting more use cases for various configurations would be great to have, currently I don't have too much time to dive into this and would be a lower priority for me.

If you'd like to make the propose changes for it, I can dedicate some time for code review on it. A quick work-around for this if possible is to give the default credentials on the instance the ability to assume the role you want and then use:

https://github.com/JuliaCloud/AWS.jl/blob/master/src/services/sts.jl#L26

To create an AWSConfig object with those credentials.

[ericphanson]

Thanks for the workaround! That worked for me. In the language of my example, I think it's this:

using AWS: @service
@service STS
function config_by_assuming_role_from_profile(profile::String, role_session_name; config = global_aws_config())
    config_file = AWS.dot_aws_config_file()
    isfile(config_file) || error("`config` file not found at $(config_file)")
    ini = read(AWS.Inifile(), config_file)
    role_arn = AWS._get_ini_value(ini, profile, "role_arn")
    role_arn === nothing && error("`role_arn` not found in profile $profile in config file $(config_file)")
    assume_role_results = STS.assume_role(role_arn, role_session_name)
    role_creds = AWS.AWSCredentials(
        assume_role_results["AssumeRoleResult"]["Credentials"]["AccessKeyId"], # access_key_id
        assume_role_results["AssumeRoleResult"]["Credentials"]["SecretAccessKey"], # secret_key
        assume_role_results["AssumeRoleResult"]["Credentials"]["SessionToken"], # token,
        assume_role_results["AssumeRoleResult"]["AssumedRoleUser"]["Arn"], # user_arn
        config.credentials.account_number #= account_number =#)
    return AWSConfig(role_creds, config.region, config.output)
end

role_Y_config = config_by_assuming_role_from_profile("role-Y", "using_role_Y_for_reasons")

and then role_Y_config has the right role, so you can do e.g.

path = S3Path("s3://my-bucket/role-Y-path/test"; config = role_Y_config)
open(path, write=true) do io
    write(io, "abc")
end # no errors!

if that role now gives you write permissions for s3://my-bucket/role-Y-path/test.

[ericphanson]

Note: the config_by_assuming_role_from_profile above does not pass a renew function when it constructs the credentials, so those creds can expire (at unfortunate times, as @femtomc found out). So if we implement this properly we should make sure to pass a renew function.