Support loading AWS credentials from AWS profiles

[rgueldem]

AWS supports storing credentials on disk and loading them into the AWS CLI as well as their SDK. We use that setup to integrate with S3 for our services and it would be great if OPA would be able to ingest these credentials as well instead of relying on environment variables.

Note, that the SDK profile credential provider supports rotation of credentials.

Expected Behavior

Ability to configure service credentials for S3 by referencing an AWS profile or by using the default AWS profile if none is set. Support for rotation of short-living credentials without restarts.

Actual Behavior

Need to pass in secrets via environment variable to set s3_signing option. Need to restart OPA when short-living credentials expire.

Steps to Reproduce the Problem

n/a

Additional Info

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html?com/amazonaws/auth/profile/ProfileCredentialsProvider.html
https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#NewSharedCredentials

[anderseknert]

@olamiko would you want to work on this?

[olamiko]

@anderseknert, yes, I will like to work on this issue. Please assign me

[rgueldem]

Hi team, are there any plans to implement this in the near to mid term? It would simplify our key rotation process since we currently need to restart our agents every 12 hours to refresh the credentials.

[tsandall]

@rgueldem we can prioritize this over the next few weeks. We should be able to land this in December.

[ashutosh-narkar]

@rgueldem it would be great to get your feedback on these changes. It would be good to know if they address your use-case.

[rgueldem]

@rgueldem it would be great to get your feedback on these changes. It would be good to know if they address your use-case.

Thanks! Going by the description this should work for us. I am a little concerned about the performance impact of reloading the file on each request. (Sorry, I just realized it loads them for each request to S3, which is perfect.) I will report back in case we see any issues after upgrading and applying the new config.