Decryption failed when different aws-profile names used

[richstokes]

This is a strange one, I seem to only be able to decrypt files that are made by the user that created the file.

If I try and decrypt another file it fails. Even though both files are encrypted with the same KMS key and both users have permissions to access the key within AWS.

sops --verbose --aws-profile saml -d madebyme.yaml

[AWSKMS]	 INFO[0000] Decryption succeeded                          arn="arn:aws:kms:us-west-2:123456:key/123123123123"
[SOPS]	 INFO[0000] Data key recovered successfully              
[SOPS]	 DEBU[0000] Decrypting tree                              
hello: Welcome to SOPS! Edit this file as you please!

sops --verbose --aws-profile saml -d madebyanother.yaml

[AWSKMS]	 INFO[0012] Decryption failed                             arn="arn:aws:kms:us-west-2:123456:key/123123123123"
[AWSKMS]	 INFO[0024] Decryption failed                             arn="arn:aws:kms:us-east-2:123456:key/456456456456"
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:us-west-2:123456:key/123123123123: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      | 	For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors
  
  arn:aws:kms:us-east-2:123456:key/456456456456: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      | 	For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

If I use the AWS CLI I can also encrypt and decrypt files manually. Any ideas what else I should try?

[richstokes]

If I do the reverse and send the file I can decrypt to another person, then THEY cannot decrypt this one with:

    - | Error decrypting key: ExpiredTokenException: The security
      | token included in the request is expired
      | 	status code: 400, request id:

Pretty confused by this since all are set to use the same KMS keys. When encrypting with --verbose set it also confirms that both files were encrypted with the same key.. hmm

[richstokes]

Its something to do with --aws-profile - looks like the files have to be decrypted with the same aws-profile name they were encrypted with.

In .aws/credentials we have another profile as users have various sets of credentials, but the name for this is liable to vary between users. How can we have sops iterate through all the credential blocks in the credentials file until it finds a match? Is that possible?

Not sure if I am misunderstanding what the --aws-profile flag does - I thought it would let you tell sops which block of credentials to use in the .aws file but apparently not. Or, sops encrypts the file with some kind of tag where only a profile with a matching name can decrypt them.

[autrilla]

It does let you tell sops which block of credentials to use. And also yes, setting --aws-profile will set the profile for that file.

[richstokes]

In that case there is a bug. Reproduce with:

1. Computer 1: Encrypt a file with an aws profile named "foo"
2. Computer 2: Try and decrypt the same file with an aws profile named "bar"

Even though both computers/users are using the same KMS ARN it will fail

[autrilla]

Is the profile the same?

…

On Fri, 24 Jan 2020 at 18:10, richstokes ***@***.***> wrote: In that case there is a bug. Reproduce with: 1. Computer 1: Encrypt a file with an aws profile named "foo" 2. Computer 2: Try and decrypt the same file with an aws profile named "foo" Even though both computers/users are using the same KMS ARN it will fail — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#614>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARH4VY65NF46JPLIBKWZM3Q7MOHZANCNFSM4KK7UCMQ> .

[richstokes]

The profiles are different - they are different users who have named their block of credentials differently

[richstokes]

The AWS profile name used by any user should be totally arbitrary, right? And have no impact on how the files are encrypted?

Wondering if I'm doing something wrong or if this is a confirmed bug

[autrilla]

I don't think it should have any impact. Are you sure you're using the correct accounts etc?

Your error:

    - | Error decrypting key: ExpiredTokenException: The security
      | token included in the request is expired
      | 	status code: 400, request id:

Seems to indicate you just need to reauth (re-MFA, or whatever your setup is)

[richstokes]

I tried reauthing, and can encrypt and decrypt my own files OK. I only see the problem when trying to decrypt files created by another user (even though we both use the same KMS keys).

And they see the reverse, they are unable to decrypt my files.

However if we both establish our AWS Credential file (~/.aws/credentials) using the "default" profile instead of a named one, everything works fine.

[richstokes]

OK I found something else, if I manually edit the encrypted file and edit the aws_profile: <name> field to match my local AWS profile name (instead of the AWS profile name used by the file creator) I am able to decrypt it!

So this is definitely a bug. sops should ignore that field when decrypting, and/or override it when the --aws-profile flag is set on the command line. So long as both users have access to the same KMS keys it should work regardless of their local aws-profile name.

Before manually editing the aws_profile: field:

sops --aws-profile il -d test.yaml                                                                                                                                                                                                               

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:us-west-2:abc123:key/abc123-abc123-4258-83fc-6603f37eeda6: FAILED
    - | Error decrypting key: ExpiredTokenException: The security
      | token included in the request is expired
      | 	status code: 400, request id:
      | 853579d7-1579-4cbe-af0c-9e51706de807

  arn:aws:kms:us-east-2:abc123:key/abc123-abc123-4245-a4bc-ad7324959fa6: FAILED
    - | Error decrypting key: ExpiredTokenException: The security
      | token included in the request is expired
      | 	status code: 400, request id:
      | ce0a9008-d447-4976-ae5e-ce4648146f59

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

After manually editing the aws_profile: field:

sops --aws-profile il -d test-editted.yaml
                                                                                                                                                                                                      
hello: This is a test 123123
example_key: example_value
# Example comment
example_array:
- example_value1
- example_value2
example_number: 1234.5679
example_booleans:
- true
- false

[autrilla]

So this is definitely a bug. sops should ignore that field when decrypting

Why? It's stored exclusively to use the same profile (by name) every time you decrypt.

, and/or override it when the --aws-profile flag is set on the command line. So long as both users have access to the same KMS keys it should work regardless of the aws-profile name.

I think --aws-profile should just not work unless you're creating a file. Similar to --pgp or --kms, I guess.

[richstokes]

Well then shouldn't --aws-profile override whatever profile name the user that created the file used.?

Otherwise all users in a company would need to make sure they use the exact same AWS profile name when working with sops. Which seems silly because the profile name is totally arbitrary and has no impact on whether they can access the same KMS keys or not.

[autrilla]

Well then shouldn't --aws-profile override whatever profile name the user that created the file used.?

No, just like using --pgp when editing a file does not override what PGP key is used.

Otherwise all users in a company would need to make sure they use the exact same AWS profile name when working with sops. Which seems silly because the profile name is totally arbitrary and has no impact on whether they can access the same KMS keys or not.

Well... yeah. That's the whole point why the feature was implemented AFAIK. I would expect people in the same company to use the same profile names.

Why not set the AWS_PROFILE env var instead of passing the flag? That should do what you want, no?

[richstokes]

The AWS_PROFILE env var behaves the same - it will only decrypt the file if the same profile name was used that encrypted the file.

The docs say that "Similarly the --aws-profile flag can be set with the command line with any of the KMS commands." And earlier in this ticket you agreed that the name should not matter. :-)

May be worth adding something to do the docs saying that you have to use the same AWS Profile name for both encrypting and decrypting. That is not clear, and in all other AWS operations the profile name is totally arbitrary and local to your machine, the name alone shouldn't impact your access/permissions/ability to do something.

[autrilla]

The AWS_PROFILE env var behaves the same - it will only decrypt the file if the same profile name was used that encrypted the file.

Hmmm, if it does, I'd consider that the bug. The AWS_PROFILE flag should be handled by the AWS SDK, so if you use it, SOPS should have no clue what profile you're using, only the AWS SDK would know. I don't see us reading that environment variable anywhere in the code though, so not sure how SOPS would know what profile name to use across different computers.

The docs say that "Similarly the --aws-profile flag can be set with the command line with any of the KMS commands." And earlier in this ticket you agreed that the name should not matter. :-)

Yeah, that doc line is not great. It kind of tells you the right thing (if you know what the right thing is): it can be set with any of the other KMS-related flags, and there is only one of those: --kms.

And yes, you're right. Sorry for misleading you!

May be worth adding something to do the docs saying that you have to use the same AWS Profile name for both encrypting and decrypting. That is not clear, and in all other AWS operations the profile name is totally arbitrary and local to your machine, the name alone shouldn't impact your access/permissions/ability to do something.

That should only be the case if you use the --aws-profile flag. Whether there's a bug or not, I think that's the intent. Perhaps we should consider renaming the flag. Either way, changing the docs to clarify this would definitely be a good thing. In all other operations the AWS profile name is irrelevant because SOPS does not ever see it.

[richstokes]

Thank you, no worries -- for now I will make sure our own internal processes are well documented to always use a specific profile name to avoid any confusion!

[casey-robertson]

@richstokes seem to be running into this too. I don't know what changed. I was working with a developer the other day and nothing would work until added a role into the .sops.yaml and/or the encrypted file. Simply would not assume a role no matter what env vars we set.

[Crysis73]

I was having a similar issue. I was able to get around needing to use any aws-profile name but running aws sts assume-role --role-arn [rolearn] --role-session-name [name].

[mmingorance-dh]

I have this issue when I configure sops with .sops.yaml to use a specific AWS_PROFILE but it seems to break when AWS_PROFILE is not set in my bash.
Once I run export AWS_PROFILE=$myprofile it does work.

According to this documentation https://github.com/mozilla/sops#25kms-aws-profiles I understand that this value should be taken from the sops configuration.

Example file:

sops:
  kms:
  - arn: '$AWS_KMS_KEY_ARN'
    aws_profile: '$AWS_PROFILE'

creation_rules:
- path_regex: ./secrets/
  kms: '$AWS_KMS_KEY_ARN'

[ruudk]

Having the same issue.

# Does work
AWS_PROFILE=production sops -i production/secrets.yaml

# Does not work
sops --aws-profile=production -i production/secrets.yaml

[ilyagorban-codefresh]

For me assuming of aws roles did not work in any way through a profile. I found the following workaround for the assuming role:

$(aws sts assume-role --role-arn $ARN_OF_ROLE_TO_ASSUME --role-session-name manual | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId) AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey) AWS_SESSION_TOKEN=\(.SessionToken)"')
sops -i production/secrets.yaml
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN

[cpflaume]

Run into the same issue and was mislead by the documentation:

Similarly the --aws-profile flag can be set with the command line with any of the KMS commands.

This flag is only taken into account during encryption and as far as I experienced ignored during decryption. '
Exporting the env var AWS_PROFILE did work for me.

[sirantd]

For me assuming of aws roles did not work in any way through a profile. I found the following workaround for the assuming role:

$(aws sts assume-role --role-arn $ARN_OF_ROLE_TO_ASSUME --role-session-name manual | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId) AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey) AWS_SESSION_TOKEN=\(.SessionToken)"')
sops -i production/secrets.yaml
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN

The only solution for me so far when I'm using SSO for initial profile. Without SSO everything working as expected but with SSO I have:

Group 0: FAILED
  arn:aws:kms:us-east-1:XXXXXX:key/mrk-xxxxx: FAILED
    - | Error decrypting key: SSOProviderInvalidToken: the SSO
      | session has expired or is invalid

but if I do aws sts assume-role I have no errors and then able to decrypt values.

[denisaka1]

It works for SSO if you are using their manual legacy way https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-legacy.html

1. Create manually a profile in ~/.aws/config
   [profile my-dev-profile] sso_start_url = https://my-sso-portal.awsapps.com/start sso_region = us-east-1 sso_account_id = 123456789011 sso_role_name = readOnly region = us-west-2 output = json

2. login to sso aws sso login --profile my-dev-profile

3. run the sops AWS_PROFILE=my-dev-profile sops /path/to/file