backend/s3: Add support for specifying an external credentials process directly

[gdavison]

Terraform Version

Target version: 1.6.x

Use Cases

The backend can currently use an external credentials process from shared config files with a named profile. Support should be added to directly set it on the backend configuration block.

Potential Configuration

backend "s3" {
  # ...
  credential_process {
    command = "..."        # Required. Can be the full command or the executable name
    args    = ["...", ...] # Optional. Can contain arguments for the executable
  }
}

Needs investigation to see if environment variables can be passed as well.

The credentials process can be used along with assume_role but cannot be used with directly passed credentials, profile, or assume_role_with_web_identity.

Attempted Solutions

N/A

Proposal

backend "s3" {
  # ...
  credential_process {
    command = "..."        # Required. Can be the full command or the executable name
    args    = ["...", ...] # Optional. Can contain arguments for the executable
  }
}

Needs investigation to see if environment variables can be passed as well.

The credentials process can be used along with assume_role but cannot be used with directly passed credentials, profile, or assume_role_with_web_identity.

This is related to hashicorp/terraform-provider-aws#24885, and should be implemented at the same time. Will depend on hashicorp/aws-sdk-go-base#291

References

No response

[apparentlymart]

One thing we should consider here:

This seems like it would allow a root module to execute arbitrary code controlled by the module author during terraform init.

I'm not sure if that actually is a problem in practice. Here is some thinking aloud that will hopefully help decide:

• The ability to execute external code at all is not new here. It has technically been possible to ship a credentials configuration file along with the module and refer to it, and that file could specify a credentials helper program. Therefore we might argue this is an extension of what's already possible, only making it more convenient to use correctly.
• It will remain possible to use terraform init -backend=false to get dependencies installed without running any backend code, so there's still a path for those who want to insert an extra step of scanning their dependencies before giving them any opportunity to run.
• This is only under the control of the root module, so it cannot be exploited by a maliciously-modified third-party module. We typically assume that the user trusts their own root module because Terraform does not install that one itself.

My initial instinct based on the above is that this is not a problem, but I'm curious on your take, or if you can see any mistaken assumptions I'm making.

[lorengordon]

Fwiw, the particular use case I had in mind is that it is common for a credential_process utility to accept a role_arn, or various components thereof (account ID, role name, etc), as an argument to the command. The utility then uses that information to retrieve temporary credentials for the desired role.

Our current workaround is to specify an aws-cli profile in the provider config, where that profile contains the credential_process, and to have a separate profile for every possible role_arn. By supporting the argument directly in the provider, it becomes possible to use some interpolation available to terraform at run time to construct the command. For our use case, that would allow us to eliminate all those different profiles.