Data source and Resource for generic signed API request

[KyleKotowick]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

There are many things supported by the AWS CLI that aren't supported by the Terraform AWS provider. Unfortunately, this will always be the case as it takes months, or sometimes years, for new AWS features to be supported in the Terraform AWS provider.

To use these features, engineers often resort to using the AWS CLI with the external data source. The problem is, how do you provide appropriate credentials to the AWS CLI commands? The easiest way is to pass it a profile argument, but it becomes complicated if you want to be able to support the same credential sources as the AWS provider (access keys, profiles, environment variables, assumed roles, etc.). Additionally, it requires having the AWS CLI installed, which may not always be the case, and it becomes very difficult to track the state of resources created/modified this way.

What if we could provide a data source and resource for generic, signed requests against the AWS APIs? Allow engineers to specify an API endpoint and request parameters, have the provider format and sign the request using the same credentials the provider is configured with, make the request, and then output the response parameters? For the data source variant, do not track anything in state and re-run the request each time; for the resource variant, allow providing on_create, on_update, and on_delete API endpoints/requests that get run at their respective points in the lifecycle.

New or Affected Resource(s)

• data aws_api_request
• aws_api_request

Potential Terraform Configuration

An example of how you would use this to add S3 replication after a bucket has been created (a workaround for issue #749):

resource "aws_signed_request" "s3_replication" {

  // Example from https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketReplication.html#API_PutBucketReplication_Examples
  on_create {
    method = "POST"
    host = "examplebucket.s3.<Region>.amazonaws.com"
    uri = "/?replication"
    request = <<EOT
<ReplicationConfiguration>
  <Role>arn:aws:iam::35667example:role/CrossRegionReplicationRoleForS3</Role>
  <Rule>
    <ID>rule1</ID>
    <Status>Enabled</Status>
    <Priority>1</Priority>
    <DeleteMarkerReplication>
       <Status>Disabled</Status>
    </DeleteMarkerReplication>
    <Filter>
       <And>
           <Prefix>TaxDocs</Prefix>
           <Tag>
             <Key>key1</Key>
             <Value>value1</Value>
           </Tag>
           <Tag>
             <Key>key1</Key>
             <Value>value1</Value>
           </Tag>
       </And>
    </Filter>
    <Destination>
      <Bucket>arn:aws:s3:::DOC-EXAMPLE-BUCKET</Bucket>
    </Destination>
  </Rule>
</ReplicationConfiguration>
EOT
  }

  on_update {
    // Would use same configuration as `on_create`, since PutBucketReplication is idempotent
    ...
  }

  on_delete {
    method = "DELETE"
    host = "examplebucket.s3.<Region>.amazonaws.com"
    uri = "/?replication"
    // This endpoint doesn't accept a request body
    request = null
  }
}

In this example, on_create and on_update would be the same since PutBucketReplication is idempotent.

For a different use case, such as Kinesis Firehose CreateDeliveryStream, the on_create, on_update, and on_delete blocks might use the CreateDeliveryStream, UpdateDestination, and DeleteDeliveryStream actions, respectively.

The resource variant would support multiple on_create, on_update, and on_delete blocks to allow for multiple actions to be taken at each stage in the lifecycle.

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[KyleKotowick]

Not stale, still just waiting on implementation.