aws_dms_endpoint: Add Kinesis as a target endpoint type

[jbergknoff-rival]

This PR adds Kinesis as a DMS target endpoint type. #6658 #8053

Also, the acceptance test for MongoDB DMS endpoints was broken, and I implemented a fix (or workaround?) here as well. #7492

Finally, I'd just like to request some input on @chrisjharding's #2305. Without that functionality, it's really not feasible to use DMS via Terraform (it's currently necessary to do work outside of Terraform to start a replication task, and it's not possible to modify an in-progress replication via Terraform). If the author of that PR is available to respond to review, great. If not, maybe I can implement whatever changes are needed to get it merged.

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Fixes:

• aws_dms_endpoint TestAccAwsDmsEndpointMongoDb acceptance test is failing #7492 (aws_dms_endpoint TestAccAwsDmsEndpointMongoDb acceptance test is failing)
• AWS DMS aws_dms_endpoint - Support 'kinesis data stream' target support #8053 (AWS DMS aws_dms_endpoint - Support 'kinesis data stream' target support)
• Using kinesis firehose as a aws_dms_endpoint #6658 (Using kinesis firehose as a aws_dms_endpoint)

Release note for CHANGELOG:

resource/aws_dms_endpoint: Add Kinesis as a target

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAwsDmsEndpoint'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAwsDmsEndpoint -timeout 120m
=== RUN   TestAccAwsDmsEndpointBasic
=== PAUSE TestAccAwsDmsEndpointBasic
=== RUN   TestAccAwsDmsEndpointS3
=== PAUSE TestAccAwsDmsEndpointS3
=== RUN   TestAccAwsDmsEndpointDynamoDb
=== PAUSE TestAccAwsDmsEndpointDynamoDb
=== RUN   TestAccAwsDmsEndpointKinesis
=== PAUSE TestAccAwsDmsEndpointKinesis
=== RUN   TestAccAwsDmsEndpointMongoDb
=== PAUSE TestAccAwsDmsEndpointMongoDb
=== RUN   TestAccAwsDmsEndpointDocDB
=== PAUSE TestAccAwsDmsEndpointDocDB
=== CONT  TestAccAwsDmsEndpointBasic
=== CONT  TestAccAwsDmsEndpointMongoDb
=== CONT  TestAccAwsDmsEndpointKinesis
=== CONT  TestAccAwsDmsEndpointDocDB
=== CONT  TestAccAwsDmsEndpointDynamoDb
=== CONT  TestAccAwsDmsEndpointS3
--- PASS: TestAccAwsDmsEndpointDocDB (26.69s)
--- PASS: TestAccAwsDmsEndpointBasic (26.73s)
--- PASS: TestAccAwsDmsEndpointMongoDb (32.25s)
--- PASS: TestAccAwsDmsEndpointDynamoDb (36.77s)
--- PASS: TestAccAwsDmsEndpointS3 (37.67s)
--- PASS: TestAccAwsDmsEndpointKinesis (114.30s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws   114.358s

[bordeux]

What is the status on it ? :)

[jbergknoff-rival]

What is the status on it ? :)

I don't think the maintainers have looked at this PR.

Separately, my team finished our POC using DMS to push a Postgres change stream to Kinesis, but abandoned it because the Kinesis DMS target has no support for LOB types (e.g. text columns) and the throughput was abysmal (the limiting factor was DMS, not Kinesis or Postgres).

[bflad]

Hi @jbergknoff-rival 👋 Apologies for the delayed review here. We're going to fix the merge conflicts and pull this in with some very minor adjustments (mainly just removing the mongodb changes). Looks good! 🚀

Output from acceptance testing:

--- PASS: TestAccAwsDmsEndpoint_DocDB (29.10s)
--- PASS: TestAccAwsDmsEndpoint_Db2 (29.17s)
--- PASS: TestAccAwsDmsEndpoint_Basic (29.30s)
--- PASS: TestAccAwsDmsEndpoint_MongoDb (36.13s)
--- PASS: TestAccAwsDmsEndpoint_DynamoDb (37.92s)
--- PASS: TestAccAwsDmsEndpoint_S3 (39.02s)
--- PASS: TestAccAwsDmsEndpoint_Kinesis (79.51s)

[ghost]

This has been released in version 2.58.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!