[Bug]: datePartitionDelimiter is not a valid parameter in extra connection attributes

[thundermann]

Terraform Core Version

1.3.2

AWS Provider Version

4.45.0

Affected Resource(s)

aws_dms_endpoint

Expected Behavior

Terraform should not send the default value for datePartitionDelimiter when and endpoint_type = "source"

Actual Behavior

Endpoint fails to create because date_partition_delimiter attribute is not valid for dms s3 as a source. Terraform is sending default value of date_partition_delimiter="slash" causing the call to fail.

I am assuming this applies to the other data_partition attributes as well.

  + s3_settings {
          + add_column_name                             = false
          + bucket_folder                               = "client_analytics"
          + bucket_name                                 = "cb-dev-data-lake"
          + canned_acl_for_objects                      = "none"
          + cdc_inserts_and_updates                     = false
          + cdc_inserts_only                            = false
          + cdc_max_batch_interval                      = 60
          + cdc_min_file_size                           = 32
          + compression_type                            = "NONE"
          + csv_delimiter                               = ","
          + csv_null_value                              = "NULL"
          + csv_row_delimiter                           = "\\n"
          + data_format                                 = "csv"
          + data_page_size                              = 1048576
          + date_partition_delimiter                    = "slash"
          + date_partition_enabled                      = false
          + date_partition_sequence                     = "yyyymmdd"
          + dict_page_size_limit                        = 1048576
          + enable_statistics                           = true
          + encoding_type                               = "rle-dictionary"
          + encryption_mode                             = "SSE_S3"
          + external_table_definition                   = jsonencode(

Relevant Error/Panic Output Snippet

│ Error: creating DMS Endpoint (client-analytics-etl-transaction-detail-source-endpoint): InvalidParameterValueException: Unsupported value 'datePartitionDelimiter' for extra connection attributes
│       status code: 400, request id: 29946bc0-36b7-4ff4-be7f-7b3682719d79
│
│   with aws_dms_endpoint.client_analytics_etl_transaction_detail_source_dms_endpoint,
│   on dms_transaction_detail.tf line 1, in resource "aws_dms_endpoint" "client_analytics_etl_transaction_detail_source_dms_endpoint":
│    1: resource "aws_dms_endpoint" "client_analytics_etl_transaction_detail_source_dms_endpoint" {
│
╵

Terraform Configuration Files

resource "aws_dms_endpoint" "client_analytics_etl_transaction_detail_source_dms_endpoint" {
  endpoint_id                 = var.transaction_detail_source_endpoint_id
  endpoint_type               = "source"
  engine_name                 = "s3"
  //extra_connection_attributes = "ignoreHeaderRows=1;"
  ssl_mode                    = "none"
  s3_settings {
    bucket_name               = var.s3_source_bucket_name
    bucket_folder             = var.s3_source_bucket_folder
    external_table_definition = "{...}"
    service_access_role_arn   = aws_iam_role.dms_s3_role.arn
    ignore_headers_row        = "1"
  }
  tags = {
    department: var.department,
    project: var.project,
    stage: var.stage,
    status: var.status,
    team: var.team,
    cluster_identifier: var.cluster_identifier
  }
}


terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 4.45"
    }
  }
  required_version = "1.3.2"
}

Steps to Reproduce

Create a new DMS source endpoint with S3 as source, endpoint_type = "source"

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

#23506
#27852

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[thundermann]

Just found that the resource to use is aws_dms_s3_endpoint in place of the aws_dms_endppint. Found that in this merge.
#28130

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.