fix: Set basic_auth to optional in self_managed_kafka_parameters

[vzilinas]

Description

Fixes the validation on aws_pipes_pipe.source_parameters.self_managed_kafka_parameters.credentials to make basic_auth not required. This is already correct in documentation https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/pipes_pipe#self_managed_kafka_parameters

Relations

Closes #33025

References

Output from Acceptance Testing

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

• Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
• Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

[github-actions]

Welcome @vzilinas 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

[marek-kwitek-ef]

Looks good.

[erlandsen-tech]

Is this not merged, as it is still an issue.

[abenhadjhammouda]

@justinretzolk @ewbankkit Is it possible to merge this PR please ? It is blocking for us. Thanks

[jgaynon]

This is still an issue in provider version 5.52.0.

This 1 line PR fixes a bug #33025 open for 10 months and yet it's been sitting, approved but not merged, for 6 months? This PR really needs to be merged, the bug completely blocks creating an EventBridge pipe with a self managed Kafka source that doesn't use basic auth - any serious production usage of Kafka will not be using basic auth.

[erlandsen-tech]

We are using a workaround where we deploy the pipe and then via cli we add authentication.. Any feedback on this would be appreciated from our side as well. Kind regards, John Inge Erlandsen5. juni 2024 kl. 15:52 skrev jgaynon ***@***.***>: This is still an issue in provider version 5.52.0. This 1 line PR fixes a bug #33025 open for 10 months and yet it's been sitting, approved but not merged, for 6 months? This PR really needs to be merged, the bug completely blocks creating an EventBridge pipe with a self managed Kafka source that doesn't use basic auth - any serious production usage of Kafka will not be using basic auth. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[jgaynon]

@erlandsen-tech I'm using a horrible hack workaround using a local_exec provisioner that runs the AWS CLI to update the pipe with the correct credentials but I really shouldn't need to given how simple the code fix is on the provider side:

resource "aws_pipes_pipe" "eventbridge_pipe" {

<PIPE CONFIG HERE>

# Horrendous hack to get around the bug with source_parameters
  provisioner "local-exec" {
    command = "aws pipes update-pipe --name ${self.name} --role-arn ${self.role_arn} --source-parameters \"${local.source_parameters}\" --region eu-west-1"
  }

  # Always recreate the pipe so the local-exec provisioner always runs
  lifecycle {
    replace_triggered_by = [ null_resource.always_run ]
  }
}

resource "null_resource" "always_run" {
  triggers = {
    timestamp = "${timestamp()}"
  }
}

locals {
  source_parameters = <<EOF
{\"SelfManagedKafkaParameters\": {\"Credentials\": {\"ClientCertificateTlsAuth\": \"${aws_secretsmanager_secret.kafka_cert_key.arn}\"}, \"ServerRootCaCertificate\": \"${aws_secretsmanager_secret.kafka_root_ca.arn}\"}}
  EOF
}

[ewbankkit]

LGTM 🚀.

% make testacc TESTARGS='-run=TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget' PKG=pipes
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/pipes/... -v -count 1 -parallel 20  -run=TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget -timeout 360m
=== RUN   TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget
=== PAUSE TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget
=== CONT  TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget
    pipe_test.go:1126: Error running post-test destroy, there may be dangling resources: exit status 1
        
        Error: deleting Security Group (sg-0871bf5a0727f5338): DependencyViolation: resource sg-0871bf5a0727f5338 has a dependent object
        	status code: 400, request id: 6250a9d0-3d5a-4ed0-8a63-9c8581299485
        
        
        Error: deleting EC2 Subnet (subnet-0a9de4f3eb3645acf): DependencyViolation: The subnet 'subnet-0a9de4f3eb3645acf' has dependencies and cannot be deleted.
        	status code: 400, request id: 45977352-1d7e-46e2-8c5e-a7034fb355ee
        
        
        Error: deleting EC2 Subnet (subnet-0c60e15176cb5723d): DependencyViolation: The subnet 'subnet-0c60e15176cb5723d' has dependencies and cannot be deleted.
        	status code: 400, request id: 365924a4-dd38-44df-9d28-a9d256081c4c
        
--- FAIL: TestAccPipesPipe_selfManagedKafkaSourceLambdaFunctionTarget (1460.73s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/pipes	1465.338s
FAIL
make: *** [testacc] Error 1

Errors are expected.

[ewbankkit]

@vzilinas Thanks for the contribution 🎉 👏.

[github-actions]

This functionality has been released in v5.53.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.