aws_lambda_event_source_mapping update fails with unsupported parameter when using MSK source

[delbaeth]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

terraform14 -v
Terraform v0.14.7

• provider registry.terraform.io/hashicorp/aws v3.30.0

Affected Resource(s)

• aws_lambda_event_source_mapping

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

provider "aws" {
  region = "us-east-1"
}

resource "aws_lambda_event_source_mapping" "test" {
  batch_size = 1000
  event_source_arn = "arn:aws:kafka:us-east-1:846866821192:cluster/kafka-indexer-lambda-msk-cluster-deploy-delbaeth/11f04185-8ff8-4e0a-a903-2245da2e6433-5"
  function_name = "arn:aws:lambda:us-east-1:846866821192:function:kafka-indexer-lambda-deploy-delbaeth"
  topics = [ "test.test" ]
  starting_position = "TRIM_HORIZON"
}

Debug Output

https://gist.github.com/delbaeth/91dc6cf4a9e324190dc8de0cdf168296

Panic Output

Expected Behavior

The batch size should have been updated from the current 100 to the new 1000.

Actual Behavior

The apply had an error:

terraform14 apply
aws_lambda_event_source_mapping.test: Refreshing state... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_lambda_event_source_mapping.test will be updated in-place
  ~ resource "aws_lambda_event_source_mapping" "test" {
      ~ batch_size                         = 100 -> 1000
        id                                 = "7c6f7bc7-8210-4d33-95a8-6b8b87af2647"
        # (16 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_lambda_event_source_mapping.test: Modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 10s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 20s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 30s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 40s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 50s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m0s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m10s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m20s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m30s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m40s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 1m50s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m0s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m10s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m20s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m30s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m40s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 2m50s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m0s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m10s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m20s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m30s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m40s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 3m50s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m0s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m10s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m20s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m30s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m40s elapsed]
aws_lambda_event_source_mapping.test: Still modifying... [id=7c6f7bc7-8210-4d33-95a8-6b8b87af2647, 4m50s elapsed]

Error: error updating Lambda Event Source Mapping (7c6f7bc7-8210-4d33-95a8-6b8b87af2647): InvalidParameterValueException: Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "7a5bcf25-d2da-442b-b971-9033a86fec30"
  },
  Message_: "Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type.",
  Type: "User"
}

  on main.tf line 5, in resource "aws_lambda_event_source_mapping" "test":
   5: resource "aws_lambda_event_source_mapping" "test" {

Steps to Reproduce

1. create an event source mapping for MSK
2. terraform apply
3. make a simple modification like changing batch size
4. terrafrom apply

Important Factoids

References

[ewbankkit]

Related:

• AWS Lambda support for Amazon Managed Streaming for Apache Kafka as an event source #14658
• r/aws_lambda_event_source_mapping: Add MSK as an event source #14746

Relevant AWS Lambda API reference.

The problem with values for resource parameters that weren't explicitly specified in configuration being sent on update (which this issue is a symptom of) has been noticed in #14522.

[delbaeth]

It does look like the same issue.

For other event source types it seems (?) like you could work around the problem by explicitly setting legal values
for those parameters. For MSK, if I try to set maximum_batching_window_in_seconds to a valid non zero value it
fails with:

"Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type."

which makes sense since it is not relevant for this event source type but I think it means there is no work around for this case.

[ghost]

This has been released in version 3.38.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.