Error updating Lambda event source mapping: InvalidParameterValueException: Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type.

[jeffrichie]

Our code:

  new aws.lambda.EventSourceMapping(`shopify-fetcher-consumer-esm`, {
    eventSourceArn: msk.arn,
    functionName: shopifyFetcherLambda.arn,
    topics: [topics.rawObjectRef],
    startingPosition: 'LATEST',
    batchSize: 100
  });

The error:

aws:lambda:EventSourceMapping (shopify-progress-manager-topic-_shopify-pagination-complete-consumer-esm):
      error: 1 error occurred:
      	* updating urn:pulumi:development-jeffrichie::nacelle-services-dilithium::aws:lambda/eventSourceMapping:EventSourceMapping::shopify-progress-manager-topic-_shopify-pagination-complete-consumer-esm: 1 error occurred:
      	* Error updating Lambda event source mapping: InvalidParameterValueException: Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type.
      {
        RespMetadata: {
          StatusCode: 400,
          RequestID: "a1b27789-3f3a-420d-aa16-8ac91b8db43a"
        },
        Message_: "Unsupported 'maximumBatchingWindowInSeconds' parameter for given event source mapping type.",
        Type: "User"
      }

Expected behavior

Since we aren't passing the maximumBatchingWindowInSeconds parameter, neither should Pulumi.

Steps to reproduce

1. pulumi up to create the ESM for the first time
2. pulumi refresh
3. pulumi up again and the error comes out.

[jeffrichie]

@viveklak any updates? I understand this is probably an AWS provider issue.

[tusharshahrs]

@jeffrichie,
This looks like an issue with the upstream provider.

1. Does setting the maximum_batching_window_in_seconds to a valid non zero value work? Might be worth a shot.
2. Are there steps that you can take to make it fail all the time instead of randomly?

[jeffrichie]

@tusharshahrs Thanks for the response. No, that doesn't work unfortunately. You get the same error.

Once it starts failing, it doesn't stop until you destroy the resource and then re-pulumi up

[viveklak]

Sorry for the delay and thanks @tusharshahrs for finding the related issue. It does seem like an AWS provider bug.

@jeffrichie curious if you can try dropping back to an older version of the pulumi-aws sdk - say - v3.19.0 and report back?

[jeffrichie]

@viveklak I can give that a shot next time we run into this.

[jeffrichie]

@viveklak 3.19.0 of the provider hadn't introduced MSK topic in ESM's yet. It didn't work.

[jeffrichie]

@viveklak @tusharshahrs we figured out recreation steps -

1. pulumi up to create the ESM for the first time
2. pulumi refresh
3. pulumi up again and the error comes out.

Any ideas? This is becoming a serious blocking issue.

[tusharshahrs]

Hi @jeffrichie,
Can you run pulumi version and so we know what version this happens on. The refresh was recently fixed here. The change was:

• [cli] Improve diff displays during pulumi refresh Fix pulumi refresh diffs #6568

[jeffrichie]

@tusharshahrs

jeffrichie@Jeffs-MacBook-Pro ~ % pulumi version
v2.23.2

[viveklak]

I am afraid this continues to be an issue with the upstream provider and I can't think of a reliable workaround. There is this PR that has been open for a few days which should resolve it.