aws_lambda_event_sources: Missing support for maxRecordAge=-1 value in KinesisEventSourceProps

[grollat]

Describe the bug

The service supports -1 as a value but CDK imposes a Duration object which does not support negative value. This can become an issue in case the Kinesis Stream Retention window is set to > 7 days (which is the max allowed for maxRecordAge). when this happens customer may lose records.

Expected Behavior

the KinesisEventSourceProps should support a way to set the maxRecordAge value to -1. (Duration(-1)?)

Current Behavior

Duration(-1) is not allowed and no other way to set this value

Reproduction Steps

self explanatory

Possible Solution

allow Duration.seconds(-1) as a way to pass through the value?

Additional Information/Context

No response

CDK CLI Version

latest

Framework Version

No response

Node.js Version

all

OS

all

Language

Typescript

Language Version

all

Other information

No response

[peterwoodworth]

Thanks for reporting, we may want to allow an override on Duration to allow this to work.

For now, you can work around this by overriding the template with escape hatches

[roger-zhangg]

Just FYI for who doesn't know, The default value for maxRecordAge is -1, document. So if you don't set this value at all, it will be automatically set to -1 in lambda.

E.g.:

const eventSource = new KinesisEventSource(stream, {
      startingPosition: lambda.StartingPosition.TRIM_HORIZON,
      tumblingWindow: cdk.Duration.seconds(60),
});

[TheRealAmazonKendra]

Thank you @roger-zhangg for your comment here. Since you can get the value of -1 by leaving this undefined, this seems like a documentation/doc string update is needed rather than a code change.

We should update the docstring here to reflect that not setting this value will result in -1. From the current docstring, though, it seems that it may also require not setting the field for the retention period as well (I'm not sure what that default is).

[roger-zhangg]

Thanks for the suggestion, I'll take a look at how retention period might affect the maxRecordAge.

Update:
It seems that the current docstring default @default - the retention period configured on the stream is actually saying that the default value of maxRecordAge in lambda event source mapping is -1. This setting make lambda never discard a record from Stream. And the retention period configured on the kinesis stream now decides the max record age. But for the maxRecordAge itself, it's still -1.

E.g

const fn = new TestFunction(this, 'F');
    const stream = new kinesis.Stream(this, 'Q', {retentionPeriod: cdk.Duration.days(7)});
    const eventSource = new KinesisEventSource(stream, {
      startingPosition: lambda.StartingPosition.TRIM_HORIZON,
      tumblingWindow: cdk.Duration.seconds(60),
    });

    fn.addEventSource(eventSource);

Output maxRecordAge is still -1. I'll go ahead to send a PR to make this docstring more understandable.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.