Limits to number of consumers

[sgangam]

The number of consuming streams (fanout/destination) streams seems to be limited by the parallelTargets parameter defined here: https://github.com/aws-samples/aws-lambda-fanout/blob/master/fanout.js#L34

If this parameter is 2, the data flows into two of the destination streams, if it is increased to 5 we observed that the data will flow into 5 destination streams.

Is this expected behavior? If yes, should this be documented/parameterized?

[julienlepine]

Hello, the parallelTargets parameter only defines the number of targets that will be processed in parallel.

• If you have less targets than the defined parallelTargets, there will be no impact.
• If you have more targets (5) than defined in the parallelTargets parameter (2), fanout will send 2 targets in parallel, then any time one of the targets is done (all records have been sent), it will select another to another target, in the limit of 2 in parallel.

We should parameterize that (and TBH I will need to do a refresh of the code, new Lambda features make it much easier, and some other features are duplicates of https://aws.amazon.com/blogs/compute/introducing-aws-lambda-destinations/ and https://aws.amazon.com/eventbridge/).