Skip to content

Commit

Permalink
Initial FAQ, with hopefully more contributions in the future.
Browse files Browse the repository at this point in the history
  • Loading branch information
stair-aws committed Aug 29, 2023
1 parent 78b565f commit f62dafd
Showing 1 changed file with 29 additions and 0 deletions.
29 changes: 29 additions & 0 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Frequently Asked Questions (FAQ)

## Stream Modality

Questions related to stream modality (e.g., [MultiStreamTracker][multi-stream-tracker]).

### What is the impact of transitioning my KCL app from single-stream to multi-stream?

This answer assumes the [StreamTracker][stream-tracker] implementation is being changed.
From KCL's perspective, there is no modality change by decreasing a multi-stream tracker from `N` streams to `1` stream.

The DDB `leaseKey`, used to persist metadata including lease checkpoint, has a modality-dependent format:

| Modality | `leaseKey` Format |
| --- | --- |
| single-stream | `<shardId>` |
| multi-stream | `<accountId>:<streamName>:<streamCreationTimestamp>:<shardId>` |

Transitioning an app -- either from single- to multi-, or vice versa -- creates a backwards-incompatible expectation on the `leaseKey`.
As a result, a KCL app will be blind to any `leaseKey`, and its checkpoint, that does not match the expected format.

## Resources

For additional information, please consider reading:
* https://docs.aws.amazon.com/streams/latest/dev/kcl-migration.html
* https://docs.aws.amazon.com/streams/latest/dev/shared-throughput-kcl-consumers.html

[multi-stream-tracker]: /amazon-kinesis-client/src/main/java/software/amazon/kinesis/processor/MultiStreamTracker.java
[stream-tracker]: /amazon-kinesis-client/src/main/java/software/amazon/kinesis/processor/StreamTracker.java

0 comments on commit f62dafd

Please sign in to comment.