Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider/aws: Report the correct number of shards for Kinesis streams #5401

Merged
merged 1 commit into from
Mar 5, 2016
Merged

provider/aws: Report the correct number of shards for Kinesis streams #5401

merged 1 commit into from
Mar 5, 2016

Conversation

Bowbaq
Copy link
Contributor

@Bowbaq Bowbaq commented Mar 1, 2016

This fixes (a small) part of #5352. Reporting the correct number of shards (ie. the number of open shards) at least allows to manually match in terraform whatever was set in AWS, preventing streams from being replaced and data lost.

@stack72
Copy link
Contributor

stack72 commented Mar 4, 2016

Hi @Bowbaq,

Does this relate to longer running streams? There is currently a terraform acceptance test that creates a stream with a number of shards and checks that that number os shards is correct

Paul

@Bowbaq
Copy link
Contributor Author

Bowbaq commented Mar 4, 2016

Shards are immutable. When a stream gets re-sharded, existing shards are
split or merged as needed. The resulting shards are "open" while the old
ones become "closed" (ie. the data is still available, but no new data can
be written). The only number of consequence for a stream's capacity is the
number of open shards.

Terraform doesn't support re-sharding so I'm guessing the acceptance test
didnt exercise this behavior.

On Friday, March 4, 2016, Paul Stack notifications@github.com wrote:

Hi @Bowbaq https://github.com/Bowbaq,

Does this relate to longer running streams? There is currently a terraform
acceptance test that creates a stream with a number of shards and checks
that that number os shards is correct

Paul


Reply to this email directly or view it on GitHub
#5401 (comment).

@stack72
Copy link
Contributor

stack72 commented Mar 4, 2016

Got it, so the test would be

  1. Create a stream with 1 shard
  2. Update the stream to 2 shards

Old code = shards would be 3 and TF would try and fix that

New code (your PR) = open shards are 2 are TF has no action

Right?

@Bowbaq
Copy link
Contributor Author

Bowbaq commented Mar 4, 2016

Correct

On Friday, March 4, 2016, Paul Stack notifications@github.com wrote:

Got it, so the test would be

  1. Create a stream with 1 shard
  2. Update the stream to 2 shards

Old code = shards would be 3 and TF would try and fix that

New code (your PR) = open shards are 2 are TF has no action

Right?


Reply to this email directly or view it on GitHub
#5401 (comment).

@stack72
Copy link
Contributor

stack72 commented Mar 5, 2016

Thanks @Bowbaq - trying to see if i can get a test in place for this now :)

@stack72
Copy link
Contributor

stack72 commented Mar 5, 2016

This looks like it works as expected :) Thanks for the work @Bowbaq!

stack72 added a commit that referenced this pull request Mar 5, 2016
…ount

provider/aws: Report the correct number of shards for Kinesis streams
@stack72 stack72 merged commit b66b9c9 into hashicorp:master Mar 5, 2016
@ghost
Copy link

ghost commented Apr 27, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants