Add support for Kinesis

[sebasrp]

Feature Request

I want to monitor the usage and limits of AWS Kinesis as specified in: https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html

In particular, the account limit around number of shards, which has 2 quotas depending on the region

Feature Description

I want to be able to monitor the number of shards per account and per region, as it's one of the hard limits. It should support the fact that we have different quotas depending on the region:
The default shard quota is 500 shards per AWS account for the following AWS regions: US East (N. Virginia), US West (Oregon), and Europe (Ireland). For all other regions, the default shard quota is 200 shards per AWS account.

Testing Assistance

I am willing to assist in testing pre-release code for the feature.

[sebasrp]

I will be working on this PR, but focus on a PR just for this quota, which is a bit more convoluted than other ones as:

• the quota depends on the region
• need to sum all shards across the region
• the list_streams has a weird behaviour (only 10 streams returned per API request but doesn't use the general token)

[jantman]

@sebasrp I know you've opened quite a few PRs lately, and I'm very thankful for that. I'm unable to work on them this weekend, but will do my best to review them one evening next week.

Regarding convoluted limits... in #485 I found a few limits for Fargate that were very complicated, because it would have required both enumerating all EKS pods, as well as understanding the details of ECS capacity providers. I ended up finding out that AWS now publishes usage metrics to CloudWatch for some services/limits, and this is presumably the actual number that AWS uses.

I still need to add this to the docs, but if it's a usage that's difficult to count, it might be worth looking to see if there is an AWS/Usage CloudWatch metric for it, and if so, use the new AwsService._get_cloudwatch_usage_latest method (source code).

[sebasrp]

Unfortunately I do not believe Kinesis sends those usage metrics :( - see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Service-Quota-Integration.html

There are also no vended metrics at the account level - only at the stream level: https://docs.aws.amazon.com/streams/latest/dev/monitoring-with-cloudwatch.html

[sebasrp]

Actually I stand corrected - althtough there is no CloudWatch metrics for usage, there is actually a "DescribeLimits" API that gets us the current quota and usage - neat!

[jantman]

A fix for this issue has been merged to the develop branch and is slated for release in 10.0.0 on Monday, December 7, 2020. Anyone who is able to test the fix is strongly encouraged to do so, and comment on this issue with their results. Instructions for installing awslimitchecker from a git branch are available in the Installing for Development documentation.

[jantman]

This has been fixed in 10.0.0, which is now live on PyPI and is being built on the Docker hub right now. Thank you so much for the contribution!