Add support for AWS Lambda limits.

[kilsbo]

Feature Request

So, I humbly request lambda limits, using aws cli executing aws lambda get-account-settings currently gives:

$ aws lambda get-account-settings
{
    "AccountLimit": {
        "CodeSizeUnzipped": 262144000, 
        "UnreservedConcurrentExecutions": 1000, 
        "ConcurrentExecutions": 1000, 
        "CodeSizeZipped": 52428800, 
        "TotalCodeSize": 80530636800
    }, 
    "AccountUsage": {
        "FunctionCount": 12, 
        "TotalCodeSize": 2167198
    }
}
$

Feature Description

Getting the current account limits for AWS Lambda service.

Use Cases

For serverless development, load testing accounts, I usually have to increase limits to measure my lambda code performance. Or just increasing capacity by increasing concurrency limit on an account. Several accounts are used for us, so keeping track of limits with this cli tool would help me.

Version

5.1.0

Installation Method

Using pip install: pip install awslimitchecker

Supporting Software Versions

Python 2.7.10 (macOS Mojave default).
-bash: virtualenv: command not found

Host: MacOS Mojave 10.14 (18A391)

Actual Output

Of all the service limits currently supported, Lambda ones are not there.

Expected Output

In order of what I prefer most at the top, down to less preferred.

Size limits/usage are 1kb = 1024, real kb that is.

Limits (-l)
===========

Lambda/Code Size Unzipped                     250 MiB
Lambda/Unreserved Concurrent Executions       1000
Lambda/Concurrent Executions                  1000
Lambda/Code Size Zipped                       50 MiB
Lambda/Total Code Size                        75 GiB

...or:

Lambda/Code Size Unzipped                    262,144,000 b
Lambda/Unreserved Concurrent Executions      1000
Lambda/Concurrent Executions                 1000
Lambda/Code Size Zipped                      5,2428,800 b
Lambda/Total Code Size                       80,530,636,800 b

...or:

Lambda/Code Size Unzipped                    262144000
Lambda/Unreserved Concurrent Executions      1000
Lambda/Concurrent Executions                 1000
Lambda/Code Size Zipped                      52428800
Lambda/Total Code Size                       80530636800


Usage (-u)
==========

Lambda/Function Count                        12 
Lambda/Total Code Size                       7.9 GiB

...or:

Lambda/Function Count                        12 
Lambda/Total CodeSize                        8530636800 b

...or:

Lambda/Function Count                        12 
Lambda/Total Code Size                       8530636800

Testing Assistance

Absolutely, happy to help with that!

Thank you.

[jantman]

@kilsbo Overall, it's unlikely that this will be added by me, but pull requests are certainly welcome from anyone.

There are a handful of technical issues around Lambda though, which have kept me from adding it myself thus far...

1. Do you have any explanation of what CodeSizeZipped and CodeSizeUnzipped are? Is there any obvious way to determine the current values for them, without awslimitchecker having to download and extract the contents of every Lambda in the account? And are they high value for something like awslimitchecker, i.e. are they something people would need to be warned about ahead of time, or just some limit that developers need to be aware of?
2. Concurrent Executions is an instantaneous value. How would something like awslimitchecker handle this? When I considered adding Lambda support in the past this was my sticking point... according to Accessing Amazon CloudWatch Metrics for AWS Lambda - AWS Lambda it appears that since I last looked into this they've added a "Concurrent Executions" metric in CloudWatch, which certainly makes this easier. But even with that... we'd need to pull that metric for every Lambda in the account. But what value do we really count and consider in violation of the limit? Do we have a new parameter to awslimitchecker for how far back to look in CloudWatch? Do we just look at a full 24 hours of data and consider it a problem if we hit our concurrency limit any time in the past day, and assume most people are running once a day?

The Lambda concurrency limit is a bit different from all the other limits we're currently handling, in that the current usage can change drastically in a short amount of time, and that it's something we need to pull usage from CloudWatch instead of counting actual resources.

I'm certainly open to input on this, but for my own employer's accounts, we've been using CloudWatch Alarms for lambda concurrency instead of awslimitchecker, because of the above reasons.

[kilsbo]

Thank you for commenting.

I've actually already implemented this, even though I'm not a py developer. This CLI tool is that useful, and it helps us greatly now with Lambda as well. I will fulfill what's asked for in your guidelines and set up a PR.

To answer your questions about limits and usage, first have a look at this table over data points we can get from AWS API:

Name	Metric Type	Description	Default limit
TotalCodeSize	Limit	Function storage in total per region	75GB
CodeSizeUnzipped	Limit	Size of a deployment package for a function deploy, unzipped	250MB
CodeSizeZipped	Limit	Size of a deployment package for a function deploy, zipped	50MB
ConcurrentExecutions	Limit	Number of concurrent executions of functions per region.	1000
UnreservedConcurrentExecutions	Limit	If one or more functions are configured with reserved executions, all other functions share whats left.	1000
FunctionCount	Usage	Number of existing functions in a region.
TotalCodeSize	Usage	Size of deployd code packages for your existing lambdas in a region.

...and your questions:

1. CodeSizeZipped and CodeSizeUnzipped are just when you upload a code package for one function, it will fail if it exceeds any of those two. Interesting usage here is TotalCodeSize, which is the sum of uploaded and zipped code packages for all lambdas in a region.
2. Concurrent Executions has no usage data point available in AWS API, but the limit can be increased upon request to AWS, as with many other limits. For most developers, companies, 1000 is a lot. However, there are some solutions where 1000 doesn't cut it, I'm working on two of those. We have accounts with this limit increased, but to be safe, we have accounts without it for testing purposes so cost doesn't escalate for those purposes.

To understand UnreservedConcurrentExecutions and ConcurrentExecutions, which will always be the same regardless if you have been granted a limit increase, let me quote AWS docs.

Reserved vs. Unreserved Concurrency Limits

If you set the concurrent execution limit for a function, the value is deducted from the unreserved concurrency pool. For example, if your account's concurrent execution limit is 1000 and you have 10 functions, you can specify a limit on one function at 200 and another function at 100. The remaining 700 will be shared among the other 8 functions.

Note

AWS Lambda will keep the unreserved concurrency pool at a minimum of 100 concurrent executions, so that functions that do not have specific limits set can still process requests. So, in practice, if your total account limit is 1000, you are limited to allocating 900 to individual functions.

Read more about AWS Lambda limits here in general, and AWS Lambda concurrency limits in particular here.

If you have any further questions, don't hesitate.

[kilsbo]

PR submitted, please have a look!

[jantman]

This was resolved by #366

[jantman]

This has been released in 6.0.0 and is now live on PyPI. Thank you so much!