-
Notifications
You must be signed in to change notification settings - Fork 855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve SDK cold-start latency #6
Comments
Yep! I put together a very basic Lambda that handles an S3 Event, performs a couple operations on S3 objects and then puts some messages on an SQS queue. Using the default Lambda timeouts of 6s I get timeouts. It turns out first-time invocation actually takes about 7.5s. Doing some experimentation with different data serializers leads me to attribute most of this to Jackson. |
We've found big improvements with Jackson-jr as well. Once we can switch over to it for region metadata loading, we'll be able to get using Jackson off of our startup path. We'll still be using it for unmarshalling JSON exceptions, but removing that is also on our pre-GA plan. |
@millems do you have any progress share regarding this? I'm amongst many users that are currently suffering unacceptable cold start times for serverless. |
@lwis, have you tested with the latest version? Configured correctly, it should show improvement over v1. To minimize cold start you should:
We are still working to remove Jackson from two more places which should give better cold start times once done. |
@spfink thanks for the response! Something like;
I'm unfortunately still seeing 3.5s request to response 😞 Edit: I'm aware that UrlConnection eats up to a second of that time, and network maybe accounts for a small chunk. |
What size of a Lambda are you getting 3.5 seconds on? Removing the last few Jackson use cases should help us cut another second or so off. |
@spfink 1536MB. I can put together a little test project if that's helpful? |
This is very promising. We are seeing over 50% improvement compared to the old lambda client when initializing the new like so:
with sdk v2 and a lambda with 1024MB of memory that takes around 1000ms. Instantiating the old one with |
Glad that you're seeing some of the benefits already! We still have a few more changes planned, so hopefully we'll be able to get it even lower. |
@Sandmania which version of the SDK are you using and seeing those results on? |
@spfink this is with 2.0.0-preview-12 vs 1.11.443. I'm guessing most of the improvement comes from using UrlConnectionHttpClient, as just instantiating the AmazonHttpClient in 1.11.443 (which I think relies on ApacheHttpClient?) takes 1700ms. |
see #2527 for details of cold-start improvements with a lightweight java client. |
With the rise of serverless frameworks, library's cold-start latency has become increasingly important to minimize. Libraries that rely heavily on reflection, class-path scanning, needlessly-large thread pools, etc. greatly increase the time it takes to get a lambda JVM to a state that it can process requests.
Cold-start latency of the SDK should be benchmarked and expensive libraries (I'm looking at you, Jackson databind) should be moved out of the critical startup path.
The text was updated successfully, but these errors were encountered: