Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve SDK cold-start latency #6

Closed
millems opened this issue Jun 29, 2017 · 13 comments
Closed

Improve SDK cold-start latency #6

millems opened this issue Jun 29, 2017 · 13 comments
Labels
2.0 New feature-request A feature should be added or improved.

Comments

@millems
Copy link
Contributor

millems commented Jun 29, 2017

With the rise of serverless frameworks, library's cold-start latency has become increasingly important to minimize. Libraries that rely heavily on reflection, class-path scanning, needlessly-large thread pools, etc. greatly increase the time it takes to get a lambda JVM to a state that it can process requests.

Cold-start latency of the SDK should be benchmarked and expensive libraries (I'm looking at you, Jackson databind) should be moved out of the critical startup path.

@cherrydev
Copy link

Yep! I put together a very basic Lambda that handles an S3 Event, performs a couple operations on S3 objects and then puts some messages on an SQS queue. Using the default Lambda timeouts of 6s I get timeouts. It turns out first-time invocation actually takes about 7.5s. Doing some experimentation with different data serializers leads me to attribute most of this to Jackson.
The data serializer library that had the fastest startup times was square/moshi, which had startup times that were about 10% of Jackson's when it was using the reflection serializers and 7% when using compile-time generation.
I realize that changing data serializer libraries at this point is maybe not feasible.
For reference, a cold invocation of the same lambda written in node.js was 1100ms.

@millems
Copy link
Contributor Author

millems commented Jul 28, 2018

We've found big improvements with Jackson-jr as well. Once we can switch over to it for region metadata loading, we'll be able to get using Jackson off of our startup path.

We'll still be using it for unmarshalling JSON exceptions, but removing that is also on our pre-GA plan.

@lwis
Copy link

lwis commented Sep 17, 2018

@millems do you have any progress share regarding this? I'm amongst many users that are currently suffering unacceptable cold start times for serverless.

@spfink
Copy link
Contributor

spfink commented Sep 17, 2018

@lwis, have you tested with the latest version? Configured correctly, it should show improvement over v1.

To minimize cold start you should:

  • Use the UrlConnectionHttpClient
  • Specify a region
  • Specify the environment variables credential provider

We are still working to remove Jackson from two more places which should give better cold start times once done.

@lwis
Copy link

lwis commented Sep 17, 2018

@spfink thanks for the response!

Something like;

SnsClient.builder()
            .region(Region.EU_WEST_2)
            .httpClient(UrlConnectionHttpClient.builder().build())
            .credentialsProvider(EnvironmentVariableCredentialsProvider.create())
            .build()

I'm unfortunately still seeing 3.5s request to response 😞

Edit: I'm aware that UrlConnection eats up to a second of that time, and network maybe accounts for a small chunk.

@spfink
Copy link
Contributor

spfink commented Sep 19, 2018

What size of a Lambda are you getting 3.5 seconds on?

Removing the last few Jackson use cases should help us cut another second or so off.

@lwis
Copy link

lwis commented Sep 19, 2018

@spfink 1536MB. I can put together a little test project if that's helpful?

@Sandmania
Copy link

This is very promising. We are seeing over 50% improvement compared to the old lambda client when initializing the new like so:

LambdaClient.builder()
                .region(Region.EU_CENTRAL_1)
                .httpClient(UrlConnectionHttpClient.builder().build())
                .build();

with sdk v2 and a lambda with 1024MB of memory that takes around 1000ms. Instantiating the old one with AWSLambdaClientBuilder.standard().withRegion("eu-central-1").build() and the same amount of memory takes about 2300ms.

@millems
Copy link
Contributor Author

millems commented Nov 7, 2018

Glad that you're seeing some of the benefits already! We still have a few more changes planned, so hopefully we'll be able to get it even lower.

@spfink
Copy link
Contributor

spfink commented Nov 8, 2018

@Sandmania which version of the SDK are you using and seeing those results on?

@Sandmania
Copy link

@spfink this is with 2.0.0-preview-12 vs 1.11.443. I'm guessing most of the improvement comes from using UrlConnectionHttpClient, as just instantiating the AmazonHttpClient in 1.11.443 (which I think relies on ApacheHttpClient?) takes 1700ms.

@davidmoten
Copy link

see #2527 for details of cold-start improvements with a lightweight java client.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.0 New feature-request A feature should be added or improved.
Projects
None yet
Development

No branches or pull requests

8 participants