aws-lambda-swift-sprinter

The goal of this project is to provide an environment to build an AWS Lambda Custom Runtime for the Swift programming language and provide the required support to run swift-nio 2.0.

The project helps building a Swift Lambda based on the framework LambdaSwiftSprinter.

The project contains also some Examples:

HelloWorld: A basic Lambda Swift example
HTTPSRequest: A basic example showing how to perform an HTTPS request from the Swift Lambda using the LambdaSwiftSprinterNioPlugin
S3Test: A basic example showing how to access an S3 bucket from the Swift Lambda using https://github.com/swift-aws/aws-sdk-swift.
RedisDemo: A basic example showing how to connect to Redis from the Swift Lambda using the RediStack client.
PostgreSQLDemo: A basic example showing how to connect to Redis from the Swift Lambda using the PostgresNIO client.

Introduction

The AWS Lambdas run on Amazon Linux.

To build swift on Amazon Linux is required to:

building the code on the official Docker Swift
extracting the build and all the runtime's shared libraries
packaging the artefacts and use them as AWS Lambda Custom Runtime

The artefacts required to run the Swift Lambda are split in two parts:

Lambda Layer: A layer is a ZIP archive that contains libraries, a custom runtime, all the shared libraries required to run the Swift lambda.
Lambda code depends on the LambdaSwiftSprinter framework: A zip package containing the Swift Lambda main executable and third-party libraries.

Lambda Development Workflow

1) Requirements

Install Docker from here
Clone this repository. From the command line type:

git clone https://github.com/swift-sprinter/aws-lambda-swift-sprinter
cd aws-lambda-swift-sprinter

Ensure you can run make:

make --version

the Makefile was developed with this version:

GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

2) Prepare a custom docker image

The docker image swift:latest contains the latest release of Swift. This image does not contain the libraries libssl-dev and libicu-dev required to build swift-nio.

The Dockerfile contains the recipe to build a custom docker image based on the swift:latest with the addition of the swift-nio requirements.

To build the image use the command:

make docker_build

This will build and tag a local docker image called nio-swift:latest.

It's possible to check it by using the following command:

docker image ls

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nio-swift           latest              b2b9d0b2a68d        8 days ago          1.45GB
swift               latest              1e9a2a744b48        13 days ago         1.35GB

3) Build the lambda layer

To prepare the AWS Lambda layer containing all the swift runtime libraries and the bootstrap file:

make package_layer

The output of this command is a ZIP file called swift-lambda-runtime.zip under the folder .build

4) Write the lambda code

Here a basic example of the Lambda code:

import Foundation
import LambdaSwiftSprinter

struct Event: Codable {
    let name: String
}

struct Response: Codable {
    let message: String
}

let syncLambda: SyncCodableLambda<Event, Response> = { (event, context) throws -> Response in
    let message = "Hello World! Hello \(event.name)!"
    return Response(message: message)
}

//...

do {
    let sprinter = try SprinterCURL()
    sprinter.register(handler: "helloWorld", lambda: syncLambda)
    try sprinter.run()
} catch {
    log(String(describing: error))
}

More details on how to code a Swift Lambda are documented under the Examples folder:

Refer to the LambdaSwiftSprinter framework documentation to know more.

5) Build the lambda

By default, the Makefile will build the HelloWorld example contained in this repository.

The following command will build and zip the Lambda:

make package_lambda

The output of this command is a ZIP file called lambda.zip under the folder .build.

Lambda Configuration

Parametrized calls / Automation

Can pass values to your make command either before or after the call like below. This will fit better in a script per example to set up continuous integration in your project.

Before `make`

SWIFT_EXECUTABLE=HTTPSRequest \
SWIFT_PROJECT_PATH=Examples/HTTPSRequest \
LAMBDA_FUNCTION_NAME=HTTPSRequest \
LAMBDA_HANDLER=${SWIFT_EXECUTABLE}.getHttps \
    make invoke_lambda

After `make`

make invoke_lambda \
    SWIFT_EXECUTABLE=HTTPSRequest \
    SWIFT_PROJECT_PATH=Examples/HTTPSRequest \
    LAMBDA_FUNCTION_NAME=HTTPSRequest \
    LAMBDA_HANDLER=HTTPSRequest.getHttps

Parameters you can pass

Key	Usage	Default
AWS_PROFILE	An AWS AIM profile you create to authenticate to your account.	default
IAM_ROLE_NAME	The execution role created that will be assumed by the Lambda.	lambda_sprinter_basic_execution
AWS_BUCKET	The AWS S3 bucket where the layer and lambdas zip files get uploaded.	aws-lambda-swift-sprinter
SWIFT_VERSION	Version of Swift used / Matches Dockerfile location too from `docker/` folder.	5.2.3
LAYER_VERSION	Version of the Swift layer that will be created and uploaded for the Lambda to run on.	5-2-3
DOCKER_OS	amazonlinux2 / xenial	amazonlinux2
SWIFT_EXECUTABLE	Name of the binary file.	HelloWorld
SWIFT_PROJECT_PATH	Path to your Swift project.	Examples/HelloWorld
LAMBDA_FUNCTION_NAME	Display name of your Lambda in AWS.	HelloWorld
LAMBDA_HANDLER	Name of your lambda handler function. If you declare it using `sprinter.register(handler: "FUNCTION_NAME", lambda: syncLambda)` you should declare it as `<SWIFT_EXECUTABLE>.<FUNCTION_NAME>`.	$(SWIFT_EXECUTABLE).helloWorld

Manual change

You can also edit the Makefile to build a different Example by commenting the following lines and uncommenting the line related to the example you want to build.

...

SWIFT_EXECUTABLE?=HelloWorld
SWIFT_PROJECT_PATH?=Examples/HelloWorld
LAMBDA_FUNCTION_NAME?=HelloWorld
LAMBDA_HANDLER?=$(SWIFT_EXECUTABLE).helloWorld

...

Lambda Deployment Workflow

The following tutorial describes how to deploy the lambda in your AWS account from the command line using AWS Command Line Interface through the Makefile.

The goal of the deployment is to create and configure a lambda with lambda.zip and the layer with swift-lambda-runtime.zip.

Steps:

Create a Lambda layer with the swift-lambda-runtime.zip
Create the lambda with the lambda.zip code and the correct configuration
- Name
- Runtime
- Handler
- Execution role
- Function code
- Test event

There are many ways to achieve a lambda deployment (AWS Console, SAM, CloudFormation ...), please refer to the latest AWS Lambda documentation to know more.

Requirements

an AWS account for test purpose.
aws cli: Install the aws cli. Here the instructions.
If you want to deploy your lambdas and layers using S3 you need to make sure the bucket in the Makefile already exists. If it doesn't you can create it using the command make create_s3_bucket which will use the value of the variable AWS_BUCKET as a name.
if your AWS account it doesn't have admin priviledges:
- Review(*) the policy contained in the file LambdaDeployerPolicyExample.json
- Attach the policy to your user. Here is the official documentation.

(*) Note: It would be better to restrict the policy to the function and layer you want to create. It's suggested to test the following scripts before using in production.

1) Upload the Lambda layer

Create a new lambda layer using the swift-lambda-runtime.zip file. This step is required once, however you are free to run it if you need to update your layer.

Upload the Lambda using S3

make upload_lambda_layer_with_s3

Datetime based versions are created and uploaded to S3 every time your version is created.

Upload the Lambda directly

make upload_lambda_layer

2) Create the Lambda

You can create a new lambda which might take a few minutes using one of the options below:

Create the Lambda using S3

make create_lambda_with_s3

Datetime based versions are created and uploaded to S3 every time your version is created.

Create the Lambda directly

make create_lambda

The lambda is created with the following parameters:

function-name: $(LAMBDA_FUNCTION_NAME)
- HelloWorld
runtime: provided
handler: $(LAMBDA_HANDLER)
- HelloWorld.helloWorld
role: "$(IAM_ROLE_ARN)"
- a new role will be created with the name: lambda_sprinter_basic_execution
zip-file: $(LAMBDA_BUILD_PATH)/$(LAMBDA_ZIP) --> ./build/lambda/lambda.zip
layers: $(LAMBDA_LAYER_ARN)
- it will use the arn contained in the file generated by the make upload_lambda_layer

This step is required once, if you need to update the lambda use the step 5.

4) Invoke the Lambda

Now the lambda function is ready for testing. The following command invokes the lambda with using the file event.json contained in the project folder.

make invoke_lambda

The output:

{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}

Result:

{"message":"Hello World! Hello Swift-Sprinter!"}

Note:

The lambda invocation may require some policy to access other AWS Resources. Check the S3Test example to know more.

5) Update the Lambda (optional)

If needed, you will also be able to update your Lambda using one of the commands below:

Update the Lambda using S3

make update_lambda_with_s3

Update the Lambda directly

make update_lambda

6) Nuclear option

Sometimes you want to go back to a clean slate and we have a command for that which will rely on the parameters you use.

make nuke

That command will clean your local build folders then delete lambdas and layers created based off the configuration you use.

7) Delete S3 bucket

You can delete the S3 bucket where you hold your lambdas and layers with the command below:

make delete_s3_bucket

Update the lambda layer

The docker image could require security updates or could require updates.

To patch the lambda with a new layer:

Rebuild the docker image

make docker_build

package the layer

make package_layer

upload the layer to AWS

make upload_lambda_layer

update the lambda

make update_lambda

Local development

Development and test can be simplified by using a sandboxed local environment replicating of the AWS Lambda by using the lambci/lambda:provided Docker image.

The local sandboxed Lambda can be launched just to test the Lambda once or more than once.

To run the sandboxed Lambda environment it's required to share the files contained in the layer, the boostrap and the files required by the build with the lambci/lambda:provided Docker image.

docker run --rm \
-v "$(PWD)/$(LOCAL_LAMBDA_PATH)":/var/task:ro,delegated \
-v "$(PWD)/bootstrap":/opt/bootstrap:ro,delegated \
-v "$(PWD)/$(SHARED_LIBS_FOLDER)":/opt/swift-shared-libs:ro,delegated \
lambci/lambda:provided $(LAMBDA_HANDLER) $(LOCAL_LAMBDA_EVENT)

$LOCAL_LAMBDA_PATH: the path containing the build
$SHARED_LIBS_FOLDER: the path containing the extracted swift runtime libraries
$LAMBDA_HANDLER: the lambda handler
$LOCAL_LAMBDA_EVENT: the event JSON string

To run the Lambda locally follow the Lambda development workflow to the step 4:

Requirements: Clone the repository and install Docker
Prepare a custom docker image: make docker_build
Build the lambda layer: make package_layer
Write the lambda code

To simplify the local execution swift-sprinter added some useful commands:

Lambda Local

Build lambda locally

Build the lambda and copy the artefacts under $LOCAL_LAMBDA_PATH

make build_lambda_local

Invoke lambda local once

Invoke the Lambda locally once with Docker and LambCI on port 9001

make invoke_lambda_local_once

Start lambda local environment

Start the local environment with Docker and LambCI on port 9001

make start_lambda_local_env

Keep it running and open a new terminal window to invoke the lambda.

Use ctrl^C to stop it.

Invoke the lambda locally

Invoke the lambda locally using the endpoint http://localhost:9001 with the aws cli.

make invoke_lambda_local

Use of Docker Compose

To test lambda locally with a more complex environment it's possible to use Docker Compose.

All the examples in the repository have docker-compose.yml file to run the example locally listening on the port 9001.

Start the Docker Compose

Start the docker-compose test environment

make start_docker_compose_env

Invoke the lambda locally

Invoke the lambda locally using the endpoint http://localhost:9001 with the aws cli.

make invoke_lambda_local

Stop the Docker Compose

Stop the docker-compose test environment

make stop_docker_compose_env

Manual deployment from AWS Console

Requirements:

Use MacOS or Linux
Install Docker
Clone the repo git clone https://github.com/swift-sprinter/aws-lambda-swift-sprinter.git

Build:

From the command line run make docker_build
From the command line run make package_layer
From the command line run make package_lambda

Configure:

Go to AWS Lambda -> Layers in AWS Console and create a new layer from scratch
Enter layer name "swift-lambda-runtime-5-1-5"
Upload the zip file build/swift-lambda-runtime-5-1-5.zip
Leave "Compatible runtimes" empty.
Click "Create"
Copy the arn from the created layer, it's required to set up the lambda.
Go to AWS Lambda in AWS Console and create a new function from scratch
Enter function name "benchmark-swift-hello" and select "Provide your own bootstrap" as runtime
Choose the execution role created above
Upload the zip file created with function code, it can be found under build/lambda.zip
Input "HelloWorld.helloWorld" in "Handler" (Executable.Handler)
Click "Layers"
Click "Add Layer"
Click "Provide a layer version"
Add the arn of the layer you have uploaded previously.
Click "Add"
Click "Save"

Test:

Test the function by clicking Test in the top right corner and add the following event:

{
    "name": "Swift-Sprinter"
}

Give a name to the event, save and then test it.

Contributions

Contributions are more than welcome! Follow this guide to contribute.

Acknowledgements

This project has been inspired by the amazing work of the following people:

Matthew Burke, Capital One : https://medium.com/capital-one-tech/serverless-computing-with-swift-f515ff052919
Justin Sanders : https://medium.com/@gigq/using-swift-in-aws-lambda-6e2a67a27e03
Claus Höfele : https://medium.com/@claushoefele/serverless-swift-2e8dce589b68
Kohki Miki, Cookpad : https://github.com/giginet/aws-lambda-swift-runtime
Toni Sutter : https://github.com/tonisuter/aws-lambda-swift
Sébastien Stormacq : https://github.com/sebsto/swift-custom-runtime-lambda

A special thanks to BJSS to sustain me in delivering this project.

This project has been awarded AWS Open Source Promotional Credits for 2020. @AWSOpen

Name		Name	Last commit message	Last commit date
Latest commit History 138 Commits
.github/ISSUE_TEMPLATE		.github/ISSUE_TEMPLATE
Examples		Examples
docker		docker
images		images
scripts		scripts
.gitignore		.gitignore
.swift-version		.swift-version
.swiftformat		.swiftformat
.swiftlint.yml		.swiftlint.yml
.travis.yml		.travis.yml
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE.txt		LICENSE.txt
LambdaDeployerPolicyExample.json		LambdaDeployerPolicyExample.json
Makefile		Makefile
README.md		README.md

License

Andrea-Scuderi/aws-lambda-swift-sprinter

Folders and files

Latest commit

History

Repository files navigation

aws-lambda-swift-sprinter

Introduction

Lambda Development Workflow

1) Requirements

2) Prepare a custom docker image

3) Build the lambda layer

4) Write the lambda code

5) Build the lambda

Lambda Configuration

Parametrized calls / Automation

Before make

After make

Parameters you can pass

Manual change

Lambda Deployment Workflow

Requirements

1) Upload the Lambda layer

Upload the Lambda using S3

Upload the Lambda directly

2) Create the Lambda

Create the Lambda using S3

Create the Lambda directly

4) Invoke the Lambda

5) Update the Lambda (optional)

Update the Lambda using S3

Update the Lambda directly

6) Nuclear option

7) Delete S3 bucket

Update the lambda layer

Local development

Lambda Local

Build lambda locally

Invoke lambda local once

Start lambda local environment

Invoke the lambda locally

Use of Docker Compose

Start the Docker Compose

Invoke the lambda locally

Stop the Docker Compose

Manual deployment from AWS Console

Requirements:

Build:

Configure:

Test:

Contributions

Acknowledgements

About

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

Packages 0

Languages

Before `make`

After `make`

Packages