Implement Layer support for .NET Core Lambda Functions

[Kralizek]

Hi,

One of the announcements at re:Invent was the introduction of layers in Lambda.

Whilst it's pretty easy to see the fit in dynamic runtimes like node.js and python, I wonder how applicable is this new feature to a build-ahead runtime like .NET Core.

I can imagine something like dynamic binding of any assembly present in /opt but the list of useful usages of dynamic binding is pretty short in .NET Core (as opposed to .NET Framework where assemblies might be referenced in the app.config file)

[normj]

What I have been thinking of for .NET Core is we use layers to publish runtime package stores. Our deployment tooling can add an additional runtimeconfig.dev.json file to the bundle that adds the /opt/ to the list of package stores to search through by the .NET Core runtime. The trick on the developer side is they would need to provide the runtime package store manifest to our deployment tool that could be passed down to the dotnet publish command so that the packages in the runtime package store are not included.

[ThomasJaeger]

Layers are very important for .NET Core developers who need/want to design a large scale SaaS system that includes a domain based on DDD principles and especially with CQRS and EventSource (ES) system. Lambdas are only the entry points into the core domain in such a system similar to the idea of an application service entry point in an ES system.

This means the domain (the domain .NET assemblies) will need to be shared between many lambdas which I have been forced to do. This is a pain and I see layers as an extreamly important step towards better maintability and consitency if we can just somehow make use of it.

There are currently no examples in the documentation for C# (https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html). This is very disappointing. I'm still not clear how this could be done with statically linked assemblies in C#. A simple example would help a lot.

[andymac4182]

Is there a chance you will release your layers so that the community can help support newer runtimes?

[normj]

@andymac4182 Open sourcing our Lambda runtime is something we are looking into. Our runtime is using the pre BYOR (Bring your own runtime) APIs so before we open source it we need to convert it. I can't promise when we will be able to release it but it is high on our priority list.

[normj]

I have finally cleared some space on my worklog and I'm getting back to Layers.

Does anybody have any issues with our tooling using the description of the layer to store metadata needed for publishing? With the approach of using .NET runtime package stores I need at the time of function publishing the manifest of the NuGet packages in the layer as well as what sub directory under /opt directory the runtime package store will be hosted at. I was thinking of putting a JSON document in the description of the layer identifying the directory and a pointer to S3 for the manifest.

[normj]

Moving issue to aws-extensions-for-dotnet-cli repo where this will be implemented.

[normj]

For those tracking this feature the work is being done in the lambda-layers branch.

The design is written up here

If you want to try the work in progress follow these steps to build and install from source:

• Clone Repo
• checkout lambda-layers branch
• Execute the following command at the repo root
  • dotnet msbuild buildtools/build.proj /t:build-nuget-packages
• Uninstall current tool
  • dotnet tool uninstall -g amazon.lambda.tools
• Install the newly built package
  • dotnet tool install -g --add-source ./Deployment/nuget-packages/ Amazon.Lambda.Tools --version 3.2.0-beta1

Current status is the publish-layer, deploy-function and package command have been updated to support layers. The deploy-serverless and package-ci commands need to be updated. I would also like to add some new commands to view and inspect the available layers.

Appreciate any feedback. Keep in mind this is still a work and process and the design is open to change.

[bjorg]

Very exciting! This will be a huge win for projects with a large number of common assemblies between Lambda functions.

[ThomasJaeger]

Thank you @normj . So far, we have created our own NuGet server to solve the problem with layers and statically linked libraries. NuGet packages have addressed our need to share core code such as the domain model of a large system and using lambdas as only entry points into the core.

I'm still not clear what layers could achive for statically linked libraries that NuGet packages do not address. Would you say Layers operate conceptually the same as NuGet packages?

[normj]

The way I have been working on Layers for .NET is it essentially provides a cache of NuGet packages to your Lambda functions. So when you deploy your Lambda functions they won't have to include the NuGet packages with you function.

For your local development though you will still need access to the NuGet packages since MSBuild will not be able to resolve the dependencies via a Lambda layer. Although that would be cool.