Build identical artifacts only once

[Dunedan]

Previously when running sam build one artifact would be built
per AWS Serverless Function. That caused the same artifacts getting
built over and over again during a single build build, when multiple AWS
Serverless Functions shared the same runtime and CodeUri.

With this commit, instead of building one artifact per AWS Serverless
Function, one is built for every combination of runtime and CodeUri
present in the SAM template.

This speeds up the build time dramatically when multiple functions share
the same runtime and CodeUri.

Checklist:

• Write unit tests
• make pr passes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[Dunedan]

Seems like fixing the integration tests would be some more effort, especially as there is no 1:1 mapping between function and artifacts anymore. I'm open to suggestions how to approach that best.

[sriram-mv]

This speeds up the build time dramatically when multiple functions share the same runtime and CodeUri

Can you explain the usecase here? if they are just the same functions. Cant you define once and !Ref it.

[Dunedan]

While the functions share the same runtime and CodeUri, they don't share the same handler. Other details like events, memory settings, etc. might differ as well. So it's not the same function which could be re-used throughout the template. The use case are artifacts which contain the code for multiple functions.

[sanathkr]

@Dunedan Hey this is a great start! I there are a few nuances that needs more thought & iteration:

• For languages like Golang, Handler also matters in addition to Runtime & CodeUri.
• We need to retain the output folder structure inside the build folder. The folder structure is a contract between build command & other commands. 1:1 mapping between artifact folder & resource simplifies the integration.

Would you mind updating the sam build design document with the changes you are making? It might help place changes in context. We can review the design real quick and then go towards fixing the implementation.

Thoughts?

[Dunedan]

Sounds great, but it might take a bit until I find time to get to it. 👍

[lbalceda]

@thesriram could you elaborate on your !Ref from other functions ? can you use !Ref for codeuri?

@Dunedan have you tried using layers? I wanted to explore that to see if it would help at all, since all of my functions have shared/similar artifacts.

[benkehoe]

I would be interested for this implementation to separate the build of a CodeUri from its eventual existence as a function-named directory under build. In particular, I am interested in getting SAM to have the capability for a function to have multiple CodeUris and get the overlay of those directories as the function code. This would be easier to implement later if, in this implementation, CodeUris got built as, say, build/codeuri/code_uri_id and then copied from there to build/MyFunctionName

[jfuss]

@benkehoe

In particular, I am interested in getting SAM to have the capability for a function to have multiple CodeUris and get the overlay of those directories as the function code.

Can you describe this more in depth? This PR might not be the right place to discuss, maybe in #1236, but am interested in what you are trying to solve.

[benkehoe]

I thought I had an issue open for it, but I am not able to find it if I do (I think potentially it's even a Lambda API request, but SAM CLI could solve it client-side). I have opened #1257 for this.

[jfuss]

We do want to enable this support but this needs further design before we can accept. sam build was built with the assumption that each Lambda Function should have its now artifact. This is why sam build will generate built artifacts in ./.aws-sam/build/<LogicalId>. This PR fundamentally changes this and will need updates to the design doc to ensure we understand the change fully and do not break any exiting customers or tools that generate artifacts in this format already.

For anyone finding this, we would encourage a design doc PR so that we can understand the change before implementing.

Closing this as we cannot accept in the current state.