fix(lambda-python): asset hash is non-deterministic

[eladb]

When a Python handler uses external dependencies, the hash calculated on the output is non-determinstic due to the fact that it includes timestamps. To resolve that, change the asset hash strategy to SOURCE which means that the bundle will only be re-created if one of the source files changes.

Additionally, the Python image hash, which is also included in the asset hash (to ensure that if the build image changes, the bundle is invalidated) included the absolute path for the Dockerfile. This caused the image hash itself to change every time the image was built on a different system. To fix this, we stage the dependency files & dockerfile into a temp directory and build the image from there.

Fixes #12770
Fixes #12684

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[gitpod-io]

[iliapolo]

This means effectively forcing users to modify their requirements in order to push an updated version of a dependency. Maybe make this configurable?

[eladb]

This means effectively forcing users to modify their requirements in order to push an updated version of a dependency. Maybe make this configurable?

What do you mean by configurable? What kind of API would you imagine? deployOnEveryUpdate? This conflicts with one of the foundational tenets we have in the CDK which says that only when the source changes your app changes. We have multiple places in our tooling (CLI/CDK pipelines) which rely on that tenet.

Technically, they don't need to modify requirements.txt, just to change something in their source. So they can just add a comment somewhere and bump a random value in order for their bundle to be recreated.

Let me know how do you propose to model this.

[iliapolo]

This conflicts with one of the foundational tenets we have in the CDK which says that only when the source changes your app changes.

Yeah, and I'm all for it. But we do have this option of using OUTPUT, which presumably addresses changes in dependencies (?). I'm just wondering if we should address the use case of users explicitly not locking their dependencies, expecting them to update on build, which is what happens now.

As far as API, yeah something like deployOnEveryUpdate, or maybe even straight up expose hashType for advanced users. I can try to come up with something less awkward if you choose to pursue it. But i'm also good with using SOURCE for now and possibly extending later.

Added do-not-merge.

[fsmanuel]

@eladb thanks for working on this!
@iliapolo I can partially understand your concerns. As I tried to explain in #12770 I'm worried about the current side effects.

I think the best solution would be to allow to set assetHashType via PythonFunctionProps and PythonLayerVersionProps. We can stick to the current behavior but make it possible to change it to SOURCE if desired.

[eladb]

I'll add a way to control this, but I think the default should be SOURCE

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
• Commit ID: c81557b
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).