aws_lambda_python_alpha: Using PIP_INDEX_URL and Codeartifact cause Docker to rebuild lambda every synth

[kcp-chewie]

Describe the bug

When following the instructions for using CodeArtifact, the lambda artifact is rebuilt anytime the CodeArtifact token changes, despite the actual code not changing at all.

Expected Behavior

The lambda shouldn't be rebuilt unless the actual code changes.

Current Behavior

The lambda is rebuilt with a new hash, which causes an unnecessary redeploy. This slows down the synth and deploy steps, and causes cold starts for all lambdas.

Reproduction Steps

Define any lambda function using the Custom Bundling with Code Artifact instructions. Observe that the lambda is rebuilt on each synth, even with no code changes.

Possible Solution

The Token included in the CodeArtifact URL should be ignored when computing the hash that determines if the lambda is rebuilt. Alternately, provide an option to ignore the PIP_INDEX_URL entirely.

Additional Information/Context

No response

CDK CLI Version

2.95.1

Framework Version

No response

Node.js Version

v20.6.1

OS

OSX 12.2

Language

Python

Language Version

Python 3.9.16

Other information

No response

[indrora]

I believe this is intentional due to how CodeArtifact handles these resources.

[kcp-chewie]

@indrora could you explain a bit more why this is intentional? It doesn't seem right that despite having the exact same code, the lambda gets rebuilt/redeployed.

The only thing that changes here is the CodeArtifact auth token, which doesn't seem like something that should be included in the lambda hash since it's temporary by definition!

[lpicchi]

Having same problem.
I think as a package remains same version it is not important if it has been downloaded from any arbitrary repo.
So i kindly ask what is the reason to include PIP_INDEX_URL in the lambda hash?
It should be fixable even if there is a reason to hash PIP_INDEX_URL, authentication details can be removed from URL before hashing.

[kcp-chewie]

I looked into this a bit more, and explored the fact that it's possible to pass in a custom assetHash instead of relying on the hash computed by the CDK code. Unfortunately, that won't work, as the bundling information (which includes the environment field, and thus the PIP_INDEX_URL) is included in 2 critical places, calculateHash and calculateCacheKey.

I'd be willing to take on the work here to fix this issue, but could use some guidance on the design first.

• Should this be configurable? Assuming this implementation will just stripping the auth details and not ignore the index entirely, there is a good argument that including it in the first place was a bug we should just fix.
• Should I just special case these variables directly in aws-cdk-lib/core/lib/asset-staging.ts? I honestly don't see any other great options but certainly open to suggestions!

[lpicchi]

I think the best option to go is the ability to provide custom repos urls as a bundler option. So the logic applied to build_args remains the same. So CDK could inject your repos in PIP_EXTRA_INDEX_URL or PIP_INDEX_URL and hash them using the correct strategy like credentials removal or not hashing them at all. I still can't find any utility why the repo url should be hashed. As mentioned in other comment certain package version should be the same in ever repository.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.