(aws-lambda-python): Using poetry requirements results in requirements.txt written to 'entry' directory

[huonw]

What is the problem?

We're starting to use PythonFunction from @aws-cdk/aws-lambda-python-alpha with poetry (a pyproject.toml and poetry.lock file in the directory specified by entry), and it works pretty well, thanks.

However, the exported requirements.txt is written to the asset input volume directory, meaning it ends up written to the entry directory next to pyproject.toml, which is a bit unexpected. This requires configuring .gitignore correctly, and also may impact the fingerprinting procedure.

aws-cdk/packages/@aws-cdk/aws-lambda-python/lib/packaging.ts

Line 49 in 6dfc254

exportCommand: `poetry export --with-credentials --format ${DependenciesFile.PIP} --output ${DependenciesFile.PIP}`,

(Looking at the code there, this likely applies to pipenv too)

Reproduction Steps

import * as cdk from 'aws-cdk-lib';
import * as lambdaPython from '@aws-cdk/aws-lambda-python-alpha';

export const app = new cdk.App();
const stack = new cdk.Stack(app, 'Stack', {});

new lambdaPython.PythonFunction(stack, 'Func', {
  entry: 'entry',
  index: 'entry/__init__.py',
  runtime: cdk.aws_lambda.Runtime.PYTHON_3_9,
});

Directory tree of entry (before running anything):

entry
├── entry
│   └── __init__.py
├── poetry.lock
└── pyproject.toml

poetry.lock:

package = []

[metadata]
lock-version = "1.1"
python-versions = "*"
content-hash = "115cf985d932e9bf5f540555bbdd75decbb62cac81e399375fc19f6277f8c1d8"

[metadata.files]

pyproject.toml:

[tool.poetry]
name = "entry"
version = "0.1.0"
description = ""
authors = []

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

__init__.py: empty

What did you expect to happen?

Temporary files shouldn't be written to disk (or at least, not intermixed with real source). For instance, the generated requirements.txt could be placed into any other directory in the docker image and this probably won't occur.

What actually happened?

After running cdk synth, a (empty, in this case) requirements.txt file appears:

entry
├── entry
│   └── __init__.py
├── poetry.lock
├── pyproject.toml
└── requirements.txt

CDK CLI Version

2.14.0 (build 762d71b)

Framework Version

No response

Node.js Version

v14.17.5

OS

macOS

Language

Typescript

Language Version

TypeScript 4.1.6

Other information

No response

[corymhall]

@huonw this might be a simple fix. Right now we run all our commands and then copy everything to the output directory. We might want to just copy first and then run the commands in the output directory.

I am unassigning and marking this issue as p2, which means that we are unable to work on this immediately, but would definitely welcome contributions.

We use +1s to help prioritize our work, and are happy to revaluate this issue based on community feedback. You can reach out to the cdk.dev community on Slack to solicit support for reprioritization.

[gshpychka]

@corymhall Would this be as simple as moving L98 to right after L93 here?

What would make sense as a test for this?

[corymhall]

@corymhall Would this be as simple as moving L98 to right after L93 here?

What would make sense as a test for this?

Yeah I think that would do it. I think we can update one of the integration tests to do an assertion that the function still executes successfully (see integ.bundling.ts as an example).

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[lrav35]

hello @corymhall, is this slated to be released soon? Not sure what your release cycle looks like.