Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(asset): lambda asset bundle being moved #14369

Closed
strongishllama opened this issue Apr 25, 2021 · 4 comments · Fixed by #14372
Closed

(asset): lambda asset bundle being moved #14369

strongishllama opened this issue Apr 25, 2021 · 4 comments · Fixed by #14372
Assignees
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@strongishllama
Copy link

Bunding my Go Lambda function with Docker and having the output of the bundle a zip works the first time, but subsequent bundles fail. I've had a dig through the source code and I believe it's because the bundled zip file is being moved after it is created (see here).

Below is the code being used to bundle and create the Lambda function. One thing to note, the ouputType property needs to be set to cdk.BundingOutput.ARCHIVED because Windows 10 will strip the execute permission off the binary if it isn't zipped (see here).

new lambda.Function(props.scope, props.id, {
    runtime: lambda.Runtime.GO_1_X,
    code: lambda.Code.fromAsset(path.join(__dirname, "../"), {
      assetHashType: cdk.AssetHashType.OUTPUT,
      bundling: {
        image: lambda.Runtime.GO_1_X.bundlingImage,
        command: [
          "bash", "-c", [
            `cd ${props.sourceDir}`,
            `GOOS=linux go build -o main -ldflags="-s -w"`,
            // Zip file while in Docker container so file permissions
            // aren't lost when deploying with Windows 10.
            "zip /asset-output/main.zip main",
            "rm main"
          ].join(" && ")
        ],
        user: "root",
        outputType: cdk.BundlingOutput.ARCHIVED
      }
    }),
    handler: "main"
  });

Reproduction Steps

I've attached a simple CDK project that demonstrates this issue. Simply unzip the attached project and run either cdk synth or cdk deploy twice. The first time should succeed and the second time should fail with the following error message. Error: Bundling output directory is expected to include only a single .zip or .jar file when `output` is set to `ARCHIVED`

What did you expect to happen?

The Lambda source code to be successfully bundled both times.

What actually happened?

The Lambda source code failed to be bundled the second time. Returning the following error message Error: Bundling output directory is expected to include only a single .zip or .jar file when `output` is set to `ARCHIVED`

Environment

  • CDK CLI Version: 1.100.0 (build d996c6d)
  • Framework Version: 1.100.0
  • Node.js Version: v14.16.0
  • OS : Ubuntu 20.04 LTS
  • Language (Version): v4.1.3

This is 🐛 Bug Report

@strongishllama strongishllama added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 25, 2021
@github-actions github-actions bot added @aws-cdk/aws-lambda Related to AWS Lambda @aws-cdk/core Related to core CDK functionality labels Apr 25, 2021
@jogold
Copy link
Contributor

jogold commented Apr 26, 2021

Confirmed bug. The second time bundling runs the bundling directory already exists so bundling is correctly skipped but in this case it cannot determine if bundling resulted in a single archive file or not because the bundling directory is empty.

Working on a fix.

jogold added a commit to jogold/aws-cdk that referenced this issue Apr 26, 2021
…archive is skipped

Bundling is skipped when the bundling directory already exists on disk
(cache). If it previously produced a single archive file that has been
moved to the staging directory the current logic either failed to
determine the correct packaging type or threw because it expected the
bundling directory to include a single archive file.

Fix it by "touching" the archive file in the bundling directory after it
has been moved to the staging directory.

Fixes aws#14369
@nija-at nija-at removed their assignment May 5, 2021
@nija-at nija-at added @aws-cdk/assets Related to the @aws-cdk/assets package and removed @aws-cdk/aws-lambda Related to AWS Lambda @aws-cdk/core Related to core CDK functionality labels May 5, 2021
@nija-at nija-at assigned eladb and unassigned rix0rrr May 5, 2021
@nija-at nija-at changed the title (core): lambda asset bundle being moved (asset): lambda asset bundle being moved May 5, 2021
@github-actions github-actions bot added the @aws-cdk/aws-lambda Related to AWS Lambda label May 5, 2021
@nija-at nija-at removed the @aws-cdk/aws-lambda Related to AWS Lambda label May 5, 2021
@nija-at nija-at removed their assignment May 5, 2021
@mergify mergify bot closed this as completed in #14372 May 18, 2021
mergify bot pushed a commit that referenced this issue May 18, 2021
…archive is skipped (#14372)

Bundling is skipped when the bundling directory already exists on disk
(cache). If it previously produced a single archive file that has been
moved to the staging directory the current logic either failed to
determine the correct packaging type or threw because it expected the
bundling directory to include a single archive file.

Fix it by "touching" the archive file in the bundling directory after it
has been moved to the staging directory.

Fixes #14369


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Aug 26, 2021
…archive is skipped (aws#14372)

Bundling is skipped when the bundling directory already exists on disk
(cache). If it previously produced a single archive file that has been
moved to the staging directory the current logic either failed to
determine the correct packaging type or threw because it expected the
bundling directory to include a single archive file.

Fix it by "touching" the archive file in the bundling directory after it
has been moved to the staging directory.

Fixes aws#14369


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@saga95
Copy link

saga95 commented May 31, 2023

I was using pnpm and esbuild and got this error. Installing pnpm globally resolved me the issue

@hoegertn
Copy link
Contributor

@jogold I hit a new bug very similar to this one. The touching of the file leads to errors of the hashType is OUTPUT.

The second build will skip bundling but create a new hash for the asset because of the now empty file and then the asset will be moved again but empty.

My suggestion is to remove the temporary bundleDir in that case instead of touching the file. Any objections?

I will create a new PR for this just wanted to note it down somewhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
7 participants