[lambda] lambda.Code.FromAsset Golang bundling permission denied at execution

[strongishllama]

❓ General Issue

The Question

I've been trying to deploy a Lambda function written in Go using lambda.Code.fromAsset and the experimental BundlingOptions inside the AssetOptions parameter. I'm able to deploy the stack with no issues at all but I get a permission denied error on the Lambda when trying to execute it.

{
  "errorMessage": "fork/exec /var/task/hello-world: permission denied",
  "errorType": "PathError"
}

When I download the zipped source from S3 to look at the binaries permissions it should be able to execute it.

total 13808
drwxrwxrwx 1 strongishllama strongishllama     512 Oct  8 10:31 ./
drwxrwxrwx 1 strongishllama strongishllama     512 Oct  8 10:23 ../
-rwxrwxrwx 1 strongishllama strongishllama 9291301 Oct  8 10:23 hello-world*

Here's the Lambda function snippet from CDK.

    const handler = new lambda.Function(this, `HelloWorld-${props.suffix}`, {
      runtime: lambda.Runtime.GO_1_X,
      code: lambda.Code.fromAsset("assets/lambdas/hello-world", {
        bundling: {
          image: lambda.Runtime.GO_1_X.bundlingDockerImage,
          command: [
            "bash", "-c", [
              "go build -o /asset-output/hello-world*.go",
              "chmod +x /asset-output/hello-world"
            ].join(" && ")
          ],
          user: "root"
        }
      }),
      handler: "hello-world"
    });

I've also tried building and zipping locally then uploading it manually to the existing Lambda function and it works as expected.

Any help would be appreciated. Thanks!

Environment

• CDK CLI Version: 1.67.0 (build 2b4dd71)
• Module Version: 1.67.0
• Node.js Version: v14.10.1
• OS: Windows 10 Home
• Language (Version): TypeScript (4.0.3)

[strongishllama]

I just tried this on my Ubuntu machine and it works fine, the environment is as follows.

• CDK CLI Version: 1.67.0 (build 2b4dd71)
• Module Version: 1.67.0
• Node.js Version: v10.19.0
• OS: Ubuntu 20.04.1 LTS
• Language (Version): TypeScript (3.9.7)

[NGL321]

Hi @strongishllama!

Unfortunately I WAS able to reproduce this on Windows and had no issues on OSX, so I think this is likely a bug with the way asset bundling works on Windows. I have changed the labels so that it gets the attention of the relevant dev.

Are you able to run your builds on Ubuntu as mentioned above while this is pending? If not, the best other workaround would be to create a small secondary pipeline stack in CodePipeline to run the deployment from a Codebuild unix environment.

Let me know if that workaround works for you!

😸 😷

[strongishllama]

My primary development device is Ubuntu so all good, but I'll keep that workaround in mind. Thanks!

[pfried]

Related #7749

The issue ( most likely only on windows) is that files which are bundled will loose their permissions (like beeing executable). As long as seen from inside the container the permissions are correctly set within the asset-output directory.

Inside the container:

After running the cdk bundling the permissions are gone:

Outside the container:

It seems to me that the files are actually "recreated" somehow since the seem to have a default mask applied to them. The permissions issue is already present on the output folder, hence it has nothing to do with the zip file creation.

As docker volumes are tricky I suspect an issue with using volumes or the fs operations.

Note: I do use Docker engine v19.03.13 with WSL 2 on a Windows 10 AMD64 host

@strongishllama what is your windows setup?

[pfried]

Something I noticed: The permissions in Windows are different from the ones seen from inside the docker container. (Most likely due to filesystem differences), propably the permissions simply get lost when moving the file from the container to windows. One idea could be to require the user to zip inside the docker container (thats what I now do manually as a workaround)

[pfried]

@eladb Since you are assigned to this, would it be possible to add an option like outputFile where it would skip the archiving and just use the given output file if it exists? This would make it the users responsibility to provide the ready packaged zip file.

I guess this packaging will never work properly on windows since it will loose the permissions set inside the container.

Besides that there should be a warning at https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-assets-readme.html#asset-bundling that bundling on windows might fail (if the file needs e.g. executable permissions).

[eladb]

@pfried sounds like a good direction. @jogold what do you think?

[jogold]

@pfried sounds like a good direction. @jogold what do you think?

@eladb You still don't want to do this automagically if the content in /asset-output is a single archive file?

[pfried]

Would also be a good idea, I wonder if there should be an example in the docs how to create a zip archive

…

Am 10.12.2020 um 09:56 schrieb Jonathan Goldwasser ***@***.***>:  @pfried sounds like a good direction. @jogold what do you think? You still don't want to do this automagically if the content in /asset-output is a single archive file? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[pfried]

I would like to get back to this, I would be fine with both approaches

[eladb]

@jogold I am okay with doing this automatically if /asset-output includes a single .zip file, but let's add an option to disable this behavior (in case for some odd reason users want to create a bundle that contains a single zip file inside the bundle).

[jogold]

@jogold I am okay with doing this automatically if /asset-output includes a single .zip file, but let's add an option to disable this behavior (in case for some odd reason users want to create a bundle that contains a single zip file inside the bundle).

@eladb started the work in #13076, how would you call this option? alwaysArchive which defaults to false?

[eladb]

@eladb started the work in #13076, how would you call this option? alwaysArchive which defaults to false?

I'd probably go with an enum with three options:

1. Always zip
2. Never zip (fail if the output directory does not contain a single zip file)
3. Auto

It sure about the names...

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.