Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): ENOTDIR invalid cwd on "cdk deploy" #13145

Merged
merged 2 commits into from
Feb 19, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/@aws-cdk/aws-lambda-nodejs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ new lambda.NodejsFunction(this, 'my-handler', {
},
logLevel: LogLevel.SILENT, // defaults to LogLevel.WARNING
keepNames: true, // defaults to false
tsconfig: 'custom-tsconfig.json' // use custom-tsconfig.json instead of default,
tsconfig: 'custom-tsconfig.json' // use custom-tsconfig.json instead of default,
metafile: true, // include meta file, defaults to false
banner : '/* comments */', // by default no comments are passed
footer : '/* comments */', // by default no comments are passed
Expand Down Expand Up @@ -216,7 +216,7 @@ Use `bundling.dockerImage` to use a custom Docker bundling image:
```ts
new lambda.NodejsFunction(this, 'my-handler', {
bundling: {
dockerImage: cdk.DockerImage.fromBuild('/path/to/Dockerfile'),
dockerImage: cdk.BundlingDockerImage.fromAsset('/path/to/Dockerfile'),
},
});
```
Expand Down
2 changes: 1 addition & 1 deletion packages/@aws-cdk/aws-lambda-nodejs/test/bundling.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ beforeEach(() => {
getEsBuildVersionMock.mockReturnValue('0.8.8');
fromAssetMock.mockReturnValue({
image: 'built-image',
cp: () => 'dest-path',
cp: () => {},
run: () => {},
toJSON: () => 'built-image',
});
Expand Down
11 changes: 4 additions & 7 deletions packages/@aws-cdk/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,6 @@ runtime code.
* `lambda.Code.fromAsset(path)` - specify a directory or a .zip file in the local
filesystem which will be zipped and uploaded to S3 before deployment. See also
[bundling asset code](#bundling-asset-code).
* `lambda.Code.fromDockerBuild(path, options)` - use the result of a Docker
build as code. The runtime code is expected to be located at `/asset` in the
image and will be zipped and uploaded to S3 as an asset.

The following example shows how to define a Python function and deploy the code
from the local directory `my-lambda-handler` to it:
Expand Down Expand Up @@ -453,7 +450,7 @@ new lambda.Function(this, 'Function', {
bundling: {
image: lambda.Runtime.PYTHON_3_6.bundlingDockerImage,
command: [
'bash', '-c',
'bash', '-c',
'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
],
},
Expand All @@ -465,16 +462,16 @@ new lambda.Function(this, 'Function', {

Runtimes expose a `bundlingDockerImage` property that points to the [AWS SAM](https://github.com/awslabs/aws-sam-cli) build image.

Use `cdk.DockerImage.fromRegistry(image)` to use an existing image or
`cdk.DockerImage.fromBuild(path)` to build a specific image:
Use `cdk.BundlingDockerImage.fromRegistry(image)` to use an existing image or
`cdk.BundlingDockerImage.fromAsset(path)` to build a specific image:

```ts
import * as cdk from '@aws-cdk/core';

new lambda.Function(this, 'Function', {
code: lambda.Code.fromAsset('/path/to/handler', {
bundling: {
image: cdk.DockerImage.fromBuild('/path/to/dir/with/DockerFile', {
image: cdk.BundlingDockerImage.fromAsset('/path/to/dir/with/DockerFile', {
buildArgs: {
ARG1: 'value1',
},
Expand Down
37 changes: 0 additions & 37 deletions packages/@aws-cdk/aws-lambda/lib/code.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,22 +57,6 @@ export abstract class Code {
return new AssetCode(path, options);
}

/**
* Loads the function code from an asset created by a Docker build.
*
* By defaut, the asset is expected to be located at `/asset` in the
* image.
*
* @param path The path to the directory containing the Docker file
* @param options Docker build options
*/
public static fromDockerBuild(path: string, options: DockerBuildAssetOptions = {}): AssetCode {
const assetPath = cdk.DockerImage
.fromBuild(path, options)
.cp(options.imagePath ?? '/asset', options.outputPath);
return new AssetCode(assetPath);
}

/**
* DEPRECATED
* @deprecated use `fromAsset`
Expand Down Expand Up @@ -504,24 +488,3 @@ export class AssetImageCode extends Code {
};
}
}

/**
* Options when creating an asset from a Docker build.
*/
export interface DockerBuildAssetOptions extends cdk.DockerBuildOptions {
/**
* The path in the Docker image where the asset is located after the build
* operation.
*
* @default /asset
*/
readonly imagePath?: string;

/**
* The path on the local filesystem where the asset will be copied
* using `docker cp`.
*
* @default - a unique temporary directory in the system temp directory
*/
readonly outputPath?: string;
}
23 changes: 0 additions & 23 deletions packages/@aws-cdk/aws-lambda/test/code.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -327,29 +327,6 @@ describe('code', () => {
});
});
});

describe('lambda.Code.fromDockerBuild', () => {
test('can use the result of a Docker build as an asset', () => {
// given
const stack = new cdk.Stack();
stack.node.setContext(cxapi.ASSET_RESOURCE_METADATA_ENABLED_CONTEXT, true);

// when
new lambda.Function(stack, 'Fn', {
code: lambda.Code.fromDockerBuild(path.join(__dirname, 'docker-build-lambda')),
handler: 'index.handler',
runtime: lambda.Runtime.NODEJS_12_X,
});

// then
expect(stack).toHaveResource('AWS::Lambda::Function', {
Metadata: {
[cxapi.ASSET_RESOURCE_METADATA_PATH_KEY]: 'asset.38cd320fa97b348accac88e48d9cede4923f7cab270ce794c95a665be83681a8',
[cxapi.ASSET_RESOURCE_METADATA_PROPERTY_KEY]: 'Code',
},
}, ResourcePart.CompleteDefinition);
});
});
});

function defineFunction(code: lambda.Code, runtime: lambda.Runtime = lambda.Runtime.NODEJS_10_X) {
Expand Down

This file was deleted.

This file was deleted.

27 changes: 3 additions & 24 deletions packages/@aws-cdk/aws-s3-assets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ The following example uses custom asset bundling to convert a markdown file to h

[Example of using asset bundling](./test/integ.assets.bundling.lit.ts).

The bundling docker image (`image`) can either come from a registry (`DockerImage.fromRegistry`)
or it can be built from a `Dockerfile` located inside your project (`DockerImage.fromBuild`).
The bundling docker image (`image`) can either come from a registry (`BundlingDockerImage.fromRegistry`)
or it can be built from a `Dockerfile` located inside your project (`BundlingDockerImage.fromAsset`).

You can set the `CDK_DOCKER` environment variable in order to provide a custom
docker program to execute. This may sometime be needed when building in
Expand All @@ -114,7 +114,7 @@ new assets.Asset(this, 'BundledAsset', {
},
},
// Docker bundling fallback
image: DockerImage.fromRegistry('alpine'),
image: BundlingDockerImage.fromRegistry('alpine'),
entrypoint: ['/bin/sh', '-c'],
command: ['bundle'],
},
Expand All @@ -124,27 +124,6 @@ new assets.Asset(this, 'BundledAsset', {
Although optional, it's recommended to provide a local bundling method which can
greatly improve performance.

If the bundling output contains a single archive file (zip or jar) it will be
uploaded to S3 as-is and will not be zipped. Otherwise the contents of the
output directory will be zipped and the zip file will be uploaded to S3. This
is the default behavior for `bundling.outputType` (`BundlingOutput.AUTO_DISCOVER`).

Use `BundlingOutput.NOT_ARCHIVED` if the bundling output must always be zipped:

```ts
const asset = new assets.Asset(this, 'BundledAsset', {
path: '/path/to/asset',
bundling: {
image: DockerImage.fromRegistry('alpine'),
command: ['command-that-produces-an-archive.sh'],
outputType: BundlingOutput.NOT_ARCHIVED, // Bundling output will be zipped even though it produces a single archive file.
},
});
```

Use `BundlingOutput.ARCHIVED` if the bundling output contains a single archive file and
you don't want it to be zippped.

## CloudFormation Resource Metadata

> NOTE: This section is relevant for authors of AWS Resource Constructs.
Expand Down
30 changes: 27 additions & 3 deletions packages/@aws-cdk/aws-s3-assets/lib/asset.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import * as fs from 'fs';
import * as path from 'path';
import * as assets from '@aws-cdk/assets';
import * as iam from '@aws-cdk/aws-iam';
Expand All @@ -12,6 +13,8 @@ import { toSymlinkFollow } from './compat';
// eslint-disable-next-line no-duplicate-imports, import/order
import { Construct as CoreConstruct } from '@aws-cdk/core';

const ARCHIVE_EXTENSIONS = ['.zip', '.jar'];

export interface AssetOptions extends assets.CopyOptions, cdk.AssetOptions {
/**
* A list of principals that should be able to read this asset from S3.
Expand Down Expand Up @@ -136,12 +139,17 @@ export class Asset extends CoreConstruct implements cdk.IAsset {

this.assetPath = staging.relativeStagedPath(stack);

this.isFile = staging.packaging === cdk.FileAssetPackaging.FILE;
const packaging = determinePackaging(staging.sourcePath);

this.isFile = packaging === cdk.FileAssetPackaging.FILE;

this.isZipArchive = staging.isArchive;
// sets isZipArchive based on the type of packaging and file extension
this.isZipArchive = packaging === cdk.FileAssetPackaging.ZIP_DIRECTORY
? true
: ARCHIVE_EXTENSIONS.some(ext => staging.sourcePath.toLowerCase().endsWith(ext));

const location = stack.synthesizer.addFileAsset({
packaging: staging.packaging,
packaging,
sourceHash: this.sourceHash,
fileName: this.assetPath,
});
Expand Down Expand Up @@ -202,3 +210,19 @@ export class Asset extends CoreConstruct implements cdk.IAsset {
this.bucket.grantRead(grantee);
}
}

function determinePackaging(assetPath: string): cdk.FileAssetPackaging {
if (!fs.existsSync(assetPath)) {
throw new Error(`Cannot find asset at ${assetPath}`);
}

if (fs.statSync(assetPath).isDirectory()) {
return cdk.FileAssetPackaging.ZIP_DIRECTORY;
}

if (fs.statSync(assetPath).isFile()) {
return cdk.FileAssetPackaging.FILE;
}

throw new Error(`Asset ${assetPath} is expected to be either a directory or a regular file`);
}
90 changes: 5 additions & 85 deletions packages/@aws-cdk/core/lib/asset-staging.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import * as cxapi from '@aws-cdk/cx-api';
import { Construct } from 'constructs';
import * as fs from 'fs-extra';
import * as minimatch from 'minimatch';
import { AssetHashType, AssetOptions, FileAssetPackaging } from './assets';
import { BundlingOptions, BundlingOutput } from './bundling';
import { AssetHashType, AssetOptions } from './assets';
import { BundlingOptions } from './bundling';
import { FileSystem, FingerprintOptions } from './fs';
import { Names } from './names';
import { Cache } from './private/cache';
Expand All @@ -17,8 +17,6 @@ import { Stage } from './stage';
// eslint-disable-next-line
import { Construct as CoreConstruct } from './construct-compat';

const ARCHIVE_EXTENSIONS = ['.zip', '.jar'];

/**
* A previously staged asset
*/
Expand Down Expand Up @@ -140,9 +138,6 @@ export class AssetStaging extends CoreConstruct {

private readonly cacheKey: string;

private _packaging = FileAssetPackaging.ZIP_DIRECTORY;
private _isArchive = true;

constructor(scope: Construct, id: string, props: AssetStagingProps) {
super(scope, id);

Expand Down Expand Up @@ -208,20 +203,6 @@ export class AssetStaging extends CoreConstruct {
return this.assetHash;
}

/**
* How this asset should be packaged.
*/
public get packaging(): FileAssetPackaging {
return this._packaging;
}

/**
* Whether this asset is an archive (zip or jar).
*/
public get isArchive(): boolean {
return this._isArchive;
}

/**
* Return the path to the staged asset, relative to the Cloud Assembly (manifest) directory of the given stack
*
Expand Down Expand Up @@ -300,16 +281,11 @@ export class AssetStaging extends CoreConstruct {
const bundleDir = this.determineBundleDir(this.assetOutdir, assetHash);
this.bundle(bundling, bundleDir);

// Check bundling output content and determine if we will need to archive
const bundlingOutputType = bundling.outputType ?? BundlingOutput.AUTO_DISCOVER;
const bundledAsset = determineBundledAsset(bundleDir, bundlingOutputType);
this._packaging = bundledAsset.packaging;

// Calculate assetHash afterwards if we still must
assetHash = assetHash ?? this.calculateHash(this.hashType, bundling, bundledAsset.path);
const stagedPath = path.resolve(this.assetOutdir, renderAssetFilename(assetHash, bundledAsset.extension));
assetHash = assetHash ?? this.calculateHash(this.hashType, bundling, bundleDir);
const stagedPath = path.resolve(this.assetOutdir, renderAssetFilename(assetHash));

this.stageAsset(bundledAsset.path, stagedPath, 'move');
this.stageAsset(bundleDir, stagedPath, 'move');
return { assetHash, stagedPath };
}

Expand Down Expand Up @@ -347,8 +323,6 @@ export class AssetStaging extends CoreConstruct {
const stat = fs.statSync(sourcePath);
if (stat.isFile()) {
fs.copyFileSync(sourcePath, targetPath);
this._packaging = FileAssetPackaging.FILE;
this._isArchive = ARCHIVE_EXTENSIONS.includes(path.extname(sourcePath).toLowerCase());
} else if (stat.isDirectory()) {
fs.mkdirSync(targetPath);
FileSystem.copyDirectory(sourcePath, targetPath, this.fingerprintOptions);
Expand Down Expand Up @@ -528,57 +502,3 @@ function sortObject(object: { [key: string]: any }): { [key: string]: any } {
}
return ret;
}

/**
* Returns the single archive file of a directory or undefined
*/
function singleArchiveFile(directory: string): string | undefined {
if (!fs.existsSync(directory)) {
throw new Error(`Directory ${directory} does not exist.`);
}

if (!fs.statSync(directory).isDirectory()) {
throw new Error(`${directory} is not a directory.`);
}

const content = fs.readdirSync(directory);
if (content.length === 1) {
const file = path.join(directory, content[0]);
const extension = path.extname(content[0]).toLowerCase();
if (fs.statSync(file).isFile() && ARCHIVE_EXTENSIONS.includes(extension)) {
return file;
}
}

return undefined;
}

interface BundledAsset {
path: string,
packaging: FileAssetPackaging,
extension?: string
}

/**
* Returns the bundled asset to use based on the content of the bundle directory
* and the type of output.
*/
function determineBundledAsset(bundleDir: string, outputType: BundlingOutput): BundledAsset {
const archiveFile = singleArchiveFile(bundleDir);

// auto-discover means that if there is an archive file, we take it as the
// bundle, otherwise, we will archive here.
if (outputType === BundlingOutput.AUTO_DISCOVER) {
outputType = archiveFile ? BundlingOutput.ARCHIVED : BundlingOutput.NOT_ARCHIVED;
}

switch (outputType) {
case BundlingOutput.NOT_ARCHIVED:
return { path: bundleDir, packaging: FileAssetPackaging.ZIP_DIRECTORY };
case BundlingOutput.ARCHIVED:
if (!archiveFile) {
throw new Error('Bundling output directory is expected to include only a single .zip or .jar file when `output` is set to `ARCHIVED`');
}
return { path: archiveFile, packaging: FileAssetPackaging.FILE, extension: path.extname(archiveFile) };
}
}
Loading