Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): local bundling provider #9564

Merged
merged 17 commits into from
Aug 11, 2020
27 changes: 27 additions & 0 deletions packages/@aws-cdk/aws-s3-assets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,33 @@ docker program to execute. This may sometime be needed when building in
environments where the standard docker cannot be executed (see
https://github.com/aws/aws-cdk/issues/8460 for details).

Use `local` to specify a local bundling provider. The provider implements a
method `tryBundle()` which should return `true` if local bundling was performed.
If `false` is returned, docker bundling will be done:

```ts
new assets.Asset(this, 'BundledAsset', {
path: '/path/to/asset',
bundling: {
local: {
tryBundler(outputDir: string, options: BundlingOptions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be tryBundle no?

if (canRunLocally) {
// perform local bundling here
return true;
}
return false;
},
},
// Docker bundling fallback
image: BundlingDockerImage.fromRegistry('alpine'),
command: ['bundle'],
},
});
```

Although optional, it's recommended to provide a local bundling method which can
greatly improve performance.

## CloudFormation Resource Metadata

> NOTE: This section is relevant for authors of AWS Resource Constructs.
Expand Down
24 changes: 15 additions & 9 deletions packages/@aws-cdk/core/lib/asset-staging.ts
Original file line number Diff line number Diff line change
Expand Up @@ -178,21 +178,27 @@ export class AssetStaging extends Construct {
...options.volumes ?? [],
];

let localBundling: boolean | undefined;
try {
process.stderr.write(`Bundling asset ${this.construct.path}...\n`);
options.image._run({
command: options.command,
user,
volumes,
environment: options.environment,
workingDirectory: options.workingDirectory ?? AssetStaging.BUNDLING_INPUT_DIR,
});

localBundling = options.local?.tryBundle(bundleDir, options);
if (!localBundling) {
options.image._run({
command: options.command,
user,
volumes,
environment: options.environment,
workingDirectory: options.workingDirectory ?? AssetStaging.BUNDLING_INPUT_DIR,
});
}
} catch (err) {
throw new Error(`Failed to run bundling Docker image for asset ${this.construct.path}: ${err}`);
throw new Error(`Failed to bundle asset ${this.construct.path}: ${err}`);
}

if (FileSystem.isEmpty(bundleDir)) {
throw new Error(`Bundling did not produce any output. Check that your container writes content to ${AssetStaging.BUNDLING_OUTPUT_DIR}.`);
const outputDir = localBundling ? bundleDir : AssetStaging.BUNDLING_OUTPUT_DIR;
throw new Error(`Bundling did not produce any output. Check that content is written to ${outputDir}.`);
}

return bundleDir;
Expand Down
38 changes: 34 additions & 4 deletions packages/@aws-cdk/core/lib/bundling.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ export interface BundlingOptions {
readonly image: BundlingDockerImage;

/**
* The command to run in the container.
* The command to run in the Docker container.
*
* @example ['npm', 'install']
*
Expand All @@ -30,21 +30,21 @@ export interface BundlingOptions {
readonly volumes?: DockerVolume[];

/**
* The environment variables to pass to the container.
* The environment variables to pass to the Docker container.
*
* @default - no environment variables.
*/
readonly environment?: { [key: string]: string; };

/**
* Working directory inside the container.
* Working directory inside the Docker container.
*
* @default /asset-input
*/
readonly workingDirectory?: string;

/**
* The user to use when running the container.
* The user to use when running the Docker container.
*
* user | user:group | uid | uid:gid | user:gid | uid:group
*
Expand All @@ -53,6 +53,36 @@ export interface BundlingOptions {
* @default - uid:gid of the current user or 1000:1000 on Windows
*/
readonly user?: string;

/**
* Local bundling provider.
*
* The provider implements a method `tryBundle()` which should return `true`
* if local bundling was performed. If `false` is returned, docker bundling
* will be done.
*
* @default - bundling will only be performed in a Docker container
*
* @experimental
*/
readonly local?: ILocalBundling;
jogold marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Local bundling
*
* @experimental
*/
export interface ILocalBundling {
/**
* This method is called before attempting docker bundling to allow the
* bundler to be executed locally. If the local bundler exists, and bundling
* was performed locally, return `true`. Otherwise, return `false`.
*
* @param outputDir the directory where the bundled asset should be output
* @param options bundling options for this asset
*/
tryBundle(outputDir: string, options: BundlingOptions): boolean;
jogold marked this conversation as resolved.
Show resolved Hide resolved
}

/**
Expand Down
67 changes: 65 additions & 2 deletions packages/@aws-cdk/core/test/test.staging.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import * as cxapi from '@aws-cdk/cx-api';
import * as fs from 'fs-extra';
import { Test } from 'nodeunit';
import * as sinon from 'sinon';
import { App, AssetHashType, AssetStaging, BundlingDockerImage, Stack } from '../lib';
import { App, AssetHashType, AssetStaging, BundlingDockerImage, BundlingOptions, Stack } from '../lib';

const STUB_INPUT_FILE = '/tmp/docker-stub.input';

Expand Down Expand Up @@ -281,14 +281,77 @@ export = {
image: BundlingDockerImage.fromRegistry('this-is-an-invalid-docker-image'),
command: [ DockerStubCommand.FAIL ],
},
}), /Failed to run bundling Docker image for asset stack\/Asset/);
}), /Failed to bundle asset stack\/Asset/);
test.equal(
readDockerStubInput(),
`run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input this-is-an-invalid-docker-image DOCKER_STUB_FAIL`,
);

test.done();
},

'with local bundling'(test: Test) {
// GIVEN
const app = new App();
const stack = new Stack(app, 'stack');
const directory = path.join(__dirname, 'fs', 'fixtures', 'test1');

// WHEN
let dir: string | undefined;
let opts: BundlingOptions | undefined;
new AssetStaging(stack, 'Asset', {
sourcePath: directory,
bundling: {
image: BundlingDockerImage.fromRegistry('alpine'),
command: [DockerStubCommand.SUCCESS],
local: {
tryBundle(outputDir: string, options: BundlingOptions): boolean {
dir = outputDir;
opts = options;
fs.writeFileSync(path.join(outputDir, 'hello.txt'), 'hello'); // output cannot be empty
return true;
},
},
},
});

// THEN
test.ok(dir && /asset-bundle-/.test(dir));
test.equals(opts?.command?.[0], DockerStubCommand.SUCCESS);
test.throws(() => readDockerStubInput());

if (dir) {
fs.removeSync(path.join(dir, 'hello.txt'));
}

test.done();
},

'with local bundling returning false'(test: Test) {
// GIVEN
const app = new App();
const stack = new Stack(app, 'stack');
const directory = path.join(__dirname, 'fs', 'fixtures', 'test1');

// WHEN
new AssetStaging(stack, 'Asset', {
sourcePath: directory,
bundling: {
image: BundlingDockerImage.fromRegistry('alpine'),
command: [DockerStubCommand.SUCCESS],
local: {
tryBundle(_bundleDir: string): boolean {
return false;
},
},
},
});

// THEN
test.ok(readDockerStubInput());

test.done();
},
};

function readDockerStubInput() {
Expand Down