-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Assets #371
Assets #371
Changes from 4 commits
5142934
34d213e
22128e5
47643c4
05c0e8d
1e78025
5502436
b208b22
0763282
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
*.js | ||
*.js.map | ||
*.d.ts | ||
node_modules | ||
dist | ||
tsconfig.json | ||
tslint.json | ||
|
||
.LAST_BUILD | ||
.nyc_output | ||
coverage | ||
|
||
.jsii |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# Don't include original .ts files when doing `npm pack` | ||
*.ts | ||
!*.d.ts | ||
coverage | ||
.nyc_output | ||
*.tgz |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
## AWS CDK Assets | ||
|
||
Assets are local files or directories which are needed by a CDK app. A common | ||
example is a directory which contains the handler code for a Lambda function, | ||
but assets can represent any artifact that is needed for the app's operation. | ||
|
||
When deploying a CDK app that includes constructs with assets, the CDK toolkit | ||
will first upload all the assets to S3, and only then deploy the stacks. The S3 | ||
locations of the uploaded assets will be passed in as CloudFormation Parameters | ||
to the relevant stacks. | ||
|
||
The following JavaScript example defines an directory asset which is archived as | ||
a .zip file and uploaded to S3 during deployment. | ||
|
||
[Example of a ZipDirectoryAsset](./test/integ.assets.directory.lit.ts) | ||
|
||
The following JavaScript example defines a file asset, which is uploaded as-is | ||
to an S3 bucket during deployment. | ||
|
||
[Example of a FileAsset](./test/integ.assets.file.lit.ts) | ||
|
||
## Attributes | ||
|
||
`Asset` constructs expose the following deploy-time attributes: | ||
|
||
* `s3BucketName` - the name of the assets S3 bucket. | ||
* `s3ObjectKey` - the S3 object key of the asset file (whether it's a file or a zip archive) | ||
* `s3Url` - the S3 URL of the asset (i.e. https://s3.us-east-1.amazonaws.com/mybucket/mykey.zip) | ||
|
||
In the following example, the various asset attributes are exported as stack outputs: | ||
|
||
[Example of referencing an asset](./test/integ.assets.refs.lit.ts) | ||
|
||
## Permissions | ||
|
||
IAM roles, users or groups which need to be able to read assets in runtime will should be | ||
granted IAM permissions. To do that use the `asset.grantRead(principal)` method: | ||
|
||
The following examples grants an IAM group read permissions on an asset: | ||
|
||
[Example of granting read access to an asset](./test/integ.assets.permissions.lit.ts) | ||
|
||
## How does it work? | ||
|
||
When an asset is defined in a construct, a construct metadata entry | ||
`aws:cdk:asset` is emitted with instructions on where to find the asset and what | ||
type of packaging to perform (`zip` or `file`). Furthermore, the synthesized | ||
CloudFormation template will also include two CloudFormation parameters: one for | ||
the asset's bucket and one for the asset S3 key. Those parameters are used to | ||
reference the deploy-time values of the asset (using `{ Ref: "Param" }`). | ||
|
||
Then, when the stack is deployed, the toolkit will package the asset (i.e. zip | ||
the directory), calculate an MD5 hash of the contents and will render an S3 key | ||
for this asset within the toolkit's asset store. If the file doesn't exist in | ||
the asset store, it is uploaded during deployment. | ||
|
||
> The toolkit's asset store is an S3 bucket created by the toolkit for each | ||
environment the toolkit operates in (environment = account + region). | ||
|
||
Now, when the toolkit deploys the stack, it will set the relevant CloudFormation | ||
Parameters to point to the actual bucket and key for each asset. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,187 @@ | ||
import cdk = require('@aws-cdk/core'); | ||
import cxapi = require('@aws-cdk/cx-api'); | ||
import iam = require('@aws-cdk/iam'); | ||
import s3 = require('@aws-cdk/s3'); | ||
import * as fs from 'fs'; | ||
import * as path from 'path'; | ||
|
||
/** | ||
* Defines the way an asset is packaged before it is uploaded to S3. | ||
*/ | ||
export enum AssetPackaging { | ||
/** | ||
* Path refers to a directory on disk, the contents of the directory is | ||
* archived into a .zip. | ||
*/ | ||
ZipDirectory = 'zip', | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, there may be an interest for uploading a given directory (un-zipp'd). It can be painful to have to map every file individually when there are a lot... Would expect the URL in this case to be that of where the files were uploaded - with the same structure (so I can just append path elements to get a particular file). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can always add additional packaging schemes, such as |
||
|
||
/** | ||
* Path refers to a single file on disk. The file is uploaded as-is. | ||
*/ | ||
File = 'file', | ||
} | ||
|
||
export interface GenericAssetProps { | ||
/** | ||
* The disk location of the asset. | ||
*/ | ||
path: string; | ||
|
||
/** | ||
* The packaging type for this asset. | ||
*/ | ||
packaging: AssetPackaging; | ||
|
||
/** | ||
* A list of principals that should be able to read this asset from S3. | ||
* You can use `asset.grantRead(principal)` to grant read permissions later. | ||
*/ | ||
readers?: iam.IPrincipal[]; | ||
} | ||
|
||
/** | ||
* An asset represents a local file or directory, which is automatically uploaded to S3 | ||
* and then can be referenced within a CDK application. | ||
*/ | ||
export class Asset extends cdk.Construct { | ||
/** | ||
* Attribute that represents the name of the bucket this asset exists in. | ||
*/ | ||
public readonly s3BucketName: s3.BucketName; | ||
|
||
/** | ||
* Attribute which represents the S3 object key of this asset. | ||
*/ | ||
public readonly s3ObjectKey: s3.ObjectKey; | ||
|
||
/** | ||
* Attribute which represents the S3 URL of this asset. | ||
* @example https://s3.us-west-1.amazonaws.com/bucket/key | ||
*/ | ||
public readonly s3Url: s3.S3Url; | ||
|
||
/** | ||
* Resolved full-path location of this asset. | ||
*/ | ||
public readonly assetPath: string; | ||
|
||
private readonly bucket: s3.BucketRef; | ||
|
||
constructor(parent: cdk.Construct, id: string, props: GenericAssetProps) { | ||
super(parent, id); | ||
|
||
// resolve full path | ||
this.assetPath = path.resolve(props.path); | ||
|
||
if (!fs.existsSync(this.assetPath)) { | ||
throw new Error(`Cannot find asset at ${props.path}`); | ||
} | ||
|
||
if (props.packaging === AssetPackaging.ZipDirectory) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
if (!fs.statSync(this.assetPath).isDirectory()) { | ||
throw new Error(`${this.assetPath} is expected to be a directory when asset packaging is 'zip'`); | ||
} | ||
} else if (props.packaging === AssetPackaging.File) { | ||
if (!fs.statSync(this.assetPath).isFile()) { | ||
throw new Error(`${this.assetPath} is expected to be a regular file when asset packaging is 'file'`); | ||
} | ||
} else { | ||
throw new Error(`Unsupported asset packaging format: ${props.packaging}`); | ||
} | ||
|
||
// add parameters for s3 bucket and s3 key. those will be set by | ||
// the toolkit or by CI/CD when the stack is deployed and will include | ||
// the name of the bucket and the S3 key where the code lives. | ||
|
||
const bucketParam = new cdk.Parameter(this, 'S3Bucket', { | ||
type: 'String', | ||
description: `S3 bucket for asset "${this.path}"`, | ||
}); | ||
|
||
const keyParam = new cdk.Parameter(this, 'S3ObjectKey', { | ||
type: 'String', | ||
description: `S3 object for asset "${this.path}"` | ||
}); | ||
|
||
this.s3BucketName = bucketParam.value; | ||
this.s3ObjectKey = keyParam.value; | ||
|
||
// grant the lambda's role read permissions on the code s3 object | ||
|
||
this.bucket = s3.BucketRef.import(parent, 'AssetBucket', { | ||
bucketName: this.s3BucketName | ||
}); | ||
|
||
// form the s3 URL of the object key | ||
this.s3Url = this.bucket.urlForObject(this.s3ObjectKey); | ||
|
||
// attach metadata to the lambda function which includes information | ||
// for tooling to be able to package and upload a directory to the | ||
// s3 bucket and plug in the bucket name and key in the correct | ||
// parameters. | ||
|
||
const asset: cxapi.AssetMetadataEntry = { | ||
path: this.assetPath, | ||
packaging: props.packaging, | ||
s3BucketParameter: bucketParam.logicalId, | ||
s3KeyParameter: keyParam.logicalId, | ||
}; | ||
|
||
this.addMetadata(cxapi.ASSET_METADATA, asset); | ||
|
||
for (const reader of (props.readers || [])) { | ||
this.grantRead(reader); | ||
} | ||
} | ||
|
||
/** | ||
* Grants read permissions to the principal on the asset's S3 object. | ||
*/ | ||
public grantRead(principal?: iam.IPrincipal) { | ||
this.bucket.grantRead(principal, this.s3ObjectKey); | ||
} | ||
} | ||
|
||
export interface FileAssetProps { | ||
/** | ||
* File path. | ||
*/ | ||
path: string; | ||
|
||
/** | ||
* A list of principals that should be able to read this file asset from S3. | ||
* You can use `asset.grantRead(principal)` to grant read permissions later. | ||
*/ | ||
readers?: iam.IPrincipal[]; | ||
} | ||
|
||
/** | ||
* An asset that represents a file on disk. | ||
*/ | ||
export class FileAsset extends Asset { | ||
constructor(parent: cdk.Construct, id: string, props: FileAssetProps) { | ||
super(parent, id, { packaging: AssetPackaging.File, ...props }); | ||
} | ||
} | ||
|
||
export interface ZipDirectoryAssetProps { | ||
/** | ||
* Path of the directory. | ||
*/ | ||
path: string; | ||
|
||
/** | ||
* A list of principals that should be able to read this ZIP file from S3. | ||
* You can use `asset.grantRead(principal)` to grant read permissions later. | ||
*/ | ||
readers?: iam.IPrincipal[]; | ||
} | ||
|
||
/** | ||
* An asset that represents a ZIP archive of a directory on disk. | ||
*/ | ||
export class ZipDirectoryAsset extends Asset { | ||
constructor(parent: cdk.Construct, id: string, props: ZipDirectoryAssetProps) { | ||
super(parent, id, { packaging: AssetPackaging.ZipDirectory, ...props }); | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
export * from './asset'; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
{ | ||
"name": "@aws-cdk/cdk-assets", | ||
"version": "0.7.3-beta", | ||
"description": "Integration of CDK apps with local assets", | ||
"main": "lib/index.js", | ||
"types": "lib/index.d.ts", | ||
"jsii": { | ||
"outdir": "dist", | ||
"names": { | ||
"java": "com.amazonaws.cdk.cdkassets", | ||
"dotnet": "Amazon.Cdk.CdkAssets" | ||
} | ||
}, | ||
"repository": { | ||
"type": "git", | ||
"url": "git://github.com/awslabs/aws-cdk" | ||
}, | ||
"scripts": { | ||
"build": "cdk-build", | ||
"watch": "cdk-watch", | ||
"lint": "cdk-lint", | ||
"test": "cdk-test", | ||
"integ": "cdk-integ", | ||
"pkglint": "pkglint -f" | ||
}, | ||
"keywords": [ | ||
"aws", | ||
"cdk", | ||
"constructs", | ||
"assets" | ||
], | ||
"author": { | ||
"name": "Amazon Web Services", | ||
"url": "https://aws.amazon.com" | ||
}, | ||
"license": "LicenseRef-LICENSE", | ||
"devDependencies": { | ||
"@aws-cdk/assert": "^0.7.3-beta", | ||
"aws-cdk": "^0.7.3-beta", | ||
"pkglint": "^0.7.3-beta", | ||
"cdk-build-tools": "^0.7.3-beta", | ||
"cdk-integ-tools": "^0.7.3-beta" | ||
}, | ||
"dependencies": { | ||
"@aws-cdk/s3": "^0.7.3-beta", | ||
"@aws-cdk/core": "^0.7.3-beta", | ||
"@aws-cdk/iam": "^0.7.3-beta", | ||
"@aws-cdk/cx-api": "^0.7.3-beta" | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Hello, this is a just a file! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we not using
import * as cdk from '@aws-cdk/core';
and similar for our own libraries, but using it fornode
libraries? I find this confusing.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should standardize around
import x = require('y')
for star imports across the board. It's a clean and concise syntax.