Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cloudformation): aws custom resource #1850

Merged
merged 31 commits into from
May 29, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
4a8e4b6
feat(cloudformation): aws sdk js custom resource
jogold Feb 23, 2019
5ab6588
Log AWS SDK version
jogold Feb 25, 2019
15a6ef8
Add tests
jogold Feb 25, 2019
34a3ebc
Merge branch 'master' into aws-sdk-js-resource
jogold Mar 1, 2019
ad86457
Return API response as output attributes
jogold Mar 4, 2019
84c73ab
Remove default on onUpdate
jogold Mar 4, 2019
5ba7765
Add options to specify physical resource id
jogold Mar 4, 2019
6f1c0c6
Merge branch 'master' into aws-sdk-js-resource
jogold Mar 26, 2019
5811c94
Rename to AwsCustomResource
jogold Mar 26, 2019
941913b
Add readonly
jogold Mar 28, 2019
552a5e7
Merge branch 'master' into aws-sdk-js-resource
jogold Mar 28, 2019
ae446b9
Write provider in ts
jogold Mar 28, 2019
2fb5163
Support path for physical resource id
jogold Mar 28, 2019
04c8df5
Typo
jogold Mar 28, 2019
8ddf39f
Merge branch 'master' into aws-sdk-js-resource
jogold Mar 28, 2019
82d1ea1
Fix bad pkglint version after merge
jogold Mar 28, 2019
30d3226
Merge branch 'master' into aws-sdk-js-resource
jogold Apr 2, 2019
a26be81
Add option to catch API errors
jogold Apr 2, 2019
7e9b21c
Merge branch 'aws-sdk-js-resource' of github.com:jogold/aws-cdk into …
jogold Apr 2, 2019
b5fc424
Restore package-lock.json in aws-codepipeline-actions
jogold Apr 2, 2019
25691a0
Merge branch 'master' into aws-sdk-js-resource
jogold May 13, 2019
1b7f392
CustomResourceProvider
jogold May 13, 2019
598f605
exclude construct-ctor-props-optional
jogold May 13, 2019
290d77c
remove duplicate statements check
jogold May 13, 2019
8981600
add option to lock api version
jogold May 13, 2019
c423f35
JSDoc
jogold May 13, 2019
d7b66cc
fix booleans in parameters
jogold May 14, 2019
e8813f5
update integration test
jogold May 14, 2019
1b204d8
update README
jogold May 14, 2019
700fa4d
Merge branch 'master' into aws-sdk-js-resource
jogold May 27, 2019
e02480f
update integ test
jogold May 27, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!*.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
const AWS = require('aws-sdk');

exports.handler = async function(event, context) {
try {
console.log(JSON.stringify(event));

if (event.ResourceProperties[event.RequestType]) {
const { service, action, parameters } = event.ResourceProperties[event.RequestType];
const awsService = new AWS[service]();
await awsService[action](parameters).promise();
}

await respond('SUCCESS', 'OK');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The physical resource ID is actually a pretty important thing to get right when defining custom resources. I am wondering if we can do better than logStreamName

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, if we want to avoid calling the onDelete during a UPDATE_COMPLETE_CLEANUP_IN_PROGRESS when a parameter of the onUpdate changes we need something like a constant here, no? can we use the LogicalResourceId?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a clever idea. To be useful, this resource is going to be an exercise in metaprogramming.

Logical ID wouldn't be good enough. I think a template string of sorts will need to come from the parameter. Also wonder if we might want to call multiple functions.

At least we'd want to explode the API result into a set of output attributes, I think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I think we will need to let users specify the path to where the physical ID can be found in the response JSON (at least), and this mechanism can also be used to produced a bunch of "GetAtt" attributes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the response only or also in the request? If you look at the example with the putTargets call what would you specify as physical ID?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the response only or also in the request? If you look at the example with the putTargets call what would you specify as physical ID?

The physical resource ID parameters needs to be able to be JSONPath member. It means it's not an ID by itself, but it's instructions for how to get the ID out of the API call response. So it should be able to be something like:

$.CreateThingResponse.ThingArn

And the custom resource should get the value out of the response object. Probably should be able to be different for both Create and Update calls.

Now, whether it can be EITHER a literal or a JSONPath I'm not sure on. I'm sure there are cases in which a literal would make sense, so I'm not against disallowing it, but the JSONPath-style query needs to be possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P.S: When I say JSONPath, I'm cool with faking it by splitting on . and leaving it there, I don't mean literally importing a jsonpath library, we don't have a good enough story around bundled lambda dependencies to make that work.

} catch (e) {
console.log(e);
await respond('FAILED', e.message);
}

function respond(responseStatus, reason) {
const responseBody = JSON.stringify({
Status: responseStatus,
Reason: reason,
PhysicalResourceId: context.logStreamName,
StackId: event.StackId,
RequestId: event.RequestId,
LogicalResourceId: event.LogicalResourceId,
NoEcho: false,
Data: {}
});

console.log('Responding', JSON.stringify(responseBody));

const parsedUrl = require('url').parse(event.ResponseURL);
const requestOptions = {
hostname: parsedUrl.hostname,
path: parsedUrl.path,
method: 'PUT',
headers: { 'content-type': '', 'content-length': responseBody.length }
};

return new Promise((resolve, reject) => {
try {
const request = require('https').request(requestOptions, resolve);
request.on('error', reject);
request.write(responseBody);
request.end();
} catch (e) {
reject(e);
}
});
}
}
145 changes: 145 additions & 0 deletions packages/@aws-cdk/aws-cloudformation/lib/aws-sdk-js-custom-resource.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
import iam = require('@aws-cdk/aws-iam');
import lambda = require('@aws-cdk/aws-lambda');
import cdk = require('@aws-cdk/cdk');
import metadata = require('aws-sdk/apis/metadata.json');
import path = require('path');
import { CustomResource } from './custom-resource';

/**
* AWS SDK service metadata.
*/
export type AwsSdkMetadata = {[key: string]: any};

const awsSdkMetadata: AwsSdkMetadata = metadata;

/**
* An AWS SDK call.
*/
export interface AwsSdkCall {
/**
* The service to call
*/
service: string;

/**
* The service action to call
*/
action: string;

/**
* The parameters for the service action
*/
parameters: any;
}

export interface AwsSdkJsCustomResourceProps {
/**
* The AWS SDK call to make when the resource is created.
* At least onCreate, onUpdate or onDelete must be specified.
*
* @default the call when the resource is updated
*/
onCreate?: AwsSdkCall;

/**
* The AWS SDK call to make when the resource is updated
*
* @default the call when the resource is created
*/
onUpdate?: AwsSdkCall;

/**
* THe AWS SDK call to make when the resource is deleted
*/
onDelete?: AwsSdkCall;

/**
* The IAM policy statements to allow the different calls. Use only if
* resource restriction is needed.
*
* @default Allow onCreate, onUpdate and onDelete calls on all resources ('*')
*/
policyStatements?: iam.PolicyStatement[];
}

export class AwsSdkJsCustomResource extends cdk.Construct {
/**
* The AWS SDK call made when the resource is created.
*/
public readonly onCreate?: AwsSdkCall;

/**
* The AWS SDK call made when the resource is udpated.
*/
public readonly onUpdate?: AwsSdkCall;

/**
* The AWS SDK call made when the resource is deleted.
*/
public readonly onDelete?: AwsSdkCall;

/**
* The IAM policy statements used by the lambda provider.
*/
public readonly policyStatements: iam.PolicyStatement[];

constructor(scope: cdk.Construct, id: string, props: AwsSdkJsCustomResourceProps) {
super(scope, id);

if (!props.onCreate && !props.onUpdate && !props.onDelete) {
throw new Error('At least `onCreate`, `onUpdate` or `onDelete` must be specified.');
}

this.onCreate = props.onCreate || props.onUpdate;
this.onUpdate = props.onUpdate || props.onCreate;
this.onDelete = props.onDelete;

const fn = new lambda.SingletonFunction(this, 'Function', {
code: lambda.Code.asset(path.join(__dirname, 'aws-sdk-js-caller')),
runtime: lambda.Runtime.NodeJS810,
handler: 'index.handler',
uuid: '679f53fa-c002-430c-b0da-5b7982bd2287'
});

if (props.policyStatements) {
props.policyStatements.forEach(statement => {
fn.addToRolePolicy(statement);
});
this.policyStatements = props.policyStatements;
} else { // Derive statements from AWS SDK calls
this.policyStatements = [];

[this.onCreate, this.onUpdate, this.onDelete].forEach(call => {
if (call) {
const statement = new iam.PolicyStatement()
.addAction(awsSdkToIamAction(call.service, call.action))
.addAllResources();
fn.addToRolePolicy(statement); // TODO: remove duplicates?
this.policyStatements.push(statement);
}
});
}

new CustomResource(this, 'Resource', {
lambdaProvider: fn,
properties: {
create: this.onCreate,
update: this.onUpdate,
delete: this.onDelete
}
});
}
}

/**
* Transform SDK service/action to IAM action using metadata from aws-sdk module.
* Example: CloudWatchLogs with putRetentionPolicy => logs:PutRetentionPolicy
*
* TODO: is this mapping correct for all services?
*/
function awsSdkToIamAction(service: string, action: string): string {
const srv = service.toLowerCase();
const iamService = awsSdkMetadata[srv].prefix || srv;
const iamAction = action.charAt(0).toUpperCase() + action.slice(1);
return `${iamService}:${iamAction}`;
}
96 changes: 95 additions & 1 deletion packages/@aws-cdk/aws-cloudformation/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion packages/@aws-cdk/aws-cloudformation/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,12 @@
"@aws-cdk/aws-iam": "^0.24.1",
"@aws-cdk/aws-lambda": "^0.24.1",
"@aws-cdk/aws-sns": "^0.24.1",
"@aws-cdk/cdk": "^0.24.1"
"@aws-cdk/cdk": "^0.24.1",
"aws-sdk": "^2.409.0"
},
"bundledDependencies": [
"aws-sdk"
],
"homepage": "https://github.com/awslabs/aws-cdk",
"peerDependencies": {
"@aws-cdk/aws-codepipeline-api": "^0.24.1",
Expand Down