-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(cloudformation): aws custom resource #1850
Merged
Merged
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
4a8e4b6
feat(cloudformation): aws sdk js custom resource
jogold 5ab6588
Log AWS SDK version
jogold 15a6ef8
Add tests
jogold 34a3ebc
Merge branch 'master' into aws-sdk-js-resource
jogold ad86457
Return API response as output attributes
jogold 84c73ab
Remove default on onUpdate
jogold 5ba7765
Add options to specify physical resource id
jogold 6f1c0c6
Merge branch 'master' into aws-sdk-js-resource
jogold 5811c94
Rename to AwsCustomResource
jogold 941913b
Add readonly
jogold 552a5e7
Merge branch 'master' into aws-sdk-js-resource
jogold ae446b9
Write provider in ts
jogold 2fb5163
Support path for physical resource id
jogold 04c8df5
Typo
jogold 8ddf39f
Merge branch 'master' into aws-sdk-js-resource
jogold 82d1ea1
Fix bad pkglint version after merge
jogold 30d3226
Merge branch 'master' into aws-sdk-js-resource
jogold a26be81
Add option to catch API errors
jogold 7e9b21c
Merge branch 'aws-sdk-js-resource' of github.com:jogold/aws-cdk into …
jogold b5fc424
Restore package-lock.json in aws-codepipeline-actions
jogold 25691a0
Merge branch 'master' into aws-sdk-js-resource
jogold 1b7f392
CustomResourceProvider
jogold 598f605
exclude construct-ctor-props-optional
jogold 290d77c
remove duplicate statements check
jogold 8981600
add option to lock api version
jogold c423f35
JSDoc
jogold d7b66cc
fix booleans in parameters
jogold e8813f5
update integration test
jogold 1b204d8
update README
jogold 700fa4d
Merge branch 'master' into aws-sdk-js-resource
jogold e02480f
update integ test
jogold File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
1 change: 1 addition & 0 deletions
1
packages/@aws-cdk/aws-cloudformation/lib/aws-sdk-js-caller/.gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
!*.js |
52 changes: 52 additions & 0 deletions
52
packages/@aws-cdk/aws-cloudformation/lib/aws-sdk-js-caller/index.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
const AWS = require('aws-sdk'); | ||
|
||
exports.handler = async function(event, context) { | ||
try { | ||
console.log(JSON.stringify(event)); | ||
|
||
if (event.ResourceProperties[event.RequestType]) { | ||
const { service, action, parameters } = event.ResourceProperties[event.RequestType]; | ||
const awsService = new AWS[service](); | ||
await awsService[action](parameters).promise(); | ||
} | ||
|
||
await respond('SUCCESS', 'OK'); | ||
} catch (e) { | ||
console.log(e); | ||
await respond('FAILED', e.message); | ||
} | ||
|
||
function respond(responseStatus, reason) { | ||
const responseBody = JSON.stringify({ | ||
Status: responseStatus, | ||
Reason: reason, | ||
PhysicalResourceId: context.logStreamName, | ||
StackId: event.StackId, | ||
RequestId: event.RequestId, | ||
LogicalResourceId: event.LogicalResourceId, | ||
NoEcho: false, | ||
Data: {} | ||
}); | ||
|
||
console.log('Responding', JSON.stringify(responseBody)); | ||
|
||
const parsedUrl = require('url').parse(event.ResponseURL); | ||
const requestOptions = { | ||
hostname: parsedUrl.hostname, | ||
path: parsedUrl.path, | ||
method: 'PUT', | ||
headers: { 'content-type': '', 'content-length': responseBody.length } | ||
}; | ||
|
||
return new Promise((resolve, reject) => { | ||
try { | ||
const request = require('https').request(requestOptions, resolve); | ||
request.on('error', reject); | ||
request.write(responseBody); | ||
request.end(); | ||
} catch (e) { | ||
reject(e); | ||
} | ||
}); | ||
} | ||
} |
145 changes: 145 additions & 0 deletions
145
packages/@aws-cdk/aws-cloudformation/lib/aws-sdk-js-custom-resource.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,145 @@ | ||
import iam = require('@aws-cdk/aws-iam'); | ||
import lambda = require('@aws-cdk/aws-lambda'); | ||
import cdk = require('@aws-cdk/cdk'); | ||
import metadata = require('aws-sdk/apis/metadata.json'); | ||
import path = require('path'); | ||
import { CustomResource } from './custom-resource'; | ||
|
||
/** | ||
* AWS SDK service metadata. | ||
*/ | ||
export type AwsSdkMetadata = {[key: string]: any}; | ||
|
||
const awsSdkMetadata: AwsSdkMetadata = metadata; | ||
|
||
/** | ||
* An AWS SDK call. | ||
*/ | ||
export interface AwsSdkCall { | ||
/** | ||
* The service to call | ||
*/ | ||
service: string; | ||
|
||
/** | ||
* The service action to call | ||
*/ | ||
action: string; | ||
|
||
/** | ||
* The parameters for the service action | ||
*/ | ||
parameters: any; | ||
} | ||
|
||
export interface AwsSdkJsCustomResourceProps { | ||
/** | ||
* The AWS SDK call to make when the resource is created. | ||
* At least onCreate, onUpdate or onDelete must be specified. | ||
* | ||
* @default the call when the resource is updated | ||
*/ | ||
onCreate?: AwsSdkCall; | ||
|
||
/** | ||
* The AWS SDK call to make when the resource is updated | ||
* | ||
* @default the call when the resource is created | ||
*/ | ||
onUpdate?: AwsSdkCall; | ||
|
||
/** | ||
* THe AWS SDK call to make when the resource is deleted | ||
*/ | ||
onDelete?: AwsSdkCall; | ||
|
||
/** | ||
* The IAM policy statements to allow the different calls. Use only if | ||
* resource restriction is needed. | ||
* | ||
* @default Allow onCreate, onUpdate and onDelete calls on all resources ('*') | ||
*/ | ||
policyStatements?: iam.PolicyStatement[]; | ||
} | ||
|
||
export class AwsSdkJsCustomResource extends cdk.Construct { | ||
/** | ||
* The AWS SDK call made when the resource is created. | ||
*/ | ||
public readonly onCreate?: AwsSdkCall; | ||
|
||
/** | ||
* The AWS SDK call made when the resource is udpated. | ||
*/ | ||
public readonly onUpdate?: AwsSdkCall; | ||
|
||
/** | ||
* The AWS SDK call made when the resource is deleted. | ||
*/ | ||
public readonly onDelete?: AwsSdkCall; | ||
|
||
/** | ||
* The IAM policy statements used by the lambda provider. | ||
*/ | ||
public readonly policyStatements: iam.PolicyStatement[]; | ||
|
||
constructor(scope: cdk.Construct, id: string, props: AwsSdkJsCustomResourceProps) { | ||
super(scope, id); | ||
|
||
if (!props.onCreate && !props.onUpdate && !props.onDelete) { | ||
throw new Error('At least `onCreate`, `onUpdate` or `onDelete` must be specified.'); | ||
} | ||
|
||
this.onCreate = props.onCreate || props.onUpdate; | ||
this.onUpdate = props.onUpdate || props.onCreate; | ||
this.onDelete = props.onDelete; | ||
|
||
const fn = new lambda.SingletonFunction(this, 'Function', { | ||
code: lambda.Code.asset(path.join(__dirname, 'aws-sdk-js-caller')), | ||
runtime: lambda.Runtime.NodeJS810, | ||
handler: 'index.handler', | ||
uuid: '679f53fa-c002-430c-b0da-5b7982bd2287' | ||
}); | ||
|
||
if (props.policyStatements) { | ||
props.policyStatements.forEach(statement => { | ||
fn.addToRolePolicy(statement); | ||
}); | ||
this.policyStatements = props.policyStatements; | ||
} else { // Derive statements from AWS SDK calls | ||
this.policyStatements = []; | ||
|
||
[this.onCreate, this.onUpdate, this.onDelete].forEach(call => { | ||
if (call) { | ||
const statement = new iam.PolicyStatement() | ||
.addAction(awsSdkToIamAction(call.service, call.action)) | ||
.addAllResources(); | ||
fn.addToRolePolicy(statement); // TODO: remove duplicates? | ||
this.policyStatements.push(statement); | ||
} | ||
}); | ||
} | ||
|
||
new CustomResource(this, 'Resource', { | ||
lambdaProvider: fn, | ||
properties: { | ||
create: this.onCreate, | ||
update: this.onUpdate, | ||
delete: this.onDelete | ||
} | ||
}); | ||
} | ||
} | ||
|
||
/** | ||
* Transform SDK service/action to IAM action using metadata from aws-sdk module. | ||
* Example: CloudWatchLogs with putRetentionPolicy => logs:PutRetentionPolicy | ||
* | ||
* TODO: is this mapping correct for all services? | ||
*/ | ||
function awsSdkToIamAction(service: string, action: string): string { | ||
const srv = service.toLowerCase(); | ||
const iamService = awsSdkMetadata[srv].prefix || srv; | ||
const iamAction = action.charAt(0).toUpperCase() + action.slice(1); | ||
return `${iamService}:${iamAction}`; | ||
} |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The physical resource ID is actually a pretty important thing to get right when defining custom resources. I am wondering if we can do better than logStreamName
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, if we want to avoid calling the
onDelete
during aUPDATE_COMPLETE_CLEANUP_IN_PROGRESS
when a parameter of theonUpdate
changes we need something like a constant here, no? can we use theLogicalResourceId
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a clever idea. To be useful, this resource is going to be an exercise in metaprogramming.
Logical ID wouldn't be good enough. I think a template string of sorts will need to come from the parameter. Also wonder if we might want to call multiple functions.
At least we'd want to explode the API result into a set of output attributes, I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, I think we will need to let users specify the path to where the physical ID can be found in the response JSON (at least), and this mechanism can also be used to produced a bunch of "GetAtt" attributes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the response only or also in the request? If you look at the example with the
putTargets
call what would you specify as physical ID?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The physical resource ID parameters needs to be able to be JSONPath member. It means it's not an ID by itself, but it's instructions for how to get the ID out of the API call response. So it should be able to be something like:
And the custom resource should get the value out of the response object. Probably should be able to be different for both Create and Update calls.
Now, whether it can be EITHER a literal or a JSONPath I'm not sure on. I'm sure there are cases in which a literal would make sense, so I'm not against disallowing it, but the JSONPath-style query needs to be possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P.S: When I say JSONPath, I'm cool with faking it by splitting on
.
and leaving it there, I don't mean literally importing a jsonpath library, we don't have a good enough story around bundled lambda dependencies to make that work.