feat(applets): integrate into toolkit #1039

rix0rrr · 2018-10-30T11:43:35Z

Integrate the Applet runner into the toolkit. When the --app argument
points to a .yml or .yaml file, the target will be assumed to be a
JavaScript applet, and run through the applet interpreter.

Enhancements to JS applet interpreter:

Allow using non-Stack constructs.
Multiple Stacks in one applet file by supplying an array.
Allow referencing packages directly from NPM.

Non-Stack constructs can now be used as applets. If a non-Stack applet
is selected, a Stack will be constructed around it. This makes it
possible to reuse Applet constructs in regular CDK applications (if they
meet the requirements).

Multiple applets can be define in one file by making the top-level
object in the YAML file a list. Each applet will get its own stack.

By starting an applet specifier with npm://, applet modules can
directly be referenced in NPM. You can include a version specifier
(@1.2.3) to reference specific versions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Integrate the Applet runner into the toolkit. When the --app argument points to a .yml or .yaml file, the target will be assumed to be a JavaScript applet, and run through the applet interpreter. Enhancements to JS applet interpreter: - Allow using non-Stack constructs. - Multiple Stacks in one applet file by supplying an array. - Allow referencing packages directly from NPM. Non-Stack constructs can now be used as applets. If a non-Stack applet is selected, a Stack will be constructed around it. This makes it possible to reuse Applet constructs in regular CDK applications (if they meet the requirements). Multiple applets can be define in one file by making the top-level object in the YAML file a list. Each applet will get its own stack. By starting an applet specifier with npm://, applet modules can directly be referenced in NPM. You can include a version specifier (@1.2.3) to reference specific versions.

eladb

I am wondering about the file format for multiple applets (and also the fact that the name of the stack is not explicitly defined). How about we design this to look like a classic desired state configuration file (CFN template):

applets:
    applet1:
        type: <applet-type>
        properties:
            <properties>
    applet2:
        type: <applet2-type>
        properties:
            <properties>

The benefits:

It's a familiar format, and maybe at somepoint, this will just be a CloudFormation template (there are some ideas circling around this).
It will allow us to support references between applets (${applet1.output})
The name of the applet can be used to determine the stack name.

eladb · 2018-11-01T10:10:05Z

packages/aws-cdk/lib/exec.ts

+import { debug } from './logging';
+
+/** Invokes the cloud executable and returns JSON output */
+export async function execProgram(aws: SDK, config: Settings): Promise<cxapi.SynthesizeResponse> {


The refactor makes it a bit hard to understand what had changed here.

I understand, but I couldn't bear to keep on adding to the same file anymore.

The only functional change is the addition of the executable guessing.

eladb · 2018-11-01T10:12:29Z

packages/aws-cdk/lib/exec.ts

+/**
+ * Guess the executable from the command-line argument
+ */
+async function guessExecutable(commandLine: string[]) {


I was actually thinking we can design this a bit more generally and make people happy. For example, if app is a .js file, we can determine that it can be executed with node, if it's a .dll, we can use dotnet, etc. What do you think?

I was also thinking that we can introduce auto-detection (i.e. if there's a package.json file, or pom.xml, we can auto-detect where the CDK app is), but that's a later thing.

For now, I think adding support for .js could be nice and maybe implement this a bit more generically (extension to programming mapping).

Yes, that seems reasonable.

packages/@aws-cdk/applet-js/test/test.applets.ts

packages/@aws-cdk/applet-js/bin/cdk-applet-js.ts

eladb · 2018-11-01T15:28:36Z

packages/@aws-cdk/applet-js/test/test-nonstack.yaml

-#!/usr/bin/env cdk-applet-js
-applet: ./test-applet:NoStackApplet
-argument: this should be reflected in the template description
+Applets:


Any chance I can convince you to use lower case letters for the schema?

You can, and I'm not liking this either.

But then let's not mimic the CloudFormation schema at all. Otherwise it's just going to be confusing, don't you think?

Both Azure Resource Manager and Google Cloud Deployment Manager use very similar syntaxes. This is basically the standard-de-facto for desired state configuration.

eladb

Please Update applets topic in docs

Bug Fixes ========= * **aws-autoscaling:** allow minSize to be set to 0 ([#1015](#1015)) ([67f7fa1](67f7fa1)) * **aws-codebuild:** correctly pass the timeout property to CFN when creating a Project. ([#1071](#1071)) ([b1322bb](b1322bb)) * **aws-codebuild:** correctly set S3 path when using it as artifact. ([#1072](#1072)) ([f32cba9](f32cba9)) * **aws-kms:** add output value when exporting an encryption key ([#1036](#1036)) ([cb490be](cb490be)) * Switch from `js-yaml` to `yaml` ([#1092](#1092)) ([0b132b5](0b132b5)) Features ========= * **applets:** integrate into toolkit ([#1039](#1039)) ([fdabe95](fdabe95)), closes [#849](#849) [#342](#342) [#291](#291) * **aws-codecommit:** use CloudWatch Events instead of polling by default in the CodePipeline Action. ([#1026](#1026)) ([d09d30c](d09d30c)) * **aws-dynamodb:** allow specifying partition/sort keys in props ([#1054](#1054)) ([ec87331](ec87331)), closes [#1051](#1051) * **aws-ec2:** AmazonLinuxImage supports AL2 ([#1081](#1081)) ([97b57a5](97b57a5)), closes [#1062](#1062) * **aws-lambda:** high level API for event sources ([#1063](#1063)) ([1be3442](1be3442)) * **aws-sqs:** improvements to IAM grants API ([#1052](#1052)) ([6f2475e](6f2475e)) * don't upload the same asset multiple times ([#1011](#1011)) ([35937b6](35937b6)), closes [#989](#989) * **codepipeline/cfn:** Use fewer statements for pipeline permissions ([#1009](#1009)) ([8f4c2ab](8f4c2ab)) * add a new construct library for ECS ([#1058](#1058)) ([ae03ddb](ae03ddb)) * **pkglint:** Make sure .snk files are ignored ([#1049](#1049)) ([53c8d76](53c8d76)), closes [#643](#643) * **toolkit:** deployment ui improvements ([#1067](#1067)) ([c832eaf](c832eaf)) BREAKING CHANGES ========= * The ec2.Connections object has been changed to be able to manage multiple security groups. The relevant property has been changed from `securityGroup` to `securityGroups` (an array of security group objects). * **aws-codecommit:** This modifies the default behavior of the CodeCommit Action. It also changes the internal API contract between the aws-codepipeline-api module and the CodePipeline Actions in the service packages. * **applets:** The applet schema has changed to allow Multiple applets can be define in one file by structuring the files like this: * **applets:** The applet schema has changed to allow definition of multiple applets in the same file. The schema now looks like this: applets: MyApplet: type: ./my-applet-file properties: property1: value ... By starting an applet specifier with npm://, applet modules can directly be referenced in NPM. You can include a version specifier (@1.2.3) to reference specific versions. * **aws-sqs:** `queue.grantReceiveMessages` has been removed. It is unlikely that this would be sufficient to interact with a queue. Alternatively you can use `queue.grantConsumeMessages` or `queue.grant('sqs:ReceiveMessage')` if there's a need to only grant this action.

Bug Fixes ======== * **aws-autoscaling:** allow minSize to be set to 0 ([#1015](#1015)) ([67f7fa1](67f7fa1)) * **aws-codebuild:** correctly pass the timeout property to CFN when creating a Project. ([#1071](#1071)) ([b1322bb](b1322bb)) * **aws-codebuild:** correctly set S3 path when using it as artifact. ([#1072](#1072)) ([f32cba9](f32cba9)) * **aws-kms:** add output value when exporting an encryption key ([#1036](#1036)) ([cb490be](cb490be)) * Switch from `js-yaml` to `yaml` ([#1092](#1092)) ([0b132b5](0b132b5)) Features ======== * don't upload the same asset multiple times ([#1011](#1011)) ([35937b6](35937b6)), closes [#989](#989) * **app-delivery:** CI/CD for CDK Stacks ([#1022](#1022)) ([f2fe4e9](f2fe4e9)) * add a new construct library for ECS ([#1058](#1058)) ([ae03ddb](ae03ddb)) * **applets:** integrate into toolkit ([#1039](#1039)) ([fdabe95](fdabe95)), closes [#849](#849) [#342](#342) [#291](#291) * **aws-codecommit:** use CloudWatch Events instead of polling by default in the CodePipeline Action. ([#1026](#1026)) ([d09d30c](d09d30c)) * **aws-dynamodb:** allow specifying partition/sort keys in props ([#1054](#1054)) ([ec87331](ec87331)), closes [#1051](#1051) * **aws-ec2:** AmazonLinuxImage supports AL2 ([#1081](#1081)) ([97b57a5](97b57a5)), closes [#1062](#1062) * **aws-lambda:** high level API for event sources ([#1063](#1063)) ([1be3442](1be3442)) * **aws-sqs:** improvements to IAM grants API ([#1052](#1052)) ([6f2475e](6f2475e)) * **codepipeline/cfn:** Use fewer statements for pipeline permissions ([#1009](#1009)) ([8f4c2ab](8f4c2ab)) * **pkglint:** Make sure .snk files are ignored ([#1049](#1049)) ([53c8d76](53c8d76)), closes [#643](#643) * **toolkit:** deployment ui improvements ([#1067](#1067)) ([c832eaf](c832eaf)) * Update to CloudFormation resource specification v2.11.0 BREAKING CHANGES ======== * The ec2.Connections object has been changed to be able to manage multiple security groups. The relevant property has been changed from `securityGroup` to `securityGroups` (an array of security group objects). * **aws-codecommit:** this modifies the default behavior of the CodeCommit Action. It also changes the internal API contract between the aws-codepipeline-api module and the CodePipeline Actions in the service packages. * **applets:** The applet schema has changed to allow Multiple applets can be define in one file by structuring the files like this: * **applets:** The applet schema has changed to allow definition of multiple applets in the same file. The schema now looks like this: applets: MyApplet: type: ./my-applet-file properties: property1: value ... By starting an applet specifier with npm://, applet modules can directly be referenced in NPM. You can include a version specifier (@1.2.3) to reference specific versions. * **aws-sqs:** `queue.grantReceiveMessages` has been removed. It is unlikely that this would be sufficient to interact with a queue. Alternatively you can use `queue.grantConsumeMessages` or `queue.grant('sqs:ReceiveMessage')` if there's a need to only grant this action.

rix0rrr requested review from RomainMuller and eladb October 30, 2018 11:43

rix0rrr self-assigned this Oct 30, 2018

Rico Huijbers added 2 commits November 1, 2018 10:48

Merge remote-tracking branch 'origin/master' into huijbers/applets

6d7f8f6

Remove trailing test files

23ce00c

eladb suggested changes Nov 1, 2018

View reviewed changes

Rico Huijbers added 2 commits November 1, 2018 15:08

Fix location of temporary directory

d509d03

Implement new applet syntax, factor out types

f0065c1

eladb reviewed Nov 1, 2018

View reviewed changes

Switch to lowercase schema language

ee2826f

eladb approved these changes Nov 2, 2018

View reviewed changes

Update docs

e850b13

rix0rrr merged commit fdabe95 into master Nov 2, 2018

rix0rrr deleted the huijbers/applets branch November 2, 2018 14:38

rix0rrr mentioned this pull request Nov 6, 2018

v0.15.0 #1094

Merged

NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(applets): integrate into toolkit #1039

feat(applets): integrate into toolkit #1039

rix0rrr commented Oct 30, 2018

eladb left a comment

eladb Nov 1, 2018

rix0rrr Nov 1, 2018

eladb Nov 1, 2018

rix0rrr Nov 1, 2018

eladb Nov 1, 2018

rix0rrr Nov 1, 2018

eladb Nov 1, 2018

rix0rrr Nov 2, 2018

eladb left a comment

feat(applets): integrate into toolkit #1039

feat(applets): integrate into toolkit #1039

Conversation

rix0rrr commented Oct 30, 2018

eladb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eladb left a comment

Choose a reason for hiding this comment