feat(aws-ec2): signal, download and execute helpers for UserData #6029

grbartel · 2020-01-30T15:42:37Z

User Data objects currently only supports adding commands by providing the full command as a string. This commit hopes to address this by adding the following functionality:

On Exit Commands - Both bash and powershell have the concepts of trap functions which can be used to force a function to run when a an exception is run. Using this we are able to set up a script block that will always run at the end of the script.
add Signal Command - Using the above on Exit commands we are able to make it so the User data will send a signal to a specific resource (eg. Instance/Auto scaling group) with the results of the last command.
Download S3 File Command - This adds commands to download the specified file using the aws cli on linux and AWS powershell utility on windows
Execute File Command - This adds commands to ensure that the specified file is executable then executes the file with specified arguments.

This was tested by launching instances which pulled a script from an S3 asset then signaling on completion. One example test app was:

import cdk = require('@aws-cdk/core');
import ec2 = require("@aws-cdk/aws-ec2");
import iam = require("@aws-cdk/aws-iam")
import { Asset } from '@aws-cdk/aws-s3-assets';
import path = require('path');

export class TestappStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = new ec2.Vpc(this, 'VPC');

     const linstance = new ec2.Instance(this, 'Linstance', {
      vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.NANO),
         machineImage: new ec2.AmazonLinuxImage({ generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2 }),
      vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
      keyName: "LaptopInstances",
      resourceSignalTimeout: cdk.Duration.minutes(30 ),
    });
    linstance.addToRolePolicy(new iam.PolicyStatement({
      resources: [this.stackId],
      actions: ['cloudformation:SignalResource'] }));

    linstance.userData.addSignalOnExitCommand(linstance);
    linstance.userData.addCommands( `echo $-` )
    linstance.addUserData( 'echo "THIS IS A TEST"' );

    const linAsset = new Asset(this, 'linAsset', {
      path: path.join(__dirname, 'linux_asset.sh')
    });

    const linuxLocal = linstance.userData.addS3DownloadCommand({
      bucket:linAsset.bucket,
      bucketKey:linAsset.s3ObjectKey,
    } );

    linstance.userData.addExecuteFileCommand({
      filePath:linuxLocal,
      arguments: 'google.com -c 5'
      });

    linAsset.grantRead( linstance.role );

    const winstance = new ec2.Instance(this, 'Winstance', {
      vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.NANO),
      machineImage: new ec2.WindowsImage(ec2.WindowsVersion.WINDOWS_SERVER_2019_ENGLISH_FULL_BASE),
      vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
      keyName: "LaptopInstances",
      resourceSignalTimeout: cdk.Duration.minutes(30 ),
    });

    winstance.addToRolePolicy(new iam.PolicyStatement({
      resources: [this.stackId],
      actions: ['cloudformation:SignalResource'] }));


    const winAsset = new Asset(this, 'winAsset', {
      path: path.join(__dirname, 'win_asset.bat')
    });

    winstance.userData.addSignalOnExitCommand(winstance);
    // winstance.userData.addCommands( "write-output test" );
    winstance.userData.addCommands( "Set-PSDebug -Trace 1" );


    winstance.userData.addDownloadAndExecuteS3FileCommand({
        bucket:winAsset.bucket,
        bucketKey:winAsset.s3ObjectKey,
        arguments:"google.com -n 5",
    });

    const winLocal = winstance.userData.addS3DownloadCommand({
      bucket:winAsset.bucket,
      bucketKey:winAsset.s3ObjectKey,
    } );

    winstance.userData.addExecuteFileCommand({
      filePath:winLocal,
      arguments: 'google.com -c 5'
      });


    winAsset.grantRead( winstance.role );
  }
}

The asset referenced in this test was the following script:

echo "Hello World"
echo $@
ping "$@"

Fixes #623

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Commit Message

feat(aws-ec2): signal, download and execute helpers on UserData

User Data objects currently only supports adding commands by providing the full command as a string. This commit hopes to address this by adding the following functionality:

On Exit Commands - Both bash and powershell have the concepts of trap functions which can be used to force a function to run when a an exception is run. Using this we are able to set up a script block that will always run at the end of the script.
add Signal Command - Using the above on Exit commands we are able to make it so the User data will send a signal to a specific resource (eg. Instance/Auto scaling group) with the results of the last command.
Download S3 File Command - This adds commands to download the specified file using the aws cli on linux and AWS powershell utility on windows
Execute File Command - This adds commands to ensure that the specified file is executable then executes the file with specified arguments.

User Data objects currently only supports adding commands by providing the full command as a string. This commit hopes to address this by adding the following functionality: * On Exit Commands - Both bash and powershell have the concepts of trap functions which can be used to force a function to run when a an exception is run. Using this we are able to set up a script block that will always run at the end of the script. * add Signal Command - Using the above on Exit commands we are able to make it so the User data will send a signal to a specific resource (eg. Instance/Auto scaling group) with the results of the last command. * Download and Execute a file from S3 - This writes a function into the user data which can be used for repeated calls to download and execute a file from s3 with a list of arguments. This was tested by launching instances which pulled a script from an S3 asset then signaling on completion. Fixes aws#623

aws-cdk-automation · 2020-01-30T16:31:37Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…arguments to the customer.

aws-cdk-automation · 2020-01-30T19:25:21Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

grbartel · 2020-02-05T15:07:55Z

It has been almost a week since this pull request was created and there haven't been any comments on it yet. Is there anything that needs to be updated on this or any questions about it that I can answer?

rix0rrr

Thanks for working on this! I hope we can simplify it somewhat (and make it more generic), but this is great!

rix0rrr · 2020-02-10T09:57:07Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+  /**
+   * Adds a command to download a file from S3
+   */
+  public abstract addDownloadAndExecuteS3FileCommand(params: S3DownloadAndExecuteOptions): void;


This function looks like it serves a very specific use case, that I'm not sure is common enough to warrant inclusion in the out-of-the-box experience.

I would already feel better if we could split this into separate download and execute commands.

I have split these into separate commands now.

rix0rrr · 2020-02-10T09:57:47Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+  /**
+   * Name of the bucket to download from
+   */
+  readonly bucketName: string;


Feels like this should be an IBucket, and also that the instance role should get bucket.grantRead() called.

Sure I can swap this to an IBucket.

However I disagree that the User Data should be responsible for granting read permissions for the bucket since the User Data can be created separately from the instance and then passed in later and the UserData has no knowledge of what it is being used for.

rix0rrr · 2020-02-10T09:59:35Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+  }
+
+  public addDownloadAndExecuteS3FileCommand( params: S3DownloadAndExecuteOptions ): void {
+    if (!this.functionsAdded.has('download_and_execute_s3_file')) {


Why put this in a bash function? If it's about UserData length, feels like the code can be written a lot more concisely if it wasn't in a bash function.

Is there value in considering an option to materialize a function for reuse? Say the user is downloading multiple scripts, and would like to cut down on the amount of boilerplate?

I don't think so. The final download command is 3 lines, one of which is logging. If UserData scripts executed with -x (which I'm nearly sure they do) that line is unnecessary, so it goes down to 2 lines. That's not a lot of boilerplate.

By default user data is not run under -x in fact echoing $- only gives us the flags hB on an AL2 instance

rix0rrr · 2020-02-10T10:00:09Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+          "echo \"Downloading file ${s3Path} to ${path}\";\n" +
+          "mkdir -p $(dirname ${path}) ;\n" +
+          "aws s3 cp ${s3Path} ${path};\n" +
+          "if [ $? -ne 0 ]; then exit 1;fi;\n" +


The script should be running under set -e, so this test is not necessary. (Verify that, make it true if it isn't)

By default the scripts are not being run under set -e as such the new execute file command is running set -e

rix0rrr · 2020-02-10T10:00:40Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+          "mkdir -p $(dirname ${path}) ;\n" +
+          "aws s3 cp ${s3Path} ${path};\n" +
+          "if [ $? -ne 0 ]; then exit 1;fi;\n" +
+          "chmod +x ${path};\n" +


Feels like execute should be its own function.

rix0rrr · 2020-02-10T10:02:51Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+  }
+
+  public addDownloadAndExecuteS3FileCommand( params: S3DownloadAndExecuteOptions ): void {
+    if (!this.functionsAdded.has('download_and_execute_s3_file')) {


Same comments here as about the bash function.

rix0rrr · 2020-02-10T10:05:31Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

@@ -101,7 +260,19 @@ class CustomUserData extends UserData {
    this.lines.push(...commands);
  }

+  public addOnExitCommands(...commands: string[]): void {
+    this.onExitLines.push(...commands);


You can't really pretend to support onExitLines if they're going to do something completely different. Better throw here as well. Please replace all the throws with more descriptive error messages (informing users about the likely mistake they made in THEIR code and what they should do to fix it), such as CustomUserData does not support FooCommand, use UserData.forLinux() or UserData.forWindows() instead.

Changed commands to no longer create bash/powershell functions. Added additional tests for custom userdata

Pull request has been modified.

rix0rrr · 2020-02-12T09:20:58Z

packages/@aws-cdk/aws-ec2/README.md

+A user data could be configured to run a script found in an asset through the following:
+```ts
+const asset = new Asset(this, 'Asset', {path: path.join(__dirname, 'configure.sh')});
+const instance = new ec2.Instance(this, 'Instance', {


Please fix the indentation and other whitespace here to at least be consistent.

Indentation of 2 spaces please, closing curly is not indented.

rix0rrr · 2020-02-12T09:21:28Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+/**
+ * Options when downloading files from S3
+ */
+export interface S3DownloadAndExecuteOptions {


No longer used.

rix0rrr · 2020-02-12T09:23:45Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+
+  public addExecuteFileCommand( params: ExecuteFileOptions): void {
+    this.addCommands(
+      `set -e`,


The download also needs to execute under set -e. An easier way to set this is to change the shebang.

Can you change the default shebang to #!/bin/bash -e ?

Potentially even #!/bin/bash -ex

I am not sure if we want to actually to modify the shebang like this since I would consider the following:

-e being a potentially breaking change for users. Anyone who was already adding -e manually would be fine but everyone else could run into unepected failures.

-x being a potential security risk since unless customers explicitly disabled it we could be printing secrets in the userdata.

That is a fair concern, but by that logic we can never improve any situation. At least -e should be the default for 99% of users; people who want/need something else because they do custom error handling should be able to deviate from the default if they want to.

I will take this change myself since I also predict it will be a lot of integ test work.

rix0rrr · 2020-02-12T09:27:51Z

packages/@aws-cdk/aws-ec2/lib/user-data.ts

+  /**
+   * Adds commands to download a file from S3
+   */
+  public abstract addS3DownloadCommand(params: S3DownloadOptions): string;


It's not completely obvious what this returns, so that might be worth a @returns annotation.

rix0rrr · 2020-02-12T09:29:21Z

packages/@aws-cdk/aws-ec2/package.json

@@ -73,9 +73,11 @@
  "dependencies": {
    "@aws-cdk/aws-cloudwatch": "1.23.0",
    "@aws-cdk/aws-iam": "1.23.0",
+    "@aws-cdk/aws-s3": "1.23.0"


Removed unused properties.

Pull request has been modified.

aws-cdk-automation · 2020-02-19T14:28:27Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 3882da7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-19T16:21:31Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 71fbdbe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Pull request has been modified.

aws-cdk-automation · 2020-02-26T22:25:51Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 6bfbe1a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

kiros-sc · 2020-05-04T14:00:14Z

I have a question, what would you recommend to use if someone needs to simply copy a script into an ec2 instance you are creating on cdk? In our case, my cdk creates the VPN instance and I need to upload the usercreation script to this instance....

ddneilson · 2020-05-04T14:35:18Z

I hace a question, what would you recommend to use if someone needs to simply copy a script into an ec2 instance you are creating on cdk... In our case, my cdk creates the VPN instance and I need to upload the usercreation script to this instance....

If I'm understanding your question correctly... we've used an S3 Asset ( https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3-assets.Asset.html ), and the UserData.addS3DownloadCommand() ( https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.UserData.html#add-wbr-s3-wbr-download-wbr-commandparams ) for this.

The S3 Asset is created when you deploy your stack, and is uploaded automatically to your CDK staging bucket. Then, the UserData will download the asset object from S3 to your instance during initialization; note that the return value of addS3DownloadCommand() is the full path to the file as it will appear on the instance. Follow the addS3DownloadCommand() with an addCommand() to unzip the bundle to where you want it and you're off to the races.

So, essentially, something like:

const asset = new Asset(this, "Bundle", { path: <path to your file(s) locally> });
const filepath = instance.UserData.addS3DownloadCommand({
    bucket: asset.bucket,
    bucketKey: asset.s3ObjectKey
});
instance.UserData.addCommand(
   `unzip ${filepath} -d <path you want the files in>`,
   `rm -f ${filepath}`
);

kiros-sc · 2020-05-04T15:43:17Z

@ddneilson Thank you very much, I'll try this tomorrow!!!

SomayaB assigned rix0rrr Jan 30, 2020

Changed Arguments to be a string to move responsibility for escaping …

d449165

…arguments to the customer.

rix0rrr previously requested changes Feb 10, 2020

View reviewed changes

Split Download and execute command into separate functions.

694050d

Changed commands to no longer create bash/powershell functions. Added additional tests for custom userdata

grbartel added 2 commits February 11, 2020 09:15

Merge branch 'master' into rich_user_data

d0e6595

Updated the aws-ec2 readme with information about UserData

7ec2893

rix0rrr previously requested changes Feb 12, 2020

View reviewed changes

Cleaned up README.md

0092a39

Removed unused properties.

Merge branch 'master' into rich_user_data

3882da7

Added missing Default from a docstring

71fbdbe

rix0rrr changed the title ~~feat(aws-ec2): rich user data~~ feat(aws-ec2): signal, download and execute helpers for UserData Feb 20, 2020

rix0rrr previously approved these changes Feb 20, 2020

View reviewed changes

Merge branch 'master' into rich_user_data

6bfbe1a

rix0rrr merged commit ee8f169 into aws:master Feb 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(aws-ec2): signal, download and execute helpers for UserData #6029

feat(aws-ec2): signal, download and execute helpers for UserData #6029

grbartel commented Jan 30, 2020 •

edited by rix0rrr

Loading

aws-cdk-automation commented Jan 30, 2020

aws-cdk-automation commented Jan 30, 2020

grbartel commented Feb 5, 2020

rix0rrr left a comment

rix0rrr Feb 10, 2020

grbartel Feb 11, 2020

rix0rrr Feb 10, 2020

grbartel Feb 10, 2020

rix0rrr Feb 10, 2020

ddneilson Feb 10, 2020

rix0rrr Feb 12, 2020

grbartel Feb 12, 2020

rix0rrr Feb 10, 2020

grbartel Feb 11, 2020

rix0rrr Feb 10, 2020

rix0rrr Feb 10, 2020

rix0rrr Feb 10, 2020

grbartel Feb 11, 2020

rix0rrr Feb 12, 2020

rix0rrr Feb 12, 2020

rix0rrr Feb 12, 2020

rix0rrr Feb 12, 2020

grbartel Feb 12, 2020

rix0rrr Feb 20, 2020

rix0rrr Feb 12, 2020

rix0rrr Feb 12, 2020

aws-cdk-automation commented Feb 19, 2020

aws-cdk-automation commented Feb 19, 2020

aws-cdk-automation commented Feb 26, 2020

kiros-sc commented May 4, 2020 •

edited

Loading

ddneilson commented May 4, 2020

kiros-sc commented May 4, 2020

feat(aws-ec2): signal, download and execute helpers for UserData #6029

feat(aws-ec2): signal, download and execute helpers for UserData #6029

Conversation

grbartel commented Jan 30, 2020 • edited by rix0rrr Loading

Commit Message

aws-cdk-automation commented Jan 30, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Jan 30, 2020

AWS CodeBuild CI Report

grbartel commented Feb 5, 2020

rix0rrr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Feb 19, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Feb 19, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Feb 26, 2020

AWS CodeBuild CI Report

kiros-sc commented May 4, 2020 • edited Loading

ddneilson commented May 4, 2020

kiros-sc commented May 4, 2020

grbartel commented Jan 30, 2020 •

edited by rix0rrr

Loading

kiros-sc commented May 4, 2020 •

edited

Loading