(aws-ec2): InitFile construct IDs are interpolated from target_filename when using fromAsset() or fromSource() which can cause naming collisions

[its-mirus-lu]

What is the problem?

TLDR: Naming collision of InitFile constructs for multiple ec2 instances defined in the same stack definition when using the fromAsset or fromSource methods. Affected file: https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-ec2/lib/cfn-init-elements.ts. Cause is most likely due to the underlying InitFile fromAsset and fromSource methods creating an s3 asset with interpolated id of ${targetFileName}Asset.

The Scenario

I'm deploying a fleet of instances, each requiring a configuration file (unique to each instance) to be placed in the same directory in each instance. For example instance-a has a config file that has a config file that is different to that of instance-b. In both instances, the file is placed in the path /etc/gitlab-runner/config.toml.

I've leveraged cfn-init and the cdk aws-ec2.InitFile.fromAsset() construct and method to create the file on each instance.

When synthesizing, the following error is returned:

jsii.errors.JSIIError: There is already a Construct with name '--etc--gitlab-runner--config.tomlAsset' in Stack

Reproduction Steps

1. Create a stack via cdk
2. Define an ec2 instance instance-a in the stack
3. Define a cfn-init configuration for instance-a via aws_ec2.InitConfig
4. Within the InitConfig for this instance, add an InitFile element like this:

ec2.InitFile.from_asset(
    target_file_name="/etc/gitlab-runner/config.toml",
    path=toml_file1.toml,
)

4. In the same stack, define a second ec2 for instance-b
5. Define a cfn-init configuration for instance-a via another aws_ec2.InitConfig,
6. Within the InitConfig for this instance, add an InitFile element like this:

ec2.InitFile.from_asset(
    target_file_name="/etc/gitlab-runner/config.toml",
    path=toml_file2.toml,
)

7. Attempt to synthesize CloudFormation from the code

What did you expect to happen?

I expected each InitFile to have a uniquely interpolated construct Id, and for there not to be a collision in names.

What actually happened?

Both constructs for InitFile were named --etc--gitlab-runner--config.tomlAsset resulting in the synth error jsii.errors.JSIIError: There is already a Construct with name '--etc--gitlab-runner--config.tomlAsset' in Stack [...]

The construct Id for Init file is interpolated from the target_file_name, which will cause an issue if multiple EC2 instances are defined in the same stack, each which use the InitFile construct to place files in the same folder in their respective instances.

CDK CLI Version

1.24.0

Framework Version

1.24.0

Node.js Version

v16.3.0

OS

MacOS Catalina 10.15.7

Language

Python

Language Version

Python 3.9.7

Other information

Workaround

In an InitConfig:

1. When defining an InitFile element, specify a target_file_name that contains the instance-id,
2. Add another element of type InitCommand and move the file from step 1 to the final destination

Example:

config_toml = ec2.InitConfig(
    elements=[
        
        ec2.InitFile.from_asset(
            target_file_name="/etc/gitlab-runner/config" + logical_id + ".toml",
            path=toml_path,
        ),
        ec2.InitCommand.shell_command(
            shell_command="mv /etc/gitlab-runner/config"
            + logical_id
            + ".toml /etc/gitlab-runner/config.toml"
        ),
    ]
)

Possible Solution

Looking at https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-ec2/lib/cfn-init-elements.ts there are two places where an s3 asset is used as a staging location for files before being placed on an instance. This s3 asset construct's ID is causing the naming collision in cdk.

Classes InitFile and InitSource both use the same naming convention for the s3_asset construct id:

public static fromAsset(targetDirectory: string, path: string, options: InitSourceAssetOptions = {}): InitSource {
    return new class extends InitSource {
      protected _doBind(bindOptions: InitBindOptions) {
        const asset = new s3_assets.Asset(bindOptions.scope, **_`${targetDirectory}Asset`_**, {
          path,
          ...bindOptions,
        });

A fix might be to interpolate the s3_assets.Asset id as a combination of the filename and a unique hash that wouldn't change on each synth.

[njlynch]

Thanks for the bug report and the detailed write-up!

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[peterwoodworth]

This is still an issue, reopening. More repro code can be found here thanks to @bpauwels

[macdrorepo]

Any progress here?

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.