fix: import should write overrideTemplate to assembly.outdir

[nburtsev]

Issue # (if applicable)

Addresses #22530

Reason for this change

During cdk import template file is written to the project root, not to cdk.out which fails the following stack deployment.

Description of changes

I've just added a path.join similar to how it is done in regular template generation.

Description of how you validated changes

I've tested this locally, imports stacks just fine.

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

[nburtsev]

Exemption Request

Not sure if this should be an exemption or a clarification, but this only affects cdk import which has no tests at the moment, I've tested it locally but I don't think I can come up with tests to commit on my own.

[TheRealAmazonKendra]

This isn't related to the issue linked but I'm also not quite understanding your issue here. Can you clarify what you mean by During cdk import template file is written to the project root, not to cdk.out which fails the following stack deployment.?

[TheRealAmazonKendra]

Also, we do have cdk import tests. Unit tests are in https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk/test/import.test.ts and integ tests are in https://github.com/aws/aws-cdk/tree/main/packages/%40aws-cdk-testing/cli-integ/tests/cli-integ-tests.

[nburtsev]

@TheRealAmazonKendra Thank you very much for pointing the tests to me, I completely missed them.

Let me try to explain the problem. When the template is synthesized, if it is larger than 50kb and cannot be inlined in the request, CLI writes it to filesystem to be uploaded to S3. Unlike regular deployments import overrides the template, when it does so the code responsible for writing the large template to the filesystem does it to project root.

This code reproduces the issue:

import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as sqs from "aws-cdk-lib/aws-sqs";
import * as iam from "aws-cdk-lib/aws-iam";

export class AwsCdkImportBugStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    for (let i = 0; i < 70; i++) {
      const q = new sqs.Queue(this, `AwsCdkImportBugQueue${i}`, {
        visibilityTimeout: cdk.Duration.seconds(300),
        enforceSSL: true,
      });
    }

    // new sqs.Queue(this, `AwsCdkImportBugQueue`, {
    //   visibilityTimeout: cdk.Duration.seconds(300),
    //   enforceSSL: true,
    //   removalPolicy: cdk.RemovalPolicy.RETAIN,
    // });
  }
}

Deploy the stack, uncomment the queue, and try to import. This error happens:

$ npx cdk import                                  
AwsCdkImportBugStack
AwsCdkImportBugStack/AwsCdkImportBugQueue/Resource (AWS::SQS::Queue): enter QueueUrl (empty to skip): https://sqs.eu-central-1.amazonaws.com/<cut>/AwsCdkBugStack-keep186A2ECA-IznVNwytuY1b
AwsCdkImportBugStack/AwsCdkImportBugQueue/Policy/Resource (AWS::SQS::QueuePolicy): enter Id (empty to skip): 
Skipping import of AwsCdkImportBugStack/AwsCdkImportBugQueue/Policy/Resource
AwsCdkImportBugStack: importing resources into stack...
[0%] start: Publishing 06a2c6415fd2982e3285e9d615e5f9b99d1d795a9064479fbe6b88455ac62f6c:current
[100%] fail: ENOENT: no such file or directory, open '/home/nburtsev/projects/aws-cdk-import-bug/cdk.out/AwsCdkImportBugStack.template.json-06a2c6415fd2982e3285e9d615e5f9b99d1d795a9064479fbe6b88455ac62f6c.yaml'

 ❌  AwsCdkImportBugStack failed: Error: Failed to publish one or more assets. See the error messages above for more information.
    at publishAssets (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:420:84876)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async deployStack (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:430:391)
    at async ResourceImporter.importResources (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:433:171694)
    at async ResourceImporter.importResourcesFromMap (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:433:171357)
    at async CdkToolkit.import (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:433:205058)
    at async exec4 (/home/nburtsev/projects/aws-cdk-import-bug/node_modules/aws-cdk/lib/index.js:488:54378)

Failed to publish one or more assets. See the error messages above for more information.

The file is created, it's just in the project root not in cdk.out:

~/projects/aws-cdk-import-bug $ ls -la         
total 284
drwxr-xr-x   7 nburtsev users   4096 Mar 22 10:48 .
drwxr-xr-x   8 nburtsev users   4096 Mar 22 10:11 ..
-rw-r--r--   1 nburtsev users  64131 Mar 22 10:48 AwsCdkImportBugStack.template.json-06a2c6415fd2982e3285e9d615e5f9b99d1d795a9064479fbe6b88455ac62f6c.yaml
drwxr-xr-x   2 nburtsev users   4096 Mar 22 10:12 bin
-rw-r--r--   1 nburtsev users   3185 Mar 22 10:12 cdk.json
drwxr-xr-x   2 nburtsev users   4096 Mar 22 10:48 cdk.out
<cut>

Concerning tests, now that I know where they are I will attempt to adjust them. Please advise if adding another conditional resource creation in https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk-testing/cli-integ/resources/cdk-apps/app/app.js#L122 similar to what is already there, that would create 70 queues and then attempt the import (i.e. same as my example code does) is acceptable.

[MikeDombo]

Same problem as #22530

[TheRealAmazonKendra]

Ohhhhhhh. I see. Thanks for the explanation!

[TheRealAmazonKendra]

Well, this is fun. Turns out that cdk import test is broken and passing because it's not actually testing anything. Gimme a day (maybe two) to try and get that fixed and then I'll revisit your PR. With your explanation, I think your solution is likely correct but I have strong reservations about letting a change go through when the test is not testing.

[TheRealAmazonKendra]

Do not merge label added because of issues with test, not with PR.

[aws-cdk-automation]

This PR has been in the CHANGES REQUESTED state for 3 weeks, and looks abandoned. To keep this PR from being closed, please continue work on it. If not, it will automatically be closed in a week.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 3f3f730
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[nburtsev]

Figured I'd post this here, while it is being worked on.
If you encounter this issue - just move the file that's generated into cdk.out and run the import again, as long as you have the same inputs (or use json mapping for it) it will run just fine.

[aws-cdk-automation]

This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error.

[aws-cdk-automation]

The pull request linter fails with the following errors:

❌ Fixes must contain a change to an integration test file and the resulting snapshot.
❌ CLI code has changed. A maintainer must run the code through the testing pipeline (git fetch origin pull/29509/head && git push -f origin FETCH_HEAD:test-main-pipeline), then add the 'pr-linter/cli-integ-tested' label when the pipeline succeeds.

PRs must pass status checks before we can provide a meaningful review.

If you would like to request an exemption from the status checks or clarification on feedback, please leave a comment on this PR containing Exemption Request and/or Clarification Request.

✅ A exemption request has been requested. Please wait for a maintainer's review.