feat: implement node runtime region awareness for cdk vended custom resources #30108

colifran · 2024-05-08T17:33:53Z

Reason for this change

This PR introduces node runtime region awareness into the custom resource provider framework by doing the following:

Adding a LATEST_NODE_RUNTIME_MAP fact table used to maintain the latest node runtime available per AWS partition
Introducing determineLatestNodeRuntime and determineLatestNodeRuntimeName functions which determine the latest Lambda node Runtime and the latest Lambda node Runtime name, respectively
Updating the custom resource provider framework to utilize these two functions when code generating the runtime property for providers using node runtimes

Description of changes

This PR can be segmented into the following code changes:

A LATEST_NODE_RUNTIME_MAP fact table was added which maintains the latest Lambda node runtime available per AWS partition.
Introduced determineLatestNodeRuntime and determineLatestNodeRuntimeName functions which determine the latest Lambda node Runtime and the latest Lambda node Runtime name, respectively.
The existing runtime property being code generated via the custom resource provider framework has been altered to now use the appropriate runtime determiner function, i.e., determineLatestNodeRuntimeName for CustomResourceProvider or determineLatestNodeRuntime for Function or SingletonFunction. Any custom resource provider using a python runtime will not use either of these functions.
To consolidate and manage importing of external modules a ModuleImporter class has been created. This class allows external modules to be registered as an import for a target module and prevents duplicate imports for modules that contain multiple framework components. This class also provides the ability to specify different import paths which allows all external modules to be consolidated into a single class rather than having duplicate external modules defined for different import paths. Lastly, this class determines whether or not external modules should be imported selectively or if all targets in the external module should be imported under an alias, i.e., import { Function } from 'aws-lambda' vs. import * as lambda from 'aws-lambda'
A CallableExpr class was created to allow expression proxies to be created from a specified expression name. This allows the new runtime determiner functions and other module specific functions to be called from their specified module and built into a JavaScript object that will mirror the JavaScript operations done to it in an expression tree.

Description of how you validated changes

Manually tested all individual custom resource handlers for create, update, and delete. I verified that the behavior seen was what was expected based off of the handler code. All impacted integ tests were updated. New unit tests were added to the custom resource handler framework to test that generated code correctly included determineLatestNodeRuntime for Lambda based handlers and determineLatestNodeRuntimeName for CfnResource based handlers. Added new unit tests to test the functionality of determineLatestNodeRuntime and determineLatestNodeRuntimeName for region agnostic stacks, stacks in commercial region, stacks in China regions, stacks in ADC regions, and stacks in GovCloud regions.

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

colifran · 2024-05-14T06:32:10Z

packages/@aws-cdk/custom-resource-handlers/lib/custom-resources-framework/classes.ts

-          type: scope.coreInternal
-            ? CORE_INTERNAL_CR_PROVIDER.CustomResourceProviderOptions
-            : CORE_MODULE.CustomResourceProviderOptions,


This is one example of what the ModuleImporter class helps with. We can maintain one module, e.g., CORE_MODULE, defined with all of it's types, expressions, etc. rather than needing to define a duplicate module just based on where the module would be imported from, i.e. core vs. aws-lambda. Instead, when registering an import we can just register the same module with a different fromLocation specified. Now the code looks a lot cleaner and we maintain a single module definition.

colifran · 2024-05-14T07:16:33Z

packages/@aws-cdk/custom-resource-handlers/lib/custom-resources-framework/classes.ts

-  /**
-   * External modules that this class depends on.
-   */
-  protected abstract readonly externalModules: ExternalModule[];
-
-  private importExternalModulesInto(scope: HandlerFrameworkModule) {
-    for (const module of this.externalModules) {
-      if (!scope.hasExternalModule(module)) {
-        scope.addExternalModule(module);
-        this.importExternalModuleInto(scope, module);
-      }
-    }
-  }
-
-  private importExternalModuleInto(scope: HandlerFrameworkModule, module: ExternalModule) {
-    switch (module.fqn) {
-      case PATH_MODULE.fqn: {
-        PATH_MODULE.import(scope, 'path');
-        return;
-      }
-      case CONSTRUCTS_MODULE.fqn: {
-        CONSTRUCTS_MODULE.importSelective(scope, ['Construct']);
-        return;
-      }
-      case CORE_MODULE.fqn: {
-        CORE_MODULE.importSelective(scope, [
-          'Stack',
-          'CustomResourceProviderBase',
-          'CustomResourceProviderOptions',
-        ]);
-        return;
-      }
-      case CORE_INTERNAL_CR_PROVIDER.fqn: {
-        CORE_INTERNAL_CR_PROVIDER.importSelective(scope, [
-          'CustomResourceProviderBase',
-          'CustomResourceProviderOptions',
-        ]);
-        return;
-      }
-      case CORE_INTERNAL_STACK.fqn: {
-        CORE_INTERNAL_STACK.importSelective(scope, ['Stack']);
-        return;
-      }
-      case LAMBDA_MODULE.fqn: {
-        LAMBDA_MODULE.import(scope, 'lambda');
-        return;
-      }
-    }


Lots of reduced code here due to ModuleImporter class managing imports

rix0rrr

The overall name of the framework was changed from HandlerFramework to ProviderFramework. This is because the framework is being used to code generate custom resource providers. The custom resource providers execute handler code that this framework has the responsibility of bundling, but calling this a HandlerFramework isn't accurate.

Not in fact true, as far as I'm aware. We call it "custom resource handlers" etc, but I'm pretty sure it also contains Lambda Handlers that don't go into custom resources. Yes, 90% of them are custom resources, but not 100%. So calling this all "custom resources" is a mental shortcut.

rix0rrr · 2024-05-14T08:03:50Z

packages/aws-cdk-lib/region-info/lib/fact.ts

+  /**
+   * The latest Lambda NodeJS runtime available in a given region.
+   */
+  public static readonly LATEST_NODE_RUNTIME = 'latestNodeRuntime';


I also would expect this version to be used for lambda.Runtime.LATEST_NODEJS

packages/aws-cdk-lib/custom-resources/lib/provider-framework/provider.ts

MrArnoldPalmer

Lot's of good improvements to the codegen here that makes it more robust. I'll leave others to approve but I think this handles the move back to a "deploy time" computed runtime value well. 👍🏻

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

scanlonp

Looking good, would like to go over it with you in more depth. Only small thing I noticed is we should add isVariable: true to the runtime created by the determineLatestNodeRuntime() function in runtime.ts.

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

Signed-off-by: Francis <colifran@amazon.com>

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Signed-off-by: Francis <colifran@amazon.com>

aws-cdk-automation · 2024-06-11T18:58:51Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: 50eaec9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2024-06-11T18:59:10Z

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

…esources (aws#30108) This PR introduces node runtime region awareness into the custom resource provider framework by doing the following: - Adding a `LATEST_NODE_RUNTIME_MAP` fact table used to maintain the latest node runtime available per AWS partition - Introducing `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` functions which determine the latest Lambda node `Runtime` and the latest Lambda node `Runtime` name, respectively - Updating the custom resource provider framework to utilize these two functions when code generating the runtime property for providers using node runtimes This PR can be segmented into the following code changes: - A `LATEST_NODE_RUNTIME_MAP` fact table was added which maintains the latest Lambda node runtime available per AWS partition. - Introduced `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` functions which determine the latest Lambda node `Runtime` and the latest Lambda node `Runtime` name, respectively. - The existing runtime property being code generated via the custom resource provider framework has been altered to now use the appropriate runtime determiner function, i.e., `determineLatestNodeRuntimeName` for `CustomResourceProvider` or `determineLatestNodeRuntime` for `Function` or `SingletonFunction`. Any custom resource provider using a python runtime will not use either of these functions. - To consolidate and manage importing of external modules a `ModuleImporter` class has been created. This class allows external modules to be registered as an import for a target module and prevents duplicate imports for modules that contain multiple framework components. This class also provides the ability to specify different import paths which allows all external modules to be consolidated into a single class rather than having duplicate external modules defined for different import paths. Lastly, this class determines whether or not external modules should be imported selectively or if all targets in the external module should be imported under an alias, i.e., `import { Function } from 'aws-lambda'` vs. `import * as lambda from 'aws-lambda'` - A `CallableExpr` class was created to allow expression proxies to be created from a specified expression name. This allows the new runtime determiner functions and other module specific functions to be called from their specified module and built into a JavaScript object that will mirror the JavaScript operations done to it in an expression tree. Manually tested all individual custom resource handlers for create, update, and delete. I verified that the behavior seen was what was expected based off of the handler code. All impacted integ tests were updated. New unit tests were added to the custom resource handler framework to test that generated code correctly included `determineLatestNodeRuntime` for Lambda based handlers and `determineLatestNodeRuntimeName` for `CfnResource` based handlers. Added new unit tests to test the functionality of `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` for region agnostic stacks, stacks in commercial region, stacks in China regions, stacks in ADC regions, and stacks in GovCloud regions. - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

…esources (aws#30108) ### Reason for this change This PR introduces node runtime region awareness into the custom resource provider framework by doing the following: - Adding a `LATEST_NODE_RUNTIME_MAP` fact table used to maintain the latest node runtime available per AWS partition - Introducing `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` functions which determine the latest Lambda node `Runtime` and the latest Lambda node `Runtime` name, respectively - Updating the custom resource provider framework to utilize these two functions when code generating the runtime property for providers using node runtimes ### Description of changes This PR can be segmented into the following code changes: - A `LATEST_NODE_RUNTIME_MAP` fact table was added which maintains the latest Lambda node runtime available per AWS partition. - Introduced `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` functions which determine the latest Lambda node `Runtime` and the latest Lambda node `Runtime` name, respectively. - The existing runtime property being code generated via the custom resource provider framework has been altered to now use the appropriate runtime determiner function, i.e., `determineLatestNodeRuntimeName` for `CustomResourceProvider` or `determineLatestNodeRuntime` for `Function` or `SingletonFunction`. Any custom resource provider using a python runtime will not use either of these functions. - To consolidate and manage importing of external modules a `ModuleImporter` class has been created. This class allows external modules to be registered as an import for a target module and prevents duplicate imports for modules that contain multiple framework components. This class also provides the ability to specify different import paths which allows all external modules to be consolidated into a single class rather than having duplicate external modules defined for different import paths. Lastly, this class determines whether or not external modules should be imported selectively or if all targets in the external module should be imported under an alias, i.e., `import { Function } from 'aws-lambda'` vs. `import * as lambda from 'aws-lambda'` - A `CallableExpr` class was created to allow expression proxies to be created from a specified expression name. This allows the new runtime determiner functions and other module specific functions to be called from their specified module and built into a JavaScript object that will mirror the JavaScript operations done to it in an expression tree. ### Description of how you validated changes Manually tested all individual custom resource handlers for create, update, and delete. I verified that the behavior seen was what was expected based off of the handler code. All impacted integ tests were updated. New unit tests were added to the custom resource handler framework to test that generated code correctly included `determineLatestNodeRuntime` for Lambda based handlers and `determineLatestNodeRuntimeName` for `CfnResource` based handlers. Added new unit tests to test the functionality of `determineLatestNodeRuntime` and `determineLatestNodeRuntimeName` for region agnostic stacks, stacks in commercial region, stacks in China regions, stacks in ADC regions, and stacks in GovCloud regions. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

aws-cdk-automation · 2024-07-25T21:10:27Z

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.

aws-cdk-automation requested a review from a team May 8, 2024 17:34

github-actions bot added the p2 label May 8, 2024

mergify bot added the contribution/core This is a PR that came from AWS. label May 8, 2024

colifran self-assigned this May 8, 2024

colifran changed the title ~~feat: implement region awareness for node runtimes into custom resource framework~~ feat: implement region awareness for node runtimes into custom resource provider framework May 8, 2024

aws-cdk-automation previously requested changes May 8, 2024

View reviewed changes

TheRealAmazonKendra assigned bergjaak May 13, 2024

colifran commented May 14, 2024

View reviewed changes

rix0rrr reviewed May 14, 2024

View reviewed changes

colifran mentioned this pull request May 17, 2024

aws-cdk-lib: NODEJS_LATEST still points to nodejs18.x #28125

Open

MrArnoldPalmer reviewed May 17, 2024

View reviewed changes

bergjaak removed their assignment May 21, 2024

colifran changed the title ~~feat: implement region awareness for node runtimes into custom resource provider framework~~ feat: implement region awareness for node runtimes for cdk vended custom resources May 23, 2024

colifran added the pr/do-not-merge This PR should not be merged at this time. label May 28, 2024

colifran marked this pull request as ready for review May 28, 2024 15:08

colifran changed the title ~~feat: implement region awareness for node runtimes for cdk vended custom resources~~ feat: implement node runtime region awareness for cdk vended custom resources May 28, 2024

aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label May 28, 2024

scanlonp reviewed May 30, 2024

View reviewed changes

github-actions bot mentioned this pull request Jun 1, 2024

Monthly PR metrics report #30414

Closed

TheRealAmazonKendra force-pushed the colifran/region-aware-framework branch 2 times, most recently from 8305b3a to 73379cf Compare June 6, 2024 02:06

aws-cdk-automation previously requested changes Jun 6, 2024

View reviewed changes

ruining colin's life

df9727c

TheRealAmazonKendra force-pushed the colifran/region-aware-framework branch from 73379cf to df9727c Compare June 6, 2024 02:27

colifran added 2 commits June 5, 2024 19:30

migrate amplify handlers to framework and update snaps

e7bd0e2

Signed-off-by: Francis <colifran@amazon.com>

alpha modules

b8c4da9

Signed-off-by: Francis <colifran@amazon.com>

TheRealAmazonKendra mentioned this pull request Jun 6, 2024

fix(custom-resource-handlers): better fallback for require failures #30469

Closed

1 task

colifran added 4 commits June 10, 2024 19:24

snaps

d8f2816

Signed-off-by: Francis <colifran@amazon.com>

snaps

3173902

Signed-off-by: Francis <colifran@amazon.com>

snaps

9386ae3

Signed-off-by: Francis <colifran@amazon.com>

snaps

fcdce0a

Signed-off-by: Francis <colifran@amazon.com>

aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jun 11, 2024

Merge branch 'main' into colifran/region-aware-framework

50eaec9

TheRealAmazonKendra approved these changes Jun 11, 2024

View reviewed changes

mergify bot merged commit 69c5dde into main Jun 11, 2024
13 checks passed

mergify bot deleted the colifran/region-aware-framework branch June 11, 2024 18:59

tmokmss mentioned this pull request Jun 21, 2024

feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

Merged

1 task

dreamorosi mentioned this pull request Jun 27, 2024

Maintenance: downgrade AWS CDK to 2.145.0 aws-powertools/powertools-lambda-typescript#2699

Closed

2 tasks

heitorlessa mentioned this pull request Jun 27, 2024

chore(layers): downgrade aws cdk to 2.145.0 aws-powertools/powertools-lambda-typescript#2700

Merged

This was referenced Jun 27, 2024

Maintenance: downgrade AWS CDK to 2.145.0 aws-powertools/powertools-lambda-python#4639

Closed

chore(layers): downgrade aws cdk to 2.145.0 aws-powertools/powertools-lambda-python#4640

Merged

dreamorosi mentioned this pull request Jun 27, 2024

cdk: missing ca-west-1 region in aws-entities causes cdk vended custom resources to fail deployment #30694

Closed

NOUIY mentioned this pull request Jul 5, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#107

Open

mazyu36 mentioned this pull request Jul 6, 2024

feat(stepfunctions-tasks): enable resultReuseConfiguration in AthenaStartQueryExecution #30447

Merged

1 task

This was referenced Jul 11, 2024

Maintenance: downgrade AWS CDK to 2.145.0 aws-powertools/powertools-lambda-python#4738

Closed

chore(layers): downgrade aws cdk to 2.145.0 aws-powertools/powertools-lambda-python#4739

Merged

This was referenced Jul 12, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#108

Open

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#109

Open

scanlonp mentioned this pull request Jul 18, 2024

Colifran/region aware framework branch test #30474

Closed

1 task

This was referenced Jul 19, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#110

Open

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#111

Open

aws locked as resolved and limited conversation to collaborators Jul 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement node runtime region awareness for cdk vended custom resources #30108

feat: implement node runtime region awareness for cdk vended custom resources #30108

colifran commented May 8, 2024 •

edited

Loading

aws-cdk-automation left a comment

colifran May 14, 2024 •

edited

Loading

colifran May 14, 2024

rix0rrr left a comment

rix0rrr May 14, 2024

MrArnoldPalmer left a comment

scanlonp left a comment

aws-cdk-automation left a comment

aws-cdk-automation commented Jun 11, 2024

mergify bot commented Jun 11, 2024

aws-cdk-automation commented Jul 25, 2024

feat: implement node runtime region awareness for cdk vended custom resources #30108

feat: implement node runtime region awareness for cdk vended custom resources #30108

Conversation

colifran commented May 8, 2024 • edited Loading

Reason for this change

Description of changes

Description of how you validated changes

Checklist

aws-cdk-automation left a comment

Choose a reason for hiding this comment

colifran May 14, 2024 • edited Loading

Choose a reason for hiding this comment

colifran May 14, 2024

Choose a reason for hiding this comment

rix0rrr left a comment

Choose a reason for hiding this comment

rix0rrr May 14, 2024

Choose a reason for hiding this comment

MrArnoldPalmer left a comment

Choose a reason for hiding this comment

scanlonp left a comment

Choose a reason for hiding this comment

aws-cdk-automation left a comment

Choose a reason for hiding this comment

aws-cdk-automation commented Jun 11, 2024

AWS CodeBuild CI Report

mergify bot commented Jun 11, 2024

aws-cdk-automation commented Jul 25, 2024

colifran commented May 8, 2024 •

edited

Loading

colifran May 14, 2024 •

edited

Loading