CDk diff: Inconsistent Lambda Layer Ordering in CDK Diff #31703
Labels
bug
This issue is a bug.
effort/small
Small work item – less than a day of effort
p2
package/tools
Related to AWS CDK Tools or CLI
Comments
eboily
added
bug
ashishdhingra
added
p2
needs-reproduction
|
@eboily Good afternoon. Thanks for opening the issue. Somehow, I'm unable to reproduce the issue. Below is CDK stack used with associated Lambda function and layers. Used CDK version src/lambdas/lambda.ts import * as logic from '/opt/business-logic';
import * as security from '/opt/security-logic'
export const handler = async (event: any = {}): Promise<any> => {
console.log(`Addition:${logic.add(2, 3)}`);
console.log(`SECURITY:${security.makesecure()}`);
};src/layers/business-logic/logic.ts export function add(a: number, b: number) {
return a + b;
}src/layers/security-logic/logic.ts export function makesecure() {
return 'SECURED';
}lib/lambda-with-layers-stack.ts import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, NodejsFunctionProps } from 'aws-cdk-lib/aws-lambda-nodejs';
import path = require('path');
export class LambdaWithLayersStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const logicLayer = new lambda.LayerVersion(this, 'logic-layer', {
compatibleRuntimes: [
lambda.Runtime.NODEJS_18_X,
],
code: lambda.Code.fromAsset('src/layers/business-logic'),
description: 'Business Logic layer',
});
const securityLayer = new lambda.LayerVersion(this, 'security-layer', {
compatibleRuntimes: [
lambda.Runtime.NODEJS_18_X,
],
code: lambda.Code.fromAsset('src/layers/security-logic'),
description: 'Security Logic layer',
});
const nodeJsFnProps: NodejsFunctionProps = {
bundling: {
externalModules: [
'aws-sdk', // Use the 'aws-sdk' available in the Lambda runtime
],
},
runtime: lambda.Runtime.NODEJS_18_X,
timeout: cdk.Duration.minutes(3),
memorySize: 256,
};
const lambdaWithLayer = new NodejsFunction(this, 'lambdaWithLayer', {
entry: path.join(__dirname, '../src/lambdas', 'lambda.ts'),
...nodeJsFnProps,
functionName: 'lambdaWithLayer',
handler: 'handler',
layers: [ logicLayer, securityLayer ],
});
}
}tsconfig.json {
"compilerOptions": {
"target": "ES2020",
"module": "commonjs",
"lib": [
"es2020",
"dom"
],
"declaration": true,
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noImplicitThis": true,
"alwaysStrict": true,
"noUnusedLocals": false,
"noUnusedParameters": false,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": false,
"inlineSourceMap": true,
"inlineSources": true,
"experimentalDecorators": true,
"strictPropertyInitialization": false,
"typeRoots": [
"./node_modules/@types"
],
"paths": {
"/opt/business-logic": ["./src/layers/business-logic/logic"],
"/opt/security-logic": ["./src/layers/security-logic/logic"]
}
},
"exclude": [
"node_modules",
"cdk.out"
]
}lib/lambda-with-layers-stack.ts import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, NodejsFunctionProps } from 'aws-cdk-lib/aws-lambda-nodejs';
import path = require('path');
export class LambdaWithLayersStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const logicLayer = new lambda.LayerVersion(this, 'logic-layer', {
compatibleRuntimes: [
lambda.Runtime.NODEJS_18_X,
],
code: lambda.Code.fromAsset('src/layers/business-logic'),
description: 'Business Logic layer',
});
const securityLayer = new lambda.LayerVersion(this, 'security-layer', {
compatibleRuntimes: [
lambda.Runtime.NODEJS_18_X,
],
code: lambda.Code.fromAsset('src/layers/security-logic'),
description: 'Security Logic layer',
});
const nodeJsFnProps: NodejsFunctionProps = {
bundling: {
externalModules: [
'aws-sdk', // Use the 'aws-sdk' available in the Lambda runtime
],
},
runtime: lambda.Runtime.NODEJS_18_X,
timeout: cdk.Duration.minutes(3),
memorySize: 256,
};
const lambdaWithLayer = new NodejsFunction(this, 'lambdaWithLayer', {
entry: path.join(__dirname, '../src/lambdas', 'lambda.ts'),
...nodeJsFnProps,
functionName: 'lambdaWithLayer',
handler: 'handler',
layers: [ logicLayer, securityLayer ],
});
}
}bin/cdktest.ts #!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { LambdaWithLayersStack } from '../lib/lambda-with-layers-stack';
const app = new cdk.App();
new LambdaWithLayersStack(app, 'LambdaWithLayersStack');
Unsure if issue at your end is related to Thanks, |
ashishdhingra
added
response-requested
|
Hi Ashish, Thank you for looking into the issue. I understand that it might be difficult to reproduce the problem consistently, as the reordering of Lambda layers seems to occur intermittently. Here’s some more context based on our observations: The problem is that the order of Lambda layers in the CloudFormation template for certain functions sometimes changes unexpectedly during deployment. The issue does not consistently affect the same Lambda function and may vary between different functions in each deployment. For example, during some deployments, one Lambda function might see its layers reordered, while in others, a different Lambda function shows the same behavior. for example, here are two consecutive then, when I re-issue the The layer ordering issue affects other lambdas. Thanks for your support!
|
|
@eboily Thanks for your response. Could you please check if you have feature flag
This feature flag affects the logic here to sort layers. Thanks, |
ashishdhingra
added
response-requested
|
Hi Ashish,
Thanks,
|
@eboily Since the feature flag is set to true in your environment, it should have mitigated the issue. I'm unsure if it's feasible to reproduce the issue. Do you see any differences in your CDK project compared to if you create a new CDK project all together? Or, is it possible to share self-contained minimal CDK project alog with instructions, to try reproduce the issue? Thanks, |
ashishdhingra
added
response-requested
|
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
github-actions
bot
added
the
closing-soon
|
Hi Ashish, sorry for the answer’s delay, I’m fighting COVID. |
github-actions
bot
removed
closing-soon
|
Comments on closed issues and PRs are hard for our team to see. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Describe the bug
When deploying an AWS Lambda function with multiple layers using AWS CDK (v2), the order of layers in the CloudFormation template (AWS::Lambda::Function resource) changes between deployments. This causes the
cdk diffcommand to show unnecessary changes, even though the functionality and deployment outcome remain the same.Regression Issue
Last Known Working CDK Version
No response
Expected Behavior
The order of layers in the AWS::Lambda::Function resource should remain consistent across deployments unless explicitly modified by the developer. This would prevent unnecessary changes from being reported in
cdk diff.Current Behavior
The order of layers in the AWS::Lambda::Function resource changes arbitrarily between deployments. This results in diffs like the following:
Despite no functional changes, the difference in layer order causes unnecessary diffs.
Reproduction Steps
Possible Solution
Ensure that the order of Lambda layers is preserved and consistent between deployments unless the developer explicitly changes the order in the CDK code. The
cdk diffcommand should only report changes when actual modifications (such as version changes or new layers) occur, and not simply due to arbitrary reordering of layers.Additional Information/Context
Environment:
Workaround:
Manually enforce layer order in the CDK code by specifying the order explicitly in the array when defining the Lambda function’s layers. However, this is cumbersome and not ideal for long-term maintenance.
Impact:
This issue leads to confusion and unnecessary diffs during deployment, which may slow down CI/CD pipelines and cause developers to spend time analyzing non-functional changes.
CDK CLI Version
2.161.1 (build 0a606c9)
Framework Version
No response
Node.js Version
v20.6.1
OS
macOS 15.0
Language
TypeScript
Language Version
Actually javascript, not typescript
Other information
No response
The text was updated successfully, but these errors were encountered: