Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

custom-resources: multiple AwsCustomResource constructs ignores role property when set in one of them #32533

Open
1 task
tmokmss opened this issue Dec 16, 2024 · 1 comment
Labels
@aws-cdk/custom-resources Related to AWS CDK Custom Resources bug This issue is a bug. effort/small Small work item – less than a day of effort p2

Comments

@tmokmss
Copy link
Contributor

tmokmss commented Dec 16, 2024

Describe the bug

When we create multiple AwsCustomResource resources and provided role to only one of them, the role is ignored and we get a permission error on deployment. (see the code to reproduce)

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

Deployment successes.

Current Behavior

Deployment fails with a permission error like below:

Received response status [FAILED] from custom resource. Message returned: User: arn:aws:sts::REDACTED:assumed-role/CrDupRoleReproStack-AWS679f53fac002430cb0da5b7982bd-219ls3LkG2Ki/CrDupRoleReproStack-AWS679f53fac002430cb0da5b7982b-CnUgdL71XPPf is not authorized to perform: sqs:sendmessage on resource: arn:aws:sqs:ap-northeast-1:REDACTED:CrDupRoleReproStack-Queue4A7E3555-ubUVQOF1lini because no identity-based policy allows the sqs:sendmessage action

Reproduction Steps

Deploy the below stack:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as cr from 'aws-cdk-lib/custom-resources';
import { PolicyStatement, Role } from 'aws-cdk-lib/aws-iam';
import { Queue } from 'aws-cdk-lib/aws-sqs';

export class CrDupRoleReproStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const queue = new Queue(this, 'Queue');

    const role = new Role(this, 'Role', {
      assumedBy: new cdk.aws_iam.ServicePrincipal('lambda.amazonaws.com'),
    });

    new cr.AwsCustomResource(this, 'API1', {
      onCreate: {
        service: 's3',
        action: 'listBuckets',
        parameters: {
          MaxBuckets: 1,
        },
        physicalResourceId: cr.PhysicalResourceId.of('...'),
      },
      policy: cr.AwsCustomResourcePolicy.fromStatements([
        new PolicyStatement({
          actions: ['s3:ListAllMyBuckets'],
          resources: ['*'],
        }),
      ]),
      // If we set role to all of the custom resources, it successfully deploys.
      // role,
    });

    new cr.AwsCustomResource(this, 'API2', {
      onCreate: {
        service: 'sqs',
        action: 'sendMessage',
        parameters: {
          QueueUrl: queue.queueUrl,
          MessageBody: 'hello',
        },
        physicalResourceId: cr.PhysicalResourceId.of('...'),
      },
      role,
    });
    queue.grantSendMessages(role);
  }
}

Possible Solution

The root cause of this is we use a shared SingletonFunction for all the AwsCustomResource, and we can set only one IAM role for a Lambda function. So the possible solution can be:

  1. create multiple SingletonFunction based on which role to use
  2. Just throw an error when it got a role property which becomes invalid

Additional Information/Context

It would be great if we could write something like this to avoid creating a new role:

    const api2 = new cr.AwsCustomResource(this, 'API2', {
      onCreate: {
        service: 'sqs',
        action: 'sendMessage',
        parameters: {
          QueueUrl: queue.queueUrl,
          MessageBody: 'hello',
        },
        physicalResourceId: cr.PhysicalResourceId.of('...'),
      },
    });
    queue.grantSendMessages(api2);

However, we cannot do this because we have to set at least one of policy or role property.

CDK CLI Version

2.173.1

Framework Version

2.173.1

Node.js Version

22

OS

macOS

Language

TypeScript

Language Version

No response

Other information

No response

@tmokmss tmokmss added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Dec 16, 2024
@github-actions github-actions bot added the @aws-cdk/custom-resources Related to AWS CDK Custom Resources label Dec 16, 2024
@ashishdhingra ashishdhingra self-assigned this Dec 19, 2024
@ashishdhingra ashishdhingra added p2 effort/small Small work item – less than a day of effort investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels Dec 19, 2024
@ashishdhingra
Copy link
Contributor

Synthesizing the provided code generated below CloudFormation template:

Resources:
  Queue4A7E3555:
    Type: AWS::SQS::Queue
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: CdktestStackNew/Queue/Resource
  Role1ABCC5F0:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
        Version: "2012-10-17"
    Metadata:
      aws:cdk:path: CdktestStackNew/Role/Resource
  RoleDefaultPolicy5FFB7DAB:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
          - Action:
              - sqs:GetQueueAttributes
              - sqs:GetQueueUrl
              - sqs:SendMessage
            Effect: Allow
            Resource:
              Fn::GetAtt:
                - Queue4A7E3555
                - Arn
        Version: "2012-10-17"
      PolicyName: RoleDefaultPolicy5FFB7DAB
      Roles:
        - Ref: Role1ABCC5F0
    Metadata:
      aws:cdk:path: CdktestStackNew/Role/DefaultPolicy/Resource
  API19FA083CD:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS679f53fac002430cb0da5b7982bd22872D164C4C
          - Arn
      Create: '{"service":"s3","action":"listBuckets","parameters":{"MaxBuckets":1},"physicalResourceId":{"id":"..."}}'
      InstallLatestAwsSdk: false
    DependsOn:
      - API1CustomResourcePolicyF301AF28
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: CdktestStackNew/API1/Resource/Default
  API1CustomResourcePolicyF301AF28:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
          - Action: s3:ListAllMyBuckets
            Effect: Allow
            Resource: "*"
        Version: "2012-10-17"
      PolicyName: API1CustomResourcePolicyF301AF28
      Roles:
        - Ref: AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2
    Metadata:
      aws:cdk:path: CdktestStackNew/API1/CustomResourcePolicy/Resource
  AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
        Version: "2012-10-17"
      ManagedPolicyArns:
        - Fn::Join:
            - ""
            - - "arn:"
              - Ref: AWS::Partition
              - :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
    Metadata:
      aws:cdk:path: CdktestStackNew/AWS679f53fac002430cb0da5b7982bd2287/ServiceRole/Resource
  AWS679f53fac002430cb0da5b7982bd22872D164C4C:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: cdk-hnb659fds-assets-<<ACCOUNT-ID>>-us-east-2
        S3Key: ce2f3595a340d6c519a65888ef97e3b9b64f053f83608e32cc28162e22d7d99a.zip
      Handler: index.handler
      Role:
        Fn::GetAtt:
          - AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2
          - Arn
      Runtime: nodejs20.x
      Timeout: 120
    DependsOn:
      - AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2
    Metadata:
      aws:cdk:path: CdktestStackNew/AWS679f53fac002430cb0da5b7982bd2287/Resource
      aws:asset:path: asset.ce2f3595a340d6c519a65888ef97e3b9b64f053f83608e32cc28162e22d7d99a
      aws:asset:is-bundled: false
      aws:asset:property: Code
  API2C3CE4D25:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS679f53fac002430cb0da5b7982bd22872D164C4C
          - Arn
      Create:
        Fn::Join:
          - ""
          - - '{"service":"sqs","action":"sendMessage","parameters":{"QueueUrl":"'
            - Ref: Queue4A7E3555
            - '","MessageBody":"hello"},"physicalResourceId":{"id":"..."}}'
      InstallLatestAwsSdk: false
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: CdktestStackNew/API2/Resource/Default
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/12MQW6DMBBFz5L9MG0gUrNNkLpu4QDIMQNyMGOVsYMiy3eP4kQsuvrvPY2mxP1XheVOrVLofiqsuWBsvdITqFW6KH+C8TdQIKgHzpDAqBlj42xueX+cNfr+1BclsGq+9Apja3i05B1/B9beOIYN6mGLCaTqlAh5wdNzQCo8Bz2RPyuhBA2JC4sm0EG8m7vl7YKnVerctpP/OvDG+Xfr1Wh4TMCuJ7zKx21/xPITD7urGFMsgb2ZCZvXPgA+pVKUIgEAAA==
    Metadata:
      aws:cdk:path: CdktestStackNew/CDKMetadata/Default
Parameters:
  BootstrapVersion:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]

Clearly custom resource AWS::Lambda::Function is missing the role.

If we comment 1st custom resource API1 or uncomment role assignation, it generates the below CloudFormation resource definition for the custom resource AWS::Lambda::Function resource (with role set):

AWS679f53fac002430cb0da5b7982bd22872D164C4C:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: cdk-hnb659fds-assets-<<ACCOUNT-ID>>-us-east-2
        S3Key: ce2f3595a340d6c519a65888ef97e3b9b64f053f83608e32cc28162e22d7d99a.zip
      Handler: index.handler
      Role:
        Fn::GetAtt:
          - Role1ABCC5F0
          - Arn
      Runtime: nodejs20.x
      Timeout: 120
    DependsOn:
      - RoleDefaultPolicy5FFB7DAB
      - Role1ABCC5F0
    Metadata:
      aws:cdk:path: CdktestStackNew/AWS679f53fac002430cb0da5b7982bd2287/Resource
      aws:asset:path: asset.ce2f3595a340d6c519a65888ef97e3b9b64f053f83608e32cc28162e22d7d99a
      aws:asset:is-bundled: false
      aws:asset:property: Code

@ashishdhingra ashishdhingra removed the investigating This issue is being investigated and/or work is in progress to resolve the issue. label Dec 19, 2024
@ashishdhingra ashishdhingra removed their assignment Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/custom-resources Related to AWS CDK Custom Resources bug This issue is a bug. effort/small Small work item – less than a day of effort p2
Projects
None yet
Development

No branches or pull requests

2 participants