Pulumi cannot find created SES ConfigurationSetEventDestination

[Jimmy89]

What happened?

I create a SES ConfigurationSetEventDestination. After creation I get a timeout of 'getResource' because the so-called ConfigurationSetName would be null (which it is not). If I go to the AWS console I see that the resource was created and functions. I don't know whether this is Pulumi's fault or that from AWS Cloud Control API.

Example

The following program was used:

import * as pulumi from "@pulumi/pulumi";
import * as awsClassic from "@pulumi/aws";
import * as aws from "@pulumi/aws-native";

const { name: regionName } = await awsClassic.getRegion();
const { accountId } = await aws.getAccountId();

const broadKeyPolicy = awsClassic.iam.getPolicyDocumentOutput({
    statements: [
        {
            sid: "Enable all User Permissions for root account",
            actions: ["kms:*"],
            resources: ["*"],
            effect: "Allow",
            principals: [{
                type: "AWS",
                identifiers: [`arn:aws:iam::${accountId}:root`],
            }]
        },
        {
            actions: [
                "kms:*"
            ],
            resources: ["*"],
            effect: "Allow",
            principals: [{
                type: "Service",
                identifiers: ["sns.amazonaws.com"],
            }],
            conditions: [{
                test: "StringEquals",
                variable: "AWS:SourceOwner",
                values: [accountId]
            }],
        },
        {
            actions: [
                "kms:*"
            ],
            resources: ["*"],
            effect: "Allow",
            principals: [{
                type: "Service",
                identifiers: ["ses.amazonaws.com"],
            }]
        }
    ],
}).json;

const key = new aws.kms.Key("kms", {
    name: "kms-log",
    enabled: true,
    keySpec: "SYMMETRIC_DEFAULT",
    multiRegion: true,
    pendingWindowInDays: 7,
    keyPolicy: broadKeyPolicy,
}) 

const keyAlias = new aws.kms.Alias("kms-alias", {
    aliasName: `alias/testing/1`,
    targetKeyId: key.arn
}, { parent: key });

const snsDefaultRole = new awsClassic.iam.Role(`sns-write-logs`, {
  description: "Allow sns service to write logs",
  assumeRolePolicy: awsClassic.iam.getPolicyDocumentOutput({
    statements: [{
      actions: ["sts:AssumeRole"],
      principals: [{
        identifiers: ["sns.amazonaws.com"],
        type: "Service",
      }],
      effect: "Allow",
    }],
  }).json,
  managedPolicyArns: [
    awsClassic.iam.ManagedPolicy.AmazonSNSRole,
  ]
});

const sesEmailNotifications = new awsClassic.sns.Topic("ses-email-notifications", {
  kmsMasterKeyId: keyAlias.aliasName,
  fifoTopic: false,
  deliveryPolicy: JSON.stringify({
    "http": {
      "defaultHealthyRetryPolicy": {
        "minDelayTarget": 20,
        "maxDelayTarget": 20,
        "numRetries": 1,
        "numMaxDelayRetries": 0,
        "numNoDelayRetries": 0,
        "numMinDelayRetries": 0,
        "backoffFunction": "linear"
      },
      "disableSubscriptionOverrides": false,
      "defaultRequestPolicy": {
        "headerContentType": "text/plain; charset=UTF-8"
      }
    }
  }),
  policy: awsClassic.iam.getPolicyDocumentOutput({
    version: "2008-10-17",
    statements: [{
      actions: ["SNS:Publish",
        "SNS:RemovePermission",
        "SNS:SetTopicAttributes",
        "SNS:DeleteTopic",
        "SNS:ListSubscriptionsByTopic",
        "SNS:GetTopicAttributes",
        "SNS:AddPermission",
        "SNS:Subscribe"],
      principals: [{ type: "AWS", identifiers: ["*"] }],
      resources: [pulumi.concat( "arn:aws:sns:", regionName, ":", accountId, ":", "sesEmailNotifications*")],
      effect: "Allow",
      conditions: [{
        test: "StringEquals",
        variable: "AWS:SourceOwner",
        values: [accountId]
      }]
    }],
  }).json,
  namePrefix: "sesEmailNotifications",
  displayName: "ses-email-notification",
  sqsSuccessFeedbackRoleArn: snsDefaultRole.arn, // Default role to create logs
  sqsFailureFeedbackRoleArn: snsDefaultRole.arn,
  sqsSuccessFeedbackSampleRate: 100,
  lambdaFailureFeedbackRoleArn: snsDefaultRole.arn,
  lambdaSuccessFeedbackRoleArn: snsDefaultRole.arn,
  lambdaSuccessFeedbackSampleRate: 100,
  tracingConfig: "PassThrough",
});

const confName = "email-noTracking";
new aws.ses.ConfigurationSet("email-conf-set", {
    name: confName,
    deliveryOptions: {
        tlsPolicy: "OPTIONAL",
    },
    reputationOptions: {
        reputationMetricsEnabled: false,
    },
    sendingOptions: {
        sendingEnabled: true,
    },
    vdmOptions: {
        dashboardOptions: {
            engagementMetrics: "DISABLED",
        },
        guardianOptions: {
            optimizedSharedDelivery: "DISABLED",
        }
    }
});

new aws.ses.ConfigurationSetEventDestination("email-event-destination", {
    configurationSetName: confName,
    eventDestination: {
        enabled: true,
        name: "allSendingAndDeliveryTypes-SNS",
        matchingEventTypes: [
            "send", "reject", "bounce", "complaint", "delivery", "renderingFailure",
            "deliveryDelay", "subscription",
        ],
        snsDestination: {
            topicArn: sesEmailNotifications.arn,
        }
    }
});

Excuse me if there are too many sidesteps, I just want to make sure everything is in place to mimic the exact same circumstances.

1. Pulumi up the program.
2. Note that eventually you get an error. (see below)
3. Check in the AWS console that the resource is actually created.

  aws-native:ses:ConfigurationSetEventDestination (email-event-destination):
    error: reading resource state: reading resource state: operation error CloudControl: GetResource, exceeded maximum number of attempts, 25, https response error StatusCode: 500, RequestID: XXXX, HandlerInternalFailureException: AWS::SES::ConfigurationSetEventDestination Handler returned status FAILED: Unable to marshall request to JSON: Parameter 'ConfigurationSetName' must not be null (HandlerErrorCode: InternalFailure, RequestToken: XXXXX)

Output of pulumi about

NAME                VERSION
@pulumi/aws         6.5.0
@pulumi/aws-native  0.81.0
@pulumi/pulumi      3.89.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[mikhailshilkov]

I'll forward this question to AWS Cloud Control, thank you for reporting it.

[Jimmy89]

@mikhailshilkov any chance you heard back from AWS?

[mikhailshilkov]

Unfortunately not yet, I pinged them once again.

[marcindruzgala]

@mikhailshilkov any updates? This one is still failing event though ConfigurationSetEventDestination is properly created in AWS. The issue is that Pulumi doesn't see it that way and when we try to run pulumi up again it throws an error that ConfigurationSetEventDestination with provided nam already exists

[marcindruzgala]

I should probably mention that my issue is with .NET Pulumi (Native 0.96.0), I switched to Pulumi Classic (6.21.0) resources and it worked