Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SESv2 Plan #26796

Closed
kamilturek opened this issue Sep 13, 2022 · 34 comments
Closed

SESv2 Plan #26796

kamilturek opened this issue Sep 13, 2022 · 34 comments
Labels
enhancement Requests to existing resources that expand the functionality or scope. new-service Introduces a new service. service/sesv2 Issues and PRs that pertain to the sesv2 service.

Comments

@kamilturek
Copy link
Collaborator

kamilturek commented Sep 13, 2022

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

SESv2

This issue contains a plan for SESv2 service implementation. It includes a list of planned resources along with proposed sample schemas. Any feedback is very appreciated.

Not all SESv2 API actions are covered. Attention was primarily paid to main SESv2 resources and the most up-voted SES(v2)-related issues. Although, the remaining SESv2 resources might be also implemented in the future.

This issue also aims to track the progress of SESv2 service implementation.

Table of Contents

Resource List

This section lists all service resources with proposed schemas and relevant AWS
API actions. Corresponding data sources (singular and plural) are skipped in the
list but also should be implemented.

Click here to expand

aws_sesv2_configuration_set

Schema

resource "aws_sesv2_configuration_set" "example" {
    configuration_set_name = "example"

    delivery_options {
        sending_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name
        tls_policy        = "REQUIRE"
    }

    reputation_options {
        last_fresh_start           = "timestamp"
        reputation_metrics_enabled = true
    }

    sending_options {
        sending_enabled = true
    }

    suppression_options {
        suppressed_reasons = ["BOUNCE", "COMPLIANT"]
    }

    tracking_options {
        custom_redirect_domain = "example.com"
    }
}

API Actions

aws_sesv2_configuration_set_event_destination

Schema

resource "aws_sesv2_configuration_set_event_destination" "example" {
    configuration_set_name = aws_sesv2_configuration_set.example.name
    event_destination_name = "example"

    event_destination {
        enabled = true
        
        matching_event_types = ["BOUNCE", "COMPLIANT"]
        
        cloud_watch_service_destination {
             dimension_configuration {
                 default_dimension_value = "example"
                 dimension_name          = "example"
                 dimension_value_source  = "MESSAGE_TAG"
             }
        }
    
        kinesis_firehose_destination {
            delivery_stream_arn = "arn"
            iam_role_arn        = "arn"
        }
        
        pinpoint_destination {
            application_arn = "arn"
        }

        sns_destination {
            topic_arn = "arn"
        }
    }
}

API Actions

aws_sesv2_dedicated_ip_pool

Schema

resource "aws_sesv2_dedicated_ip_pool" "example" {
    pool_name = "example"
}

API Actions

aws_sesv2_dedicated_ip_assignment

Schema

resource "aws_sesv2_dedicated_ip_assignment" "example" {
    ip = "ip"
    destination_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name
}

API Actions

aws_sesv2_contact_list

Schema

resource "aws_sesv2_contact_list" "example" {
    contact_list_name = "example"
    description       = "description"

    topic {
        default_subscription_status = "OPT_OUT"
        description                 = "description"
        display_name                = "example"
        topic_name                  = "exmaple"
    }
}

API Actions

aws_sesv2_contact

Schema

resource "aws_sesv2_contact" "example" {
    contact_list_name = aws_sesv2_contact_list.example.contact_list_name
    attributes_data   = "example"
    email_address     = "example@example.com"
    unsubscribe_all   = false

    topic_preference {
        subscription_status = "OPT_IN"
        topic_name		    = "exmaple"
    }
}

API Actions

aws_sesv2_custom_verification_email_template

To use this feature, your Amazon SES account has to be out of the sandbox.

Schema

resource "aws_sesv2_custom_verification_email_template" "example" {
    from_email_address = "example@example.com"

    failure_redirection_url = "url"
    success_redirection_url = "url"

    template_content = "content"
    template_name    = "name"
    template_subject = "subject"
}

API Actions

aws_sesv2_deliverability_test_report

The test is available after 24 hours. It is unclear how the provider plugin should behave in this case.

Schema

TBC

API Actions

aws_sesv2_email_identity

Schema

resource "aws_sesv2_email_identity" "example" {
    configuration_set_name = aws_sesv2_configuration_set.example.configuration_set_name

    dkim_signing_attributes {
        domain_signing_private_key = "a"
        domain_signing_selector	   = "b"

        # next_signing_key_length = "RSA_1024_BIT"
    }

    email_identity = "example@example.com"
}

API Actions

aws_sesv2_email_identity_policy

Schema

resource "aws_sesv2_email_identity_policy" "example" {
    email_identity = aws_sesv2_email_identity.example.email_identity
    policy_name    = "name"

    policy = <<EOT
{
  "Id":"ExampleAuthorizationPolicy",
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AuthorizeAccount",
      "Effect":"Allow",
      "Resource":"arn:aws:ses:us-east-1:123456789012:identity/example.com",
      "Principal":{
        "AWS":[
          "123456789012"
        ]
      },
      "Action":[
        "ses:SendEmail",
        "ses:SendTemplatedEmail",
        "ses:SendRawEmail",
        "ses:SendBulkTemplatedEmail"
      ]
    }
  ]
}
EOT
}

API Actions

aws_sesv2_email_template

Schema

resource "aws_sesv2_email_template" "example" {
    template_name = "example"

    template_content {
        html    = "html"
        text    = "text"
        subject = "subject"
    }
}

API Actions

aws_sesv2_import_job

Schema

resource "aws_sesv2_import_job" "example" {
    import_data_source {
        data_format = "CSV"
        s3_url      = "s3://url"
    }

    import_destination {
        contact_list_destination {
            contact_list_import_action = "PUT"
            contact_list_name = aws_sesv2_contact_list.example.contact_list_name
        }

        suppression_list_destination {
            suppression_list_import_action = "DELETE"
        }
    }
}

API Actions

aws_sesv2_suppressed_destination

Schema

resource "aws_sesv2_suppressed_destination" "example" {
    email_address = "example@example.com"
    reason 	      = "reason"
}

API Actions

aws_sesv2_account

This is only a data source.

Schema

data "aws_sesv2_account" "example" {}

API Actions

Related

#14320
#10703
#17570
#21129

@kamilturek kamilturek added the enhancement Requests to existing resources that expand the functionality or scope. label Sep 13, 2022
@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/sesv2 Issues and PRs that pertain to the sesv2 service. labels Sep 13, 2022
@ewbankkit ewbankkit removed the needs-triage Waiting for first response or review from a maintainer. label Sep 28, 2022
@ewbankkit ewbankkit added new-resource Introduces a new resource. new-service Introduces a new service. and removed new-resource Introduces a new resource. labels Oct 5, 2022
@JoshiiSinfield
Copy link

Can I ask why you're planning new schemas / resource names?

There's not a SESv2 service, or any other difference within the console or resources within AWS. It's purely an SDK / API thing.

No one using Terraform will choose between a v1 or v2 resouces. It's just "ses resources".

Also, shameless plug but I've done the ground work & email identity work in here:

#26604

@kamilturek
Copy link
Collaborator Author

Hey @JoshiiSinfield

Thank you for your comment. Since the SESv1 and SESv2 are substantially different, the goal is to follow the approach of other services like waf & waf2, apigateway & apigatewayv2, kinesisanalytics &. kinesisanalytics2 and so on.

Also, nice to see your work on the identities 🙌 I think it can become part of sesv2.

@JoshiiSinfield
Copy link

Thanks @kamilturek .

It's not actually that much different?

WAFv2 was a whole different set of actual resources. As was apigwv2- a http apigw.

Slightly different for ses.

As can see from my PR, it's very similar api functions.

Not looking forward to having to swap out all ses resources tho tbh. Gonna be a lot of moved resource blocks.

Doesn't need to be separate IMO. Apologies, realise I'm not a maintainer or anything so my argument probably doesn't stand. But still wanted to air it!

Cheers - would appreciate my work not going redundant. Also shows new resources aren't needed 😂

Cheers

@jar-b
Copy link
Member

jar-b commented Oct 18, 2022

How would you feel about converting aws_sesv2_dedicated_ip to a data source and renaming the proposed resource to indicate assignment to a pool (aws_sesv2_dedicated_ip_assignment)? The justification being the IP itself must be created via an AWS support case (both when AWS provides it, or bringing your own), and the _assignment suffix describes the action of assigning the IP to a pool versus "creating" it.

# assigns an existing IP to an existing dedicated IP pool
resource "aws_sesv2_dedicated_ip_assignment" "example" {
  ip                    = "0.0.0.0"
  destination_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name
}

# a single dedicated IP, returns information on warmup percentage, status, etc.
data "aws_sesv2_dedicated_ip" "example" {
  ip = "0.0.0.0"
}

@kamilturek
Copy link
Collaborator Author

@jar-b That makes sense. Sounds good to me! 👍

@JoshiiSinfield
Copy link

@JoshiiSinfield
Copy link

@kamilturek judging by your 27260 shall I close mine then ?

@jar-b
Copy link
Member

jar-b commented Oct 20, 2022

Thanks for the link @JoshiiSinfield!

Sounds like this will require a new scaling_mode attribute on the aws_sesv2_dedicated_ip_pool resource to allow creation of managed pools. I believe the proposed aws_sesv2_dedicated_ip_assignment resource can remain because IP assignment will still be necessary for standard (non-managed) pools.

Edit: ScalingMode is already available in v1.14.0 of the sesv2 Go SDK, which will be bumped via #27321 after todays release.

@F21
Copy link
Contributor

F21 commented Nov 3, 2022

Would definitely love to see aws_sesv2_email_identity_mail_from_attributes implemented. It's a separate endpoint in Amazon's SESv2 API, but would it make sense to put it as an attribute under aws_sesv2_email_identity?

@kamilturek
Copy link
Collaborator Author

Hey @F21, thanks for your view on this! 🙌

As you mentioned, CreateEmailIdentity API action does not support specifying Mail From attributes but it can be done through a separate endpoint. I'd say we should avoid mixing the PutEmailIdentityMailFromAttributes into aws_sesv2_email_identity CRUD cycle as it increases its complexity and drifts it away from the AWS API design. If we keep adding more and more attributes to aws_sesv2_email_identity that are managed by different API actions, it can become really bloated, error-prone, and hard to maintain.

There is a general tendency of splitting out PUT APIs into their own resources in cases like that. aws_s3_bucket_* resources are a great example of this.

@F21
Copy link
Contributor

F21 commented Nov 7, 2022

I have been testing out the aws_sesv2_configuration_set resource, and I think suppressed_reasons in suppression_options should have its MinItems set to 0 rather than 1: https://github.com/kamilturek/terraform-provider-aws/blob/753ff2aacae8a69f897b64564211b728ad2c3d8c/internal/service/sesv2/configuration_set.go#L109

By setting it to 0, we can use the following:

resource "aws_sesv2_configuration_set" "my_config_set" {
  configuration_set_name = "my-config-set"
  suppression_options {
    suppressed_reasons = []
  }
}

This will allow for the case where we want to override the account-level suppression option and disable the suppression list for this configuration set.

@kamilturek
Copy link
Collaborator Author

Hey @gmx-git, maybe aws_sesv2_configuration_set_event_destination is what you're looking for.

@Scifire
Copy link

Scifire commented Feb 28, 2023

@kamilturek I guess I´m missing the same setting as @gmx-git.
The configuration sets are something different than the feedback notifications.

The feedback notifications are configured in each verified identity under notifications
image
For this I couldn´t find any TF setting.

The configuration sets are an own topic and could be configured as you linked.

@kamilturek
Copy link
Collaborator Author

kamilturek commented Feb 28, 2023

Hey @Scifire. Thank you, I haven't seen that.

Looking at this guide, I think it is available only for API v1. I can't find its equivalent in the API v2 specification. I believe it can be achieved by using the configuration set event destination and configuration set assigned to the email identity.

There is an SES v1 resource for managing these feedback notifications.
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_identity_notification_topic

@gmx-git
Copy link

gmx-git commented Feb 28, 2023

Hey guys,

I've done some research on my own on this topic as well. The APIv1 indeed contained the SetIdentityNotificationTopichttps://docs.aws.amazon.com/ses/latest/APIReference/API_SetIdentityNotificationTopic.html, while there's no correspondence in the APIv2.

I've noted a hint while creating a SES identity (see screenshot below). To me it looks like AWS wants to fade out the notifications at the identity level and recommends shifting to notifications on the configuration set level. We can create/set a default configuration set for an SES identity. The default config set will be used by default when sending emails using the SES identity. We don't need to specify it, but the sender should have the rights to use both the SES identity and the default configuration set assigned to it.

image

On my side I'll review my current design and shift the notifications to the default configuration sets.

Hope this helps!

@Manouchehri
Copy link
Contributor

Manouchehri commented Apr 30, 2023

This is a bit off-topic, but not sure where else to write this note/warning...

Be very careful when migrating from a SESv1 domain identity to SESv2. Destroying the identity doesn't actually instantly cause it to be removed; it seems to stick around, and blocks you from creating a new identity with the exact same domain name. I managed to work around it by manually messing with the state, but it wasn't something I'd recommend repeating.

terraform import aws_sesv2_email_identity.ses_domain YOUR_DOMAIN
terraform state rm aws_ses_domain_identity.ses_domain
terraform state rm aws_ses_domain_dkim.ses_domain_dkim
terraform state rm aws_ses_domain_mail_from.main

terraform apply

I don't think Terraform can address this edge case, since it's straight up the AWS API response that's returning confusing information. (i.e. it can sometimes return a 200 when creating a new identity, and then fail right after when you try go get that identity..)

@mattbnz
Copy link

mattbnz commented Jul 20, 2023

Is there any reason why the ability to request sandbox removal / production migration from Terraform doesn't seem to be mentioned in any of the plans above?

Currently I find this to be a very tedious and manual addition to our otherwise nicely declarative process for turning up in a new region.

Specifically I'm looking for a resource that matches the https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutAccountDetails.html API call.

@jar-b
Copy link
Member

jar-b commented Jul 20, 2023

Hey @mattbnz 👋 - Seems reasonable to me. If you could open a New Resource issue and link back to this one I can add it to the Meta-issue list.

Also, to set expectations, implementation of these resources has largely been community driven thus far as this issue is not currently on the maintainer roadmap.

@kamilturek
Copy link
Collaborator Author

As core SESv2 resources have been already implemented and it's becoming quite difficult to follow and manage the work through a single GitHub issue, I suggest closing this one and opening new fine-grained issues for remaining resources as needed.

I'll keep it open for a few more days in case there are any objections.

@jeremyatenovis
Copy link

Is aws_sesv2_domain_identity implemented? I can't seem to find it. Additionally, I'd very much like a way to associate a default configuration set to an SES identity.

@kamilturek
Copy link
Collaborator Author

Hey @jeremyatenovis, the aws_sesv2_email_identity resource allows you to manage both email address identities and domain identities. Also, there is a way to associate a default configuration set to it.

Please have a look at the documentation page. Let me know if you have any questions.
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sesv2_email_identity

@shamx9ir
Copy link

We are currently using aws_ses_domain_identity and would like to use aws_sesv2_email_identity to be able to assign default configuration set. How do we approach this without accidentally dropping the existing domain identity? Do we have a guide somewhere?

@raffraffraff
Copy link

raffraffraff commented Jan 25, 2024

Since there isn't a resource aws_sesv2_email_identity_policy yet, I assume there's no way to create an SESv2 policy that can allow a Cognito pool to send email via SESv2. The AWS cli has aws sesv2 create-email-identity-policy --email-identity <id> --policy <json_string>, so I suppose I could write some horrendous null-resource stuff (like I had to do with OpenSearch endpoint authorization). I really don't want to have to do that again.

@kamilturek
Copy link
Collaborator Author

@raffraffraff I opened a PR with aws_sesv2_email_identity_policy resource. I should be able to finalize it soon.
#35486

@pspot2
Copy link

pspot2 commented Mar 14, 2024

#22655

@nightpool
Copy link

What is the difference between SES and SESv2? Which one should I use for a new terraform setup? If I choose V2, am I going to run into issues where I cannot create certain resources since they're not supported yet? Why is this a completely separate resource type instead of Terraform just choosing the best API to use on the backend?

@Scifire
Copy link

Scifire commented Apr 12, 2024

@nightpool I´d recommend use SESv2. For normal setups everything should work there. I´ve started with SES but reworked it on v2 shortly later cause there are more recent features available.

@OJFord
Copy link

OJFord commented May 8, 2024

I agree with @nightpool and @JoshiiSinfield (way back at the start of this issue) - it's a real shame to have these as new resources IMO, it's confusing.

People are going to search resources on the registry for 'ses', then use v1 because it's listed first and v2 not even on the screen. If they do see v2, they'll look in the console for SESv2 and find nothing; search for information for it and find the CLI reference or a blog post, and think why am I having to think about this I'm just using terraform this is an implementation detail, and choose arbitrarily between them (v1 because it looks cleaner and is more established/battle-tested? v2 because it's newer and presumably further from being ever retired?).

@kamilturek
Copy link
Collaborator Author

Answering a few comments - it was a concious decision to keep SESv1 and SESv2 as separate services preceded by discussions and agreements. The same thing was done for apigatewayv2, elbv2 etc. The goal is to keep the provider resources aligned with how AWS structures their APIs, SDKs and the CLI.

As announced a few months agowith core SESv2 resources already implemented, I am closing this issue and encourage to open new fine-grained issues for remaining resources as needed.

Copy link

github-actions bot commented Aug 7, 2024

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

Copy link

github-actions bot commented Sep 8, 2024

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 8, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement Requests to existing resources that expand the functionality or scope. new-service Introduces a new service. service/sesv2 Issues and PRs that pertain to the sesv2 service.
Projects
None yet
Development

No branches or pull requests