terraform resources for the Cost Explorer Service #16137

b-b3rn4rd · 2020-11-11T03:31:27Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS provides a dedicated set of APIs for managing AWS Cost Explorer.
https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/Welcome.html

I'm finding it would be handy to be able to provision some of those resource using terraform particularly I'm interested in https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_CreateAnomalyMonitor.html

New or Affected Resource(s)

aws_cost_explorer_create_anomaly_monitor

Potential Terraform Configuration

resource " aws_cost_explorer_create_anomaly_monitor" "anomaly_spending" {
  monitor_name = "unexpected-aws-charges"
  monitor_type = "DIMENSIONAL"
  monitor_specification = <<EOF
{ 
         "And": [ 
            "Expression"
         ],
         "CostCategories": { 
            "Key": "string",
            "MatchOptions": [ "string" ],
            "Values": [ "string" ]
         },
         "Dimensions": { 
            "Key": "string",
            "MatchOptions": [ "string" ],
            "Values": [ "string" ]
         },
         "Not": "Expression",
         "Or": [ 
            "Expression"
         ],
         "Tags": { 
            "Key": "string",
            "MatchOptions": [ "string" ],
            "Values": [ "string" ]
         }
      },
      "MonitorType": "string"
}
EOF
}

References

https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_CreateAnomalyMonitor.html

borgoat · 2021-02-01T12:52:03Z

Also interested in this, I could work on the implementation!

I'd have a few points to discuss before working on it though:

The "Cost Explorer" API is named simply ce in the CLI, SDKs, and the endpoint. Similarly to how the "Cost and Usage Report" is referred to as cur. Therefore, I believe we should use aws_ce_* as prefix to the resources.
I would call this resource aws_ce_anomaly_monitor since the call to Create/Update actions is more of an implementation detail to Terraform than the name of the actual resource.
I believe we should create an aws_ce_anomaly_subscription based on ce:CreateAnomalySubscription since that's very needed to make use of an Anomaly Monitor.
Finally, I'm not sure what the structure should be for the monitor_specification field: the Expression object is a generic object that is also for example used in the ce:CreateCostCategoryDefinition action; and I guess in the future it may be used for other Cost Explorer actions too. I wonder if we should create a "datasource" à la aws_iam_policy_document?

sshvetsov · 2021-03-13T09:01:42Z

Looks like CloudFormation support for Anomaly Monitor (AWS::CE::AnomalyMonitor) and Anomaly Subscription (AWS::CE::AnomalySubscription) is about to go live. Providing documentation links in case they're helpful in crafting relevant Terraform resources.

In case triage team missed it, here's an old request to support Cost Categories (another Cost Explorer service): #12801.

sshvetsov · 2021-03-13T10:08:08Z

Here's updated description for this feature request, with focus on Cost Anomaly Detection.

Description

AWS has announced General Availability of AWS Cost Anomaly Detection on Dec. 16, 2020.

AWS Cost Anomaly Detection uses a multi-layered machine learning model that learns your unique, historic spend patterns to detect one-time cost spike and/or continuous cost increases, without you having to define your thresholds. Every anomaly detected will be available in the detection history tab. We send you the anomaly detection report with root-cause analysis. And the service also comes at no cost to our customers

Basically, it's a rare FREE AWS service that can alert users about anomalies in their infrastructure costs, making it an extremely useful service for anyone operating infrastructure on AWS.

New or Affected Resource(s)

aws_ce_anomaly_monitor
aws_ce_anomaly_subscription

Potential Terraform Configuration

Simple example:

resource "aws_ce_anomaly_monitor" "service_monitor" {
  name      = "AWSServiceMonitor"
  type      = "DIMENSIONAL"
  dimension = "SERVICE"
}

resource "aws_ce_anomaly_subscription" "anomaly_subscription" {
  name      = "DailyAnomalySubscription"
  threshold = 100
  frequency = "DAILY"

  monitor_arn_list = [
    aws_ce_anomaly_subscription.anomaly_subscription.arn,
  ]

  subscribers = [
    {
      type    = "EMAIL"
      address = "abc@example.com"
    }
  ]
}

Complex example:

resource "aws_ce_anomaly_monitor" "linked_account_monitor" {
  name = "LinkedAccountMonitor"
  type = "CUSTOM"

  specification = <<JSON
{
    "Dimensions": {
        "Key": "LINKED_ACCOUNT",
        "Values": [
            "123456789012",
            "123456789013"
        ]
    }
}
  JSON
}

resource "aws_sns_topic" "cost_anomaly_updates" {
  name = "CostAnomalyUpdates"
}

data "aws_iam_policy_document" "sns_topic_policy" {
  policy_id = "__default_policy_ID"

  statement {
    sid = "AWSAnomalyDetectionSNSPublishingPermissions"

    actions = [
      "SNS:Publish",
    ]

    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["costalerts.amazonaws.com"]
    }

    resources = [
      aws_sns_topic.cost_anomaly_updates.arn,
    ]
  }

  statement {
    sid = "__default_statement_ID"

    actions = [
      "SNS:Subscribe",
      "SNS:SetTopicAttributes",
      "SNS:RemovePermission",
      "SNS:Receive",
      "SNS:Publish",
      "SNS:ListSubscriptionsByTopic",
      "SNS:GetTopicAttributes",
      "SNS:DeleteTopic",
      "SNS:AddPermission",
    ]

    condition {
      test     = "StringEquals"
      variable = "AWS:SourceOwner"

      values = [
        var.account-id,
      ]
    }

    effect = "Allow"

    principals {
      type        = "AWS"
      identifiers = ["*"]
    }

    resources = [
      aws_sns_topic.cost_anomaly_updates.arn,
    ]
  }
}

resource "aws_sns_topic_policy" "default" {
  arn = aws_sns_topic.cost_anomaly_updates.arn

  policy = data.aws_iam_policy_document.sns_topic_policy.json
}

resource "aws_ce_anomaly_subscription" "realtime_subscription" {
  name      = "RealtimeAnomalySubscription"
  threshold = 0
  frequency = "IMMEDIATE"

  monitor_arn_list = [
    aws_ce_anomaly_subscription.anomaly_subscription.arn,
  ]

  subscribers = [
    {
      type    = "SNS"
      address = aws_sns_topic.cost_anomaly_updates.arn
    }
  ]

  depends_on = [
    aws_sns_topic_policy.default,
  ]
}

References

udayanms · 2022-01-24T22:18:02Z

Any planned date for this. I certainly need this.

stobias123 · 2022-04-12T03:49:04Z

Bump. This would be great, especially since tag based cost anomaly detection is now available.

williamcodes · 2022-04-12T13:23:21Z

Yes, I would love to have this as well.

headincl0ud · 2022-04-12T13:28:20Z

Be patient :) I started working on it.

breathingdust · 2022-04-26T16:19:53Z

Related #12801

mathewmoon · 2022-05-11T14:23:49Z

💯 On this. Thanks for getting this rolling. @headincl0ud Is the a projection on what release/date this could arrive?

breathingdust · 2022-05-12T16:33:22Z

Hi all 👋 Just letting you know that this is issue is featured on this quarters roadmap. If a PR exists to close the issue a maintainer will review and either make changes directly, or work with the original author to get the contribution merged. If you have written a PR to resolve the issue please ensure the "Allow edits from maintainers" box is checked. Thanks for your patience and we are looking forward to getting this merged soon!

zhelding · 2022-06-22T18:59:38Z

Hi folks! #25177 and #25224 have added support for aws_ce_anomaly_monitor and aws_ce_anomaly_subscription resources respectively. I believe this resolves this issue.

Special thanks to @brittandeyoung for his PRs! 👏🏻 And thank you to everyone in this thread for contributing to the discussion around these resources.

If there are any additional resources desired beyond those proposed by this issue: please open a new issue detailing the resource and your potential use case.

github-actions · 2022-07-23T02:37:03Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

b-b3rn4rd added the enhancement Requests to existing resources that expand the functionality or scope. label Nov 11, 2020

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Nov 11, 2020

ewbankkit added service/costexplorer new-resource Introduces a new resource. and removed needs-triage Waiting for first response or review from a maintainer. labels Nov 11, 2020

hc-github-team-terraform-aws removed the service/costexplorer label Apr 26, 2022

This was referenced Jun 4, 2022

New Resource: aws_ce_anomaly_monitor #25177

Merged

New Resource: aws_ce_anomaly_subscription #25224

Merged

zhelding closed this as completed Jun 22, 2022

zhelding self-assigned this Jun 22, 2022

This was referenced Jun 30, 2022

New data source aws_ce_anomaly_monitor or change aws_ce_anomaly_monitor #25632

Closed

aws_ce_anomaly_subscription type of subscriber address must be a "list" #25685

Closed

github-actions bot locked as resolved and limited conversation to collaborators Jul 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

terraform resources for the Cost Explorer Service #16137

terraform resources for the Cost Explorer Service #16137

b-b3rn4rd commented Nov 11, 2020

borgoat commented Feb 1, 2021

sshvetsov commented Mar 13, 2021

sshvetsov commented Mar 13, 2021

udayanms commented Jan 24, 2022

stobias123 commented Apr 12, 2022 •

edited

Loading

williamcodes commented Apr 12, 2022

headincl0ud commented Apr 12, 2022

breathingdust commented Apr 26, 2022

mathewmoon commented May 11, 2022 •

edited

Loading

breathingdust commented May 12, 2022

zhelding commented Jun 22, 2022

github-actions bot commented Jul 23, 2022

terraform resources for the Cost Explorer Service #16137

terraform resources for the Cost Explorer Service #16137

Comments

b-b3rn4rd commented Nov 11, 2020

Community Note

Description

New or Affected Resource(s)

Potential Terraform Configuration

References

borgoat commented Feb 1, 2021

sshvetsov commented Mar 13, 2021

sshvetsov commented Mar 13, 2021

Description

New or Affected Resource(s)

Potential Terraform Configuration

References

udayanms commented Jan 24, 2022

stobias123 commented Apr 12, 2022 • edited Loading

williamcodes commented Apr 12, 2022

headincl0ud commented Apr 12, 2022

breathingdust commented Apr 26, 2022

mathewmoon commented May 11, 2022 • edited Loading

breathingdust commented May 12, 2022

zhelding commented Jun 22, 2022

github-actions bot commented Jul 23, 2022

stobias123 commented Apr 12, 2022 •

edited

Loading

mathewmoon commented May 11, 2022 •

edited

Loading