aws_securityhub_organization_admin_account and ResourceConflictException

[rtoma]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

• Terraform v0.14.7
• provider registry.terraform.io/hashicorp/aws v3.31.0

Affected Resource(s)

• aws_securityhub_organization_admin_account

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

I want to delegate the SecurityHub admin account to another non-management account in same Organization. I want this done for all regions. For multi-region support I have providers defined: one per region. I have a "region-module.tf" file (see below) that calls "base" module (see below) for each region and get the regional provider passed. The "base" module acts like a proxy and calls the "securityhub" module (see below), which includes the aws_securityhub_organization_admin_account resource.

region-modules.tf:

module "ap-northeast-1" {
  count  = contains(local.enabled_regions, "ap-northeast-1") ? 1 : 0
  source = "./modules/base"
  providers = {
    aws = aws.ap-northeast-1
  }

  securityhub = {
    is_organization_management_account = local.is_organization_management_account
    admin_account_id                   = local.securityhub_admin_account_id
  }
}

module "ap-northeast-2" {
  ..
}

# and so on for all regions

modules/base/main.tf:

variable "securityhub" {}

module "securityhub_organization_regional" {
  count            = var.securityhub.is_organization_management_account ? 1 : 0
  source           = "../securityhub/organization_regional"
  admin_account_id = var.securityhub.admin_account_id
}

modules/securityhub/organization_regional/main.tf:

variable "admin_account_id" {}

resource "aws_securityhub_organization_admin_account" "admin" {
  admin_account_id = var.admin_account_id
}

Debug Output

Panic Output

Expected Behavior

16 aws_securityhub_organization_admin_account resources get created.

Actual Behavior

On first run 11 resources were created but I got 5 errors like:

Error: error enabling Security Hub Organization Admin Account (redacted): ResourceConflictException: AWS Organizations can't complete your request because it conflicts with another attempt to modify the same entity. Try again later.
{
  RespMetadata: {
    StatusCode: 409,
    RequestID: "c009298a-c872-4054-9b23-2ca84ed70186"
  },
  Code_: "ResourceConflictException",
  Message_: "AWS Organizations can't complete your request because it conflicts with another attempt to modify the same entity. Try again later."
}

On retry 4 resources got created, leaving 1 error. Another retry fixed that.

Even though the resource is regional it seems to contain a global component that is throttled or locked. I assume this makes it impossible to fix in Terraform. So I created this bug mostly so other users are aware of this behaviour.

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[anGie44]

Hi @rtoma, thank you for raising this issue! I had a chance to reproduce this error (with 4 and 6 regions) you're seeing with concurrent resource creation and I believe we can improve this within the resource by catching the exception and retrying the EnableOrganizationAdminAccount API method instead of immediately failing.

[ghost]

This has been released in version 3.35.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!