DLM policy with cross_region_copy_rule can't be created #24226

dpavlov-smartling · 2022-04-13T19:38:33Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Trying to use DLM cross region copy feature released in #12868 . Getting error message that Target parameter is invalid.
It is possible that issue is related to the AWS DLM API transition from TargetRegion parameter to Target. More details [here] (https://docs.aws.amazon.com/dlm/latest/APIReference/API_CrossRegionCopyRule.html#dlm-Type-CrossRegionCopyRule-Target)
Was able to reproduce similar behavior using old awscli tool

aws dlm create-lifecycle-policy --description 'Test' --state DISABLED --execution-role-arn arn:aws:iam::XXXXXXXXXX:role/service-role/AWSDataLifecycleManagerDefaultRoleForAMIManagement --policy-details file://policy.json

Parameter validation failed:
Missing required parameter in PolicyDetails.Schedules[0].CrossRegionCopyRules[0]: "TargetRegion"
Unknown parameter in PolicyDetails.Schedules[0].CrossRegionCopyRules[0]: "Target", must be one of: TargetRegion, Encrypted, CmkArn, CopyTags, RetainRule
exit status 252

So, the root of the issue can be in https://github.com/aws/aws-sdk-go/blob/main/service/dlm/api.go , but after brief view didn't find anything that can help.

Terraform CLI and Terraform AWS Provider Version

terraform_118 -v
Terraform v1.1.8
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.9.0

Affected Resource(s)

aws_dlm_lifecycle_policy

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_dlm_lifecycle_policy" "dlm" {
  description        = "dlm"
  execution_role_arn = "arn:aws:iam::XXXXXXXXXXXXX:role/service-role/AWSDataLifecycleManagerDefaultRoleForAMIManagement"
  state              = "DISABLED"
  policy_details {
    resource_types = ["INSTANCE"]
    policy_type    = "IMAGE_MANAGEMENT"

    schedule {
      name = "1 weeks of daily snapshots"

      create_rule {
        interval      = 24
        interval_unit = "HOURS"
        times         = ["23:45"]
      }

      retain_rule {
        count = 7
      }

      tags_to_add = {
        SnapshotCreator = "DLM"
      }

      copy_tags = true

      cross_region_copy_rule {
        target    = "us-east-2"
        encrypted = false
        copy_tags = true
        retain_rule {
          interval      = 7
          interval_unit = "DAYS"
        }
      }
    }
    parameters {
      no_reboot = true
    }

    target_tags = {
      ami_backup = "zxc"
    }
  }
}

Debug Output

2022-04-13T15:08:03.738-0400 [DEBUG] provider.terraform-provider-aws_v4.9.0_x5: [aws-sdk-go] DEBUG: Request DLM/CreateLifecyclePolicy Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST /policies HTTP/1.1
Host: dlm.us-east-1.amazonaws.com
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.1.8 (+https://www.terraform.io) terraform-provider-aws/dev (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.43.34 (go1.17.6; darwin; amd64)
Content-Length: 665
Content-Type: application/json
X-Amz-Date: 20220413T190803Z
Accept-Encoding: gzip

{"Description":"dlm","ExecutionRoleArn":"arn:aws:iam::XXXXXXXXXX:role/service-role/AWSDataLifecycleManagerDefaultRoleForAMIManagement","PolicyDetails":{"Parameters":{"NoReboot":true},"PolicyType":"IMAGE_MANAGEMENT","ResourceTypes":["INSTANCE"],"Schedules":[{"CopyTags":true,"CreateRule":{"Interval":24,"IntervalUnit":"HOURS","Times":["23:45"]},"CrossRegionCopyRules":[{"CopyTags":true,"Encrypted":false,"RetainRule":{"Interval":7,"IntervalUnit":"DAYS"},"Target":"us-east-2"}],"Name":"1 weeks of daily snapshots","RetainRule":{"Count":7},"TagsToAdd":[{"Key":"SnapshotCreator","Value":"DLM"}]}],"TargetTags":[{"Key":"ami_backup","Value":"zxc"}]},"State":"DISABLED"}
-----------------------------------------------------: timestamp=2022-04-13T15:08:03.738-0400
2022-04-13T15:08:03.840-0400 [DEBUG] provider.terraform-provider-aws_v4.9.0_x5: [aws-sdk-go] DEBUG: Response DLM/CreateLifecyclePolicy Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Content-Length: 150
Content-Type: application/json
Date: Wed, 13 Apr 2022 19:08:03 GMT
X-Amz-Apigw-Id: QiHfHF3uIAMFqAQ=
X-Amzn-Errortype: InvalidRequestException
X-Amzn-Requestid: 7cb65631-c2e6-467a-8c61-4a6ad5208843
X-Amzn-Trace-Id: Root=1-62571f93-28108edd5368f7cf34992155


-----------------------------------------------------: timestamp=2022-04-13T15:08:03.840-0400
2022-04-13T15:08:03.841-0400 [DEBUG] provider.terraform-provider-aws_v4.9.0_x5: [aws-sdk-go] {"Code":"InvalidParameter","RequiredParameters":null,"MutuallyExclusiveParameters":null,"Message":"The following parameters(s) are invalid: {Target}"}: timestamp=2022-04-13T15:08:03.840-0400
2022-04-13T15:08:03.841-0400 [DEBUG] provider.terraform-provider-aws_v4.9.0_x5: [aws-sdk-go] DEBUG: Validate Response DLM/CreateLifecyclePolicy failed, attempt 0/5, error InvalidRequestException: The following parameters(s) are invalid: {Target}
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "7cb65631-c2e6-467a-8c61-4a6ad5208843"
  },
  Code_: "InvalidParameter",
  Message_: "The following parameters(s) are invalid: {Target}"
}: timestamp=2022-04-13T15:08:03.840-0400
2022-04-13T15:08:06.174-0400 [WARN]  provider.terraform-provider-aws_v4.9.0_x5: WaitForState timeout after 2m0s: timestamp=2022-04-13T15:08:06.173-0400
2022-04-13T15:08:06.174-0400 [WARN]  provider.terraform-provider-aws_v4.9.0_x5: WaitForState starting 30s refresh grace period: timestamp=2022-04-13T15:08:06.174-0400
2022-04-13T15:08:06.176-0400 [WARN]  unexpected data: registry.terraform.io/hashicorp/aws:stderr="{"@caller":"github.com/hashicorp/terraform-plugin-sdk/v2@v2.13.0/internal/logging/helper_schema.go:21","@level":"trace","@message":"Called downstream","@module":"sdk.helper_schema","@timestamp":"2022-04-13T15:08:06.175272-04:00"}"
2022-04-13T15:08:06.196-0400 [ERROR] vertex "aws_dlm_lifecycle_policy.dlm" error: error creating DLM Lifecycle Policy: InvalidRequestException: The following parameters(s) are invalid: {Target}
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "7cb65631-c2e6-467a-8c61-4a6ad5208843"
  },
  Code_: "InvalidParameter",
  Message_: "The following parameters(s) are invalid: {Target}"
}
╷
│ Error: error creating DLM Lifecycle Policy: InvalidRequestException: The following parameters(s) are invalid: {Target}
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "7cb65631-c2e6-467a-8c61-4a6ad5208843"
│   },
│   Code_: "InvalidParameter",
│   Message_: "The following parameters(s) are invalid: {Target}"
│ }
│
│   with aws_dlm_lifecycle_policy.dlm,
│   on policy.tf line 1, in resource "aws_dlm_lifecycle_policy" "dlm":
│    1: resource "aws_dlm_lifecycle_policy" "dlm" {
│
╵

Expected Behavior

DLM policy should be created with cross copy feature

Actual Behavior

Error message

Steps to Reproduce

Apply provided terraform

Important Factoids

References

AWS DLM API transition from TargetRegion to Target

The text was updated successfully, but these errors were encountered:

nikhilpatne2910 · 2023-01-04T09:39:09Z

Hello I am getting exactly same issue. Please suggest something on this.

Kartoshnikov · 2023-07-28T14:52:23Z

It's been more than a year and nobody gives rats about it?

ymaniukevich · 2023-09-04T16:02:36Z

It doesn't work when 'resource_types = ["INSTANCE"]', with resource_types = ["VOLUME"] it works

nikhilpatne2910 · 2023-09-04T16:09:14Z

Okay thanks for the update.

…

On Mon, 4 Sept, 2023, 9:32 pm Yauheni, ***@***.***> wrote: It doesn't work when 'resource_types = ["INSTANCE"]', with resource_types = ["VOLUME"] it works — Reply to this email directly, view it on GitHub <#24226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZ6Q4UBAJOU4XJPMTBBQSSLXYX3SRANCNFSM5TLVSSIA> . You are receiving this because you commented.Message ID: ***@***.***>

Kartoshnikov · 2023-09-18T11:13:12Z

After some time I got back on this issue.
I used CloudFormation to crate a policy of a type "IMAGE_MANAGEMENT" with cross-region copy enabled.

AWSTemplateFormatVersion: 2010-09-09
Description: DLM Policy
Resources:
  DLM:
    Type: AWS::DLM::LifecyclePolicy
    Properties:
      Description: Daily DLM Policy
      State: ENABLED
      ExecutionRoleArn: arn:aws:iam::111111111111:role/service-role/AWSDataLifecycleManagerDefaultRoleForAMIManagement
      PolicyDetails:
        PolicyType: IMAGE_MANAGEMENT
        ResourceTypes:
        - INSTANCE
        TargetTags:
        - Key: test
          Value: test
        Parameters:
          NoReboot: true
        Schedules:
        - Name: DailyTest
          CopyTags: true
          CreateRule:
            Interval: 24
            IntervalUnit: HOURS
            Times:
              - 00:01
          RetainRule:
            Count: 5
          CrossRegionCopyRules:
          - CopyTags: true
            Encrypted: true
            RetainRule:
              Interval: 5
              IntervalUnit: DAYS
            TargetRegion: us-west-1
          VariableTags:
          - Key: instance-id
            Value: $(instance-id)

It worked and created DLM policy.
As you might've already noticed I used TargetRegion instead of Target.

So it IS an issue of Terraform giving us no option to choose b/w these parameters.

crisbal · 2023-10-06T06:47:51Z

I have talked with AWS support about this. Unfortunately at the moment, they told me it is expected that Target does not work for IMAGE_MANAGEMENT and to use TargetRegion, and have updated the docs accordingly.

https://docs.aws.amazon.com/dlm/latest/APIReference/API_CrossRegionCopyRule.html

This means that the AWS provider will need to be updated to support TargetRegion as an input parameter for IMAGE_MANAGEMENT policies, because right now, only Target can be specified.

I will see if I can patch this myself, otherwise I might need to create another ticket clarifying this.

…py_rule `target_region` should only be used for dlm policies of `policy_type=IMAGE_MANAGEMENT`. Creating `IMAGE_MANAGEMENT` policies using `target` raises a validation error from the AWS API. Previously the argument was marked as deprecated from the AWS docs, but after contacting the AWS support, the documentation was updated to specify that `TargetRegion` should be used only for `IMAGE_MANAGEMENT` policies, while `Target` only for `EBS_SNAPSHOT_MANAGEMENT` policies. See the relevant docs: https://docs.aws.amazon.com/dlm/latest/APIReference/API_CrossRegionCopyRule.html Relevant issues: - https://discuss.hashicorp.com/t/fail-to-create-a-cross-region-copy-rule-for-dlp/46076 - hashicorp#24226 This fixes hashicorp#24226

crisbal · 2023-10-06T07:42:55Z

#33796 Here it is, feel free to 👍 it so it gets seen easier.

github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/dlm Issues and PRs that pertain to the dlm service. labels Apr 13, 2022

justinretzolk added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Apr 14, 2022

crisbal linked a pull request Oct 6, 2023 that will close this issue

feat: dlm: Add support for target_region parameter in cross_region_copy_rule #33796

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DLM policy with cross_region_copy_rule can't be created #24226

DLM policy with cross_region_copy_rule can't be created #24226

dpavlov-smartling commented Apr 13, 2022

nikhilpatne2910 commented Jan 4, 2023

Kartoshnikov commented Jul 28, 2023

ymaniukevich commented Sep 4, 2023

nikhilpatne2910 commented Sep 4, 2023 via email

Kartoshnikov commented Sep 18, 2023 •

edited

Loading

crisbal commented Oct 6, 2023 •

edited

Loading

crisbal commented Oct 6, 2023

DLM policy with cross_region_copy_rule can't be created #24226

DLM policy with cross_region_copy_rule can't be created #24226

Comments

dpavlov-smartling commented Apr 13, 2022

Community Note

Description

Terraform CLI and Terraform AWS Provider Version

Affected Resource(s)

Terraform Configuration Files

Debug Output

Expected Behavior

Actual Behavior

Steps to Reproduce

Important Factoids

References

nikhilpatne2910 commented Jan 4, 2023

Kartoshnikov commented Jul 28, 2023

ymaniukevich commented Sep 4, 2023

nikhilpatne2910 commented Sep 4, 2023 via email

Kartoshnikov commented Sep 18, 2023 • edited Loading

crisbal commented Oct 6, 2023 • edited Loading

crisbal commented Oct 6, 2023

Kartoshnikov commented Sep 18, 2023 •

edited

Loading

crisbal commented Oct 6, 2023 •

edited

Loading