Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_datasync_task: Cloud watch log group ARN regex validation #13046

Closed
stellemd opened this issue Apr 27, 2020 · 6 comments · Fixed by #14214
Closed

aws_datasync_task: Cloud watch log group ARN regex validation #13046

stellemd opened this issue Apr 27, 2020 · 6 comments · Fixed by #14214
Assignees
Labels
service/cloudwatch Issues and PRs that pertain to the cloudwatch service. service/datasync Issues and PRs that pertain to the datasync service.
Milestone

Comments

@stellemd
Copy link

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform version: 0.12.24

Affected Resource(s)

  • aws_datasync_task

Terraform Configuration Files

resource "aws_datasync_task" "this" {
  destination_location_arn = "arn:aws:datasync:us-east-1:123456789123:location/loc-02ca0734e27104g8d"
  source_location_arn      = "arn:aws:datasync:us-east-1:123456789123:location/loc-Xcce5febb7a3a6a45"
  cloudwatch_log_group_arn = aws_cloudwatch_log_group.default.arn
  name                     = "Test Media Sync"

  options {
          atime                  = "BEST_EFFORT"
          bytes_per_second       = -1
          gid                    = "INT_VALUE"
          mtime                  = "PRESERVE"
          posix_permissions      = "PRESERVE"
          preserve_deleted_files = "REMOVE"
          preserve_devices       = "NONE"
          uid                    = "INT_VALUE"
          verify_mode            = "POINT_IN_TIME_CONSISTENT"
        }
}

resource "aws_cloudwatch_log_group" "default" {
  name              = local.cloudwatch_log_group_name
  retention_in_days = 400

  tags =  local.cloudwatch_log_group_tags
}

Debug Output

https://github.com/stellemd/aws_data_sync_task/blob/master/debug.txt

Panic Output

Expected Behavior

The aws data sync task resource will accept the cloud watch log group arn created via an aws_cloudwatch_log_group resource and the cloud watch log group will be attached for task logging.

Actual Behavior

Error: error creating DataSync Task: ValidationException: 1 validation error detected: Value 'arn:aws:logs:us-east-1:123456789123:log-group:test_media_data_sync_non_prod:*' at 'cloudWatchLogGroupArn' failed to satisfy constraint: Member must satisfy regular expression pattern: ^arn:(aws|aws-cn|aws-us-gov|aws-iso|aws-iso-b):logs:[a-z\-0-9]*:[0-9]{12}:log-group:([^:\*]*)$
        status code: 400, request id: 2883fee4-5bf5-4969-8428-bb90fe6fc23b

  on main.tf line 35, in resource "aws_datasync_task" "this":
  35: resource "aws_datasync_task" "this" {

Steps to Reproduce

  1. terraform apply

Important Factoids

References

  • #0000
@ghost ghost added service/cloudwatchlogs service/cloudwatch Issues and PRs that pertain to the cloudwatch service. service/datasync Issues and PRs that pertain to the datasync service. labels Apr 27, 2020
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Apr 27, 2020
@mehfuzh
Copy link

mehfuzh commented Jun 29, 2020

Found this workaround for now:

join("", split(":*", aws_cloudwatch_log_group.this.arn))

@bflad bflad added this to the v3.0.0 milestone Jun 30, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Jun 30, 2020
@bflad
Copy link
Contributor

bflad commented Jun 30, 2020

FYI, this should be resolved in our upcoming major release (version 3.0.0) within the next few weeks. We plan on automatically removing the :* returned in the ARN by the CloudWatch Logs API, so it is like all other Terraform AWS Provider arn attributes. I have marked this issue with that milestone so we know to circle back and close this issue when its resolved as part of that release.

@bflad bflad self-assigned this Jul 16, 2020
bflad added a commit that referenced this issue Jul 16, 2020
…ARN in API response

Reference: #13046
Reference: #13509

Previously:

```
    TestAccAWSDataSyncTask_CloudWatchLogGroupARN: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DataSync Task: ValidationException: 1 validation error detected: Value 'arn:aws:logs:us-west-2:123456789012:log-group:tf-acc-test-4735468151095290255:*' at 'cloudWatchLogGroupArn' failed to satisfy constraint: Member must satisfy regular expression pattern: ^arn:(aws|aws-cn|aws-us-gov|aws-iso|aws-iso-b):logs:[a-z\-0-9]*:[0-9]{12}:log-group:([^:\*]*)$
```

Output from acceptance testing (`aws_route53_query_log` failure related to similar issue #13510):

```
--- PASS: TestAccAWSCloudWatchLogGroup_disappears (9.19s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix (13.55s)
--- PASS: TestAccAWSCloudWatchLogGroup_generatedName (13.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_basic (15.24s)
--- PASS: TestAccAWSCloudWatchLogGroup_multiple (15.65s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix_retention (21.29s)
--- PASS: TestAccAWSCloudWatchLogGroup_retentionPolicy (24.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_kmsKey (29.00s)
--- PASS: TestAccAWSCloudWatchLogGroup_tagging (35.60s)

--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings (225.36s)
--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings_kinesis (332.67s)

--- PASS: TestAccAWSAPIGatewayV2Stage_AccessLogSettings (56.73s)

--- PASS: TestAccAWSDataSyncTask_CloudWatchLogGroupARN (304.98s)

--- PASS: TestAccAWSDirectoryServiceLogSubscription_basic (1764.25s)

--- PASS: TestAccAWSElasticSearchDomain_LogPublishingOptions (688.17s)

--- PASS: TestAccAWSFlowLog_LogDestinationType_CloudWatchLogs (26.43s)

--- FAIL: TestAccAWSRoute53QueryLog_Basic (42.80s)
    TestAccAWSRoute53QueryLog_Basic: testing.go:684: Step 0 error: errors during apply:

        Error: Provider produced inconsistent final plan

        When expanding the plan for aws_cloudwatch_log_group.test to include new
        values learned so far during apply, provider "aws" produced an invalid new
        value for .name: was
        cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com"), but
        now cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com.").

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

--- PASS: TestAccAWSStorageGatewayGateway_CloudWatchLogs (220.06s)
```
bflad added a commit that referenced this issue Jul 25, 2020
…ARN in API response

Reference: #13046
Reference: #13509

Previously:

```
    TestAccAWSDataSyncTask_CloudWatchLogGroupARN: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DataSync Task: ValidationException: 1 validation error detected: Value 'arn:aws:logs:us-west-2:123456789012:log-group:tf-acc-test-4735468151095290255:*' at 'cloudWatchLogGroupArn' failed to satisfy constraint: Member must satisfy regular expression pattern: ^arn:(aws|aws-cn|aws-us-gov|aws-iso|aws-iso-b):logs:[a-z\-0-9]*:[0-9]{12}:log-group:([^:\*]*)$
```

Output from acceptance testing (`aws_route53_query_log` failure related to similar issue #13510):

```
--- PASS: TestAccAWSCloudWatchLogGroup_disappears (9.19s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix (13.55s)
--- PASS: TestAccAWSCloudWatchLogGroup_generatedName (13.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_basic (15.24s)
--- PASS: TestAccAWSCloudWatchLogGroup_multiple (15.65s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix_retention (21.29s)
--- PASS: TestAccAWSCloudWatchLogGroup_retentionPolicy (24.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_kmsKey (29.00s)
--- PASS: TestAccAWSCloudWatchLogGroup_tagging (35.60s)

--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings (225.36s)
--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings_kinesis (332.67s)

--- PASS: TestAccAWSAPIGatewayV2Stage_AccessLogSettings (56.73s)

--- PASS: TestAccAWSDataSyncTask_CloudWatchLogGroupARN (304.98s)

--- PASS: TestAccAWSDirectoryServiceLogSubscription_basic (1764.25s)

--- PASS: TestAccAWSElasticSearchDomain_LogPublishingOptions (688.17s)

--- PASS: TestAccAWSFlowLog_LogDestinationType_CloudWatchLogs (26.43s)

--- FAIL: TestAccAWSRoute53QueryLog_Basic (42.80s)
    TestAccAWSRoute53QueryLog_Basic: testing.go:684: Step 0 error: errors during apply:

        Error: Provider produced inconsistent final plan

        When expanding the plan for aws_cloudwatch_log_group.test to include new
        values learned so far during apply, provider "aws" produced an invalid new
        value for .name: was
        cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com"), but
        now cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com.").

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

--- PASS: TestAccAWSStorageGatewayGateway_CloudWatchLogs (220.06s)
```
bflad added a commit that referenced this issue Jul 30, 2020
…ARN in API response

Reference: #13046
Reference: #13509

Previously:

```
    TestAccAWSDataSyncTask_CloudWatchLogGroupARN: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DataSync Task: ValidationException: 1 validation error detected: Value 'arn:aws:logs:us-west-2:123456789012:log-group:tf-acc-test-4735468151095290255:*' at 'cloudWatchLogGroupArn' failed to satisfy constraint: Member must satisfy regular expression pattern: ^arn:(aws|aws-cn|aws-us-gov|aws-iso|aws-iso-b):logs:[a-z\-0-9]*:[0-9]{12}:log-group:([^:\*]*)$
```

Output from acceptance testing (`aws_route53_query_log` failure related to similar issue #13510):

```
--- PASS: TestAccAWSCloudWatchLogGroup_disappears (9.19s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix (13.55s)
--- PASS: TestAccAWSCloudWatchLogGroup_generatedName (13.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_basic (15.24s)
--- PASS: TestAccAWSCloudWatchLogGroup_multiple (15.65s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix_retention (21.29s)
--- PASS: TestAccAWSCloudWatchLogGroup_retentionPolicy (24.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_kmsKey (29.00s)
--- PASS: TestAccAWSCloudWatchLogGroup_tagging (35.60s)

--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings (225.36s)
--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings_kinesis (332.67s)

--- PASS: TestAccAWSAPIGatewayV2Stage_AccessLogSettings (56.73s)

--- PASS: TestAccAWSDataSyncTask_CloudWatchLogGroupARN (304.98s)

--- PASS: TestAccAWSDirectoryServiceLogSubscription_basic (1764.25s)

--- PASS: TestAccAWSElasticSearchDomain_LogPublishingOptions (688.17s)

--- PASS: TestAccAWSFlowLog_LogDestinationType_CloudWatchLogs (26.43s)

--- FAIL: TestAccAWSRoute53QueryLog_Basic (42.80s)
    TestAccAWSRoute53QueryLog_Basic: testing.go:684: Step 0 error: errors during apply:

        Error: Provider produced inconsistent final plan

        When expanding the plan for aws_cloudwatch_log_group.test to include new
        values learned so far during apply, provider "aws" produced an invalid new
        value for .name: was
        cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com"), but
        now cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com.").

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

--- PASS: TestAccAWSStorageGatewayGateway_CloudWatchLogs (220.06s)
```
bflad added a commit that referenced this issue Jul 30, 2020
…ARN in API response (#14214)

* resource/aws_cloudwatch_log_group: Automatically trim :* suffix from ARN in API response

Reference: #13046
Reference: #13509

Previously:

```
    TestAccAWSDataSyncTask_CloudWatchLogGroupARN: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DataSync Task: ValidationException: 1 validation error detected: Value 'arn:aws:logs:us-west-2:123456789012:log-group:tf-acc-test-4735468151095290255:*' at 'cloudWatchLogGroupArn' failed to satisfy constraint: Member must satisfy regular expression pattern: ^arn:(aws|aws-cn|aws-us-gov|aws-iso|aws-iso-b):logs:[a-z\-0-9]*:[0-9]{12}:log-group:([^:\*]*)$
```

Output from acceptance testing (`aws_route53_query_log` failure related to similar issue #13510):

```
--- PASS: TestAccAWSCloudWatchLogGroup_disappears (9.19s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix (13.55s)
--- PASS: TestAccAWSCloudWatchLogGroup_generatedName (13.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_basic (15.24s)
--- PASS: TestAccAWSCloudWatchLogGroup_multiple (15.65s)
--- PASS: TestAccAWSCloudWatchLogGroup_namePrefix_retention (21.29s)
--- PASS: TestAccAWSCloudWatchLogGroup_retentionPolicy (24.99s)
--- PASS: TestAccAWSCloudWatchLogGroup_kmsKey (29.00s)
--- PASS: TestAccAWSCloudWatchLogGroup_tagging (35.60s)

--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings (225.36s)
--- PASS: TestAccAWSAPIGatewayStage_accessLogSettings_kinesis (332.67s)

--- PASS: TestAccAWSAPIGatewayV2Stage_AccessLogSettings (56.73s)

--- PASS: TestAccAWSDataSyncTask_CloudWatchLogGroupARN (304.98s)

--- PASS: TestAccAWSDirectoryServiceLogSubscription_basic (1764.25s)

--- PASS: TestAccAWSElasticSearchDomain_LogPublishingOptions (688.17s)

--- PASS: TestAccAWSFlowLog_LogDestinationType_CloudWatchLogs (26.43s)

--- FAIL: TestAccAWSRoute53QueryLog_Basic (42.80s)
    TestAccAWSRoute53QueryLog_Basic: testing.go:684: Step 0 error: errors during apply:

        Error: Provider produced inconsistent final plan

        When expanding the plan for aws_cloudwatch_log_group.test to include new
        values learned so far during apply, provider "aws" produced an invalid new
        value for .name: was
        cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com"), but
        now cty.StringVal("/aws/route53/testaccawsroute53querylog_basic-rsbvm.com.").

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

--- PASS: TestAccAWSStorageGatewayGateway_CloudWatchLogs (220.06s)
```

* docs/resource/aws_cloudwatch_log_group: Fix typo
@bflad
Copy link
Contributor

bflad commented Jul 30, 2020

Hi folks 👋 The aws_cloudwatch_log_group change mentioned above has been merged and will go out with version 3.0.0 of the Terraform AWS Provider, likely later today or tomorrow. Configurations in the meantime and with earlier versions of the provider can use the replace() function to remove the suffix. 👍

@ghost
Copy link

ghost commented Jul 31, 2020

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@MeMan-MasterOfTheUniverse

That is not the ARN. You're changing a value that doesn't originate within terraform!?

@ghost
Copy link

ghost commented Aug 29, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/cloudwatch Issues and PRs that pertain to the cloudwatch service. service/datasync Issues and PRs that pertain to the datasync service.
Projects
None yet
4 participants