CloudWatch Logs Delivery v2 #40731

ewbankkit · 2024-12-30T20:02:04Z

Description

New resources to support CloudWatch Logs Delivery v2.

Relations

Output from Acceptance Testing

% make testacc TESTARGS='-run=TestAccLogs_serial/^DeliverySource$$' PKG=logs
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogs_serial/^DeliverySource$ -timeout 360m
2024/12/31 10:55:03 Initializing Terraform AWS Provider...
=== RUN   TestAccLogs_serial
=== PAUSE TestAccLogs_serial
=== CONT  TestAccLogs_serial
=== RUN   TestAccLogs_serial/DeliverySource
=== RUN   TestAccLogs_serial/DeliverySource/basic
=== RUN   TestAccLogs_serial/DeliverySource/disappears
=== RUN   TestAccLogs_serial/DeliverySource/tags
--- PASS: TestAccLogs_serial (4493.96s)
    --- PASS: TestAccLogs_serial/DeliverySource (4493.96s)
        --- PASS: TestAccLogs_serial/DeliverySource/basic (1533.65s)
        --- PASS: TestAccLogs_serial/DeliverySource/disappears (1495.98s)
        --- PASS: TestAccLogs_serial/DeliverySource/tags (1464.33s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	4499.269s

% make testacc TESTARGS='-run=TestAccLogsDeliveryDestination_' PKG=logs ACCTEST_PARALLELISM=3
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 3  -run=TestAccLogsDeliveryDestination_ -timeout 360m
2024/12/31 14:47:41 Initializing Terraform AWS Provider...
=== RUN   TestAccLogsDeliveryDestination_basic
=== PAUSE TestAccLogsDeliveryDestination_basic
=== RUN   TestAccLogsDeliveryDestination_disappears
=== PAUSE TestAccLogsDeliveryDestination_disappears
=== RUN   TestAccLogsDeliveryDestination_tags
=== PAUSE TestAccLogsDeliveryDestination_tags
=== RUN   TestAccLogsDeliveryDestination_outputFormat
=== PAUSE TestAccLogsDeliveryDestination_outputFormat
=== RUN   TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationSameType
=== PAUSE TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationSameType
=== RUN   TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationDifferentType
=== PAUSE TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationDifferentType
=== CONT  TestAccLogsDeliveryDestination_basic
=== CONT  TestAccLogsDeliveryDestination_outputFormat
=== CONT  TestAccLogsDeliveryDestination_tags
--- PASS: TestAccLogsDeliveryDestination_basic (15.36s)
=== CONT  TestAccLogsDeliveryDestination_disappears
--- PASS: TestAccLogsDeliveryDestination_disappears (11.45s)
=== CONT  TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationDifferentType
--- PASS: TestAccLogsDeliveryDestination_outputFormat (31.01s)
=== CONT  TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationSameType
--- PASS: TestAccLogsDeliveryDestination_tags (33.50s)
--- PASS: TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationDifferentType (29.78s)
--- PASS: TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationSameType (28.83s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	65.136s

% make testacc TESTARGS='-run=TestAccLogsDeliveryDestinationPolicy_' PKG=logs ACCTEST_PARALLELISM=3 
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 3  -run=TestAccLogsDeliveryDestinationPolicy_ -timeout 360m
2024/12/31 15:42:03 Initializing Terraform AWS Provider...
=== RUN   TestAccLogsDeliveryDestinationPolicy_basic
=== PAUSE TestAccLogsDeliveryDestinationPolicy_basic
=== RUN   TestAccLogsDeliveryDestinationPolicy_disappears
=== PAUSE TestAccLogsDeliveryDestinationPolicy_disappears
=== CONT  TestAccLogsDeliveryDestinationPolicy_basic
=== CONT  TestAccLogsDeliveryDestinationPolicy_disappears
--- PASS: TestAccLogsDeliveryDestinationPolicy_disappears (13.22s)
--- PASS: TestAccLogsDeliveryDestinationPolicy_basic (15.03s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	20.331s

% make testacc TESTARGS='-run=TestAccLogs_serial/^Delivery$$/' PKG=logs
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogs_serial/^Delivery$/ -timeout 360m
2025/01/06 06:17:45 Initializing Terraform AWS Provider...
=== RUN   TestAccLogs_serial
=== PAUSE TestAccLogs_serial
=== CONT  TestAccLogs_serial
=== RUN   TestAccLogs_serial/Delivery
=== RUN   TestAccLogs_serial/Delivery/update
=== RUN   TestAccLogs_serial/Delivery/basic
=== RUN   TestAccLogs_serial/Delivery/disappears
=== RUN   TestAccLogs_serial/Delivery/tags
--- PASS: TestAccLogs_serial (4553.37s)
    --- PASS: TestAccLogs_serial/Delivery (4553.37s)
        --- PASS: TestAccLogs_serial/Delivery/update (1407.49s)
        --- PASS: TestAccLogs_serial/Delivery/basic (1401.92s)
        --- PASS: TestAccLogs_serial/Delivery/disappears (1440.87s)
        --- PASS: TestAccLogs_serial/Delivery/tags (1485.34s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	4558.615s

github-actions · 2024-12-30T20:02:19Z

Community Note

Voting for Prioritization

Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

% make testacc TESTARGS='-run=TestAccLogsDeliverySource_basic' PKG=logs make: Verifying source code with gofmt... ==> Checking that code complies with gofmt requirements... TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20 -run=TestAccLogsDeliverySource_basic -timeout 360m 2024/12/30 17:43:31 Initializing Terraform AWS Provider... === RUN TestAccLogsDeliverySource_basic === PAUSE TestAccLogsDeliverySource_basic === CONT TestAccLogsDeliverySource_basic --- PASS: TestAccLogsDeliverySource_basic (1229.44s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/logs 1234.724s

…re ForceNew.

… is ForceNew if the type changes.

…: 0 or 2; Actual: 1]'.

…rvice'.

% make testacc TESTARGS='-run=TestAccLogs_serial/^DeliverySource$$/tags' PKG=logs make: Verifying source code with gofmt... ==> Checking that code complies with gofmt requirements... TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20 -run=TestAccLogs_serial/^DeliverySource$/tags -timeout 360m 2025/01/07 08:39:51 Initializing Terraform AWS Provider... === RUN TestAccLogs_serial === PAUSE TestAccLogs_serial === CONT TestAccLogs_serial === RUN TestAccLogs_serial/DeliverySource === RUN TestAccLogs_serial/DeliverySource/tags --- PASS: TestAccLogs_serial (1444.05s) --- PASS: TestAccLogs_serial/DeliverySource (1444.05s) --- PASS: TestAccLogs_serial/DeliverySource/tags (1444.05s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/logs 1449.433s

ewbankkit · 2025-01-07T14:05:20Z

=== RUN   TestAccLogs_serial/DeliverySource/tags
    delivery_source_test.go:125: Step 3/4 error: Error running apply: exit status 1

        Error: Provider returned invalid result object after apply

        After the apply operation, the provider still indicated an unknown value for
        aws_cloudwatch_log_delivery_source.test.service. All values must be known
        after apply, so this is always a bug in the provider and should be reported
        in the provider's own repository. Terraform will still save the other known
        object values in the state.

fixed via dd1ad7c:

% make testacc TESTARGS='-run=TestAccLogs_serial/^DeliverySource$$/tags' PKG=logs
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogs_serial/^DeliverySource$/tags -timeout 360m
2025/01/07 08:39:51 Initializing Terraform AWS Provider...
=== RUN   TestAccLogs_serial
=== PAUSE TestAccLogs_serial
=== CONT  TestAccLogs_serial
=== RUN   TestAccLogs_serial/DeliverySource
=== RUN   TestAccLogs_serial/DeliverySource/tags
--- PASS: TestAccLogs_serial (1444.05s)
    --- PASS: TestAccLogs_serial/DeliverySource (1444.05s)
        --- PASS: TestAccLogs_serial/DeliverySource/tags (1444.05s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	1449.433s

jar-b

LGTM 🎉

% make testacc PKG=logs TESTARGS='-run=TestAccLogsDeliveryDestination_'
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogsDeliveryDestination_ -timeout 360m
2025/01/06 16:38:45 Initializing Terraform AWS Provider...

--- PASS: TestAccLogsDeliveryDestination_disappears (14.80s)
--- PASS: TestAccLogsDeliveryDestination_basic (18.73s)
--- PASS: TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationSameType (32.51s)
--- PASS: TestAccLogsDeliveryDestination_updateDeliveryDestinationConfigurationDifferentType (34.63s)
--- PASS: TestAccLogsDeliveryDestination_outputFormat (35.34s)
--- PASS: TestAccLogsDeliveryDestination_tags (39.56s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/logs       46.141s

% make testacc PKG=logs TESTARGS='-run=TestAccLogsDeliveryDestinationPolicy_'
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogsDeliveryDestinationPolicy_ -timeout 360m
2025/01/06 16:40:04 Initializing Terraform AWS Provider...

--- PASS: TestAccLogsDeliveryDestinationPolicy_disappears (14.75s)
--- PASS: TestAccLogsDeliveryDestinationPolicy_basic (16.79s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/logs       23.215s

%  make testacc PKG=logs TESTARGS='-run=TestAccLogs_serial/^DeliverySource$$'
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogs_serial/^DeliverySource$ -timeout 360m
2025/01/07 09:17:58 Initializing Terraform AWS Provider...

--- PASS: TestAccLogs_serial (4405.15s)
    --- PASS: TestAccLogs_serial/DeliverySource (4405.15s)
        --- PASS: TestAccLogs_serial/DeliverySource/basic (1509.33s)
        --- PASS: TestAccLogs_serial/DeliverySource/disappears (1443.28s)
        --- PASS: TestAccLogs_serial/DeliverySource/tags (1452.54s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/logs       4410.689s

% make testacc PKG=logs TESTARGS='-run=TestAccLogs_serial/^Delivery$$'
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.3 test ./internal/service/logs/... -v -count 1 -parallel 20  -run=TestAccLogs_serial/^Delivery$ -timeout 360m
2025/01/07 10:38:09 Initializing Terraform AWS Provider...

--- PASS: TestAccLogs_serial (5820.96s)
    --- PASS: TestAccLogs_serial/Delivery (5820.96s)
        --- PASS: TestAccLogs_serial/Delivery/basic (1407.35s)
        --- PASS: TestAccLogs_serial/Delivery/disappears (1463.59s)
        --- PASS: TestAccLogs_serial/Delivery/tags (1535.33s)
        --- PASS: TestAccLogs_serial/Delivery/update (1414.68s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/logs       5826.167s

internal/framework/resource_list_of_object.go

internal/service/logs/delivery_destination.go

internal/service/logs/delivery.go

internal/service/logs/delivery_source.go

kbachuHighSpot · 2025-01-08T23:12:49Z

This is great. I was searching up for it and landed here :-)

Any estimate on when this is planned to be released for all of us to use this feature?

brianhelba · 2025-01-09T00:26:35Z

@ewbankkit It looks like the aws_cloudwatch_log_delivery_source resource here does not claim to support the log_type of ACCESS_LOGS, which is necessary to support CloudFront as a delivery source.

Is using an argument log_type = "ACCESS_LOGS" possible as an undocumented feature, or will more work be required for this (and CloudFront) to be supported?

github-actions · 2025-01-09T18:09:11Z

This functionality has been released in v5.83.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

kbachuHighSpot · 2025-01-09T18:22:47Z

@ewbankkit It looks like the aws_cloudwatch_log_delivery_source resource here does not claim to support the log_type of ACCESS_LOGS, which is necessary to support CloudFront as a delivery source.

Is using an argument log_type = "ACCESS_LOGS" possible as an undocumented feature, or will more work be required for this (and CloudFront) to be supported?

Oh! I was going to ask the same question. I was hoping to use this CloudFront logs to be sent to CloudWatch. Is it not possible with https://github.com/hashicorp/terraform-provider-aws/blob/v5.83.0/CHANGELOG.md

scottharris · 2025-01-09T21:56:22Z

@ewbankkit It looks like the aws_cloudwatch_log_delivery_source resource here does not claim to support the log_type of ACCESS_LOGS, which is necessary to support CloudFront as a delivery source.
Is using an argument log_type = "ACCESS_LOGS" possible as an undocumented feature, or will more work be required for this (and CloudFront) to be supported?

Oh! I was going to ask the same question. I was hoping to use this CloudFront logs to be sent to CloudWatch. Is it not possible with https://github.com/hashicorp/terraform-provider-aws/blob/v5.83.0/CHANGELOG.md

You can set log_type to "ACCESS_LOGS" without issue, i guess this just needs a doco update.

resource "aws_cloudwatch_log_delivery_source" "example" {
  name         = "name"
  log_type     = "ACCESS_LOGS"
  resource_arn = "arn:aws:cloudfront::123456789012:distribution/123456789012"
}

There is an issue with aws_cloudwatch_log_delivery though

resource "aws_cloudwatch_log_delivery" "example" {
  delivery_source_name     = aws_cloudwatch_log_delivery_source.example.name
  delivery_destination_arn = "arn:aws:logs:us-east-1:123456789012:delivery-destination:mydestination"
}

results in error

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to aws_cloudwatch_log_delivery.example, provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: .field_delimiter: was null, but now cty.StringVal("\t").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Tried setting field_delimiter to "\t" but resulted in error also
field_delimiter = "\t"

╷
│ Error: creating CloudWatch Logs Delivery
│ 
│   with aws_cloudwatch_log_delivery.example,
│   on provider.tf line 16, in resource "aws_cloudwatch_log_delivery" "example":
│   16: resource "aws_cloudwatch_log_delivery" "example" {
│ 
│ operation error CloudWatch Logs: CreateDelivery, https response error StatusCode: 400, RequestID: 8182e943-4bc3-43c3-b271-110bcd925887, ValidationException: Provided field delimiter is not applicable for delivery destination's output
│ format
╵

I'm currently using awscc for this and it works

resource "awscc_logs_delivery" "example" {
  delivery_source_name     = aws_cloudwatch_log_delivery_source.example.name
  delivery_destination_arn = "arn:aws:logs:us-east-1:123456789012:delivery-destination:mydestination"

  tags = [ 
    {
      key   = "Name"
      value = "blah"
    }
  ]
}

ewbankkit · 2025-01-10T13:22:41Z

@scottharris @kbachuHighSpot Please open a new GitHub issue to report problems.

ewbankkit added 2 commits December 30, 2024 14:33

r/aws_cloudwatch_log_delivery_source: New resource.

eb9e135

r/aws_cloudwatch_log_delivery_source: Start to add acceptance tests.

a8543d0

ewbankkit changed the title ~~CloudWatch Logs Delivery~~ CloudWatch Logs Delivery v2 Dec 30, 2024

ewbankkit added 12 commits December 30, 2024 15:02

Correct CHANGELOG entry file name.

258a4b4

acctest: Split common configurations into their own source file.

5add296

Add 'acctest.ConfigBedrockAgentKnowledgeBaseRDSBase'.

0b175cf

Get 'TestAccLogsDeliverySource_basic' working.

ce97ad8

Add 'testAccDeliverySource_disappears'.

6866498

Add 'testAccDeliverySource_tags'.

7e5a18b

r/aws_cloudwatch_log_delivery_source: Don't forget 'ModifyPlan'.

132ac97

Delivery Source tags aren't set in the Get response.

29a2a52

r/aws_cloudwatch_log_delivery_destination: New resource.

f2a5f23

Add 'TestAccLogsDeliveryDestination_basic'.

671dd2f

r/aws_cloudwatch_log_delivery_destination: Additional acceptance tests.

a49dabf

ewbankkit added the new-resource Introduces a new resource. label Dec 31, 2024

github-actions bot added the service/bedrockagent Issues and PRs that pertain to the bedrockagent service. label Dec 31, 2024

ewbankkit added 7 commits December 31, 2024 12:15

Run 'make gen'.

9c570c3

Use 'ParallelTest'.

8fa80db

Delivery Destination tags aren't set in the Get response.

57d4508

r/aws_cloudwatch_log_delivery_destination: 'output_format' is ForceNew.

fbc6282

r/aws_cloudwatch_log_delivery_source: 'log_type' and 'resource_arn' a…

b7da267

…re ForceNew.

r/aws_cloudwatch_log_delivery_destination: 'destination_resource_arn'…

ec90ee3

… is ForceNew if the type changes.

r/aws_cloudwatch_log_delivery_destination_policy: New resource.

66df71e

ewbankkit added 7 commits January 5, 2025 16:18

Merge branch 'main' into f-CloudWatch-Logs-Delivery

ecceb1f

r/aws_cloudwatch_log_delivery: New resource.

1bdd500

Add 'framework.ResourceOptionalComputedListOfObjectAttribute'.

8a6f4f2

r/aws_cloudwatch_log_delivery: Add acceptance tests.

1e67a54

'record_fields' and 's3_delivery_configuration' are Optional+Computed.

0129275

Fix markdown-lint 'MD009/no-trailing-spaces Trailing spaces [Expected…

9852cb4

…: 0 or 2; Actual: 1]'.

Normalize FieldDelimiter on Read.

c4c1491

ixti mentioned this pull request Jan 6, 2025

Feature Request: Cloudfront Access Logs v2 support pulumi/pulumi-aws#5032

Closed

ewbankkit added 3 commits January 6, 2025 06:16

Add 'testAccDelivery_update'.

6a90f12

Correct 'record_fields' for 'testAccDelivery_update'.

cd9c108

Fix 'testAccDelivery_update'.

4b048e3

ewbankkit marked this pull request as ready for review January 6, 2025 17:57

ewbankkit requested a review from a team as a code owner January 6, 2025 17:57

ewbankkit added 2 commits January 7, 2025 08:37

r/aws_cloudwatch_log_delivery_source: Add 'UseStateForUnknown' to 'se…

dd1ad7c

…rvice'.

jar-b approved these changes Jan 7, 2025

View reviewed changes

internal/framework/resource_list_of_object.go Show resolved Hide resolved

internal/service/logs/delivery_destination.go Show resolved Hide resolved

internal/service/logs/delivery.go Show resolved Hide resolved

internal/service/logs/delivery_source.go Show resolved Hide resolved

ewbankkit merged commit d29496c into main Jan 7, 2025
44 checks passed

ewbankkit deleted the f-CloudWatch-Logs-Delivery branch January 7, 2025 20:38

github-actions bot added this to the v5.83.0 milestone Jan 7, 2025

terraform-aws-provider bot pushed a commit that referenced this pull request Jan 7, 2025

Update CHANGELOG.md for #40731

b50bc5b

github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Jan 9, 2025

scottharris mentioned this pull request Jan 12, 2025

[Bug]: Cloudwatch Logs V2 #40885

Open

sterliakov mentioned this pull request Jan 15, 2025

[Enhancement]: CloudFront V2 Logging to S3 #40925

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CloudWatch Logs Delivery v2 #40731

CloudWatch Logs Delivery v2 #40731

ewbankkit commented Dec 30, 2024 •

edited

Loading

github-actions bot commented Dec 30, 2024

ewbankkit commented Jan 7, 2025

jar-b left a comment

kbachuHighSpot commented Jan 8, 2025

brianhelba commented Jan 9, 2025 •

edited

Loading

github-actions bot commented Jan 9, 2025

kbachuHighSpot commented Jan 9, 2025

scottharris commented Jan 9, 2025 •

edited

Loading

ewbankkit commented Jan 10, 2025

CloudWatch Logs Delivery v2 #40731

CloudWatch Logs Delivery v2 #40731

Conversation

ewbankkit commented Dec 30, 2024 • edited Loading

Description

Relations

Output from Acceptance Testing

github-actions bot commented Dec 30, 2024

Community Note

ewbankkit commented Jan 7, 2025

jar-b left a comment

Choose a reason for hiding this comment

kbachuHighSpot commented Jan 8, 2025

brianhelba commented Jan 9, 2025 • edited Loading

github-actions bot commented Jan 9, 2025

kbachuHighSpot commented Jan 9, 2025

scottharris commented Jan 9, 2025 • edited Loading

ewbankkit commented Jan 10, 2025

ewbankkit commented Dec 30, 2024 •

edited

Loading

brianhelba commented Jan 9, 2025 •

edited

Loading

scottharris commented Jan 9, 2025 •

edited

Loading