Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wait for import to complete during dynamodb table creation #34848

Merged
merged 5 commits into from
Dec 15, 2023

Conversation

hsiam261
Copy link
Contributor

@hsiam261 hsiam261 commented Dec 10, 2023

Description

Terraform offers a way to create a dynamodb and initialize it with data from an s3 prefix.
This is done using the import_table option. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dynamodb_table#import_table

However, the provider doesn't validate if the import actually succeeded.
Even when the import fails, the table creation shows to be successful.

This PR resolves the issue by waiting for dynamodb import to become COMPLETE before moving on and if it fails, will throw an exception.

Relations

Closes #34847

References

Output from Acceptance Testing

Tried running make testacc TESTS=dynamo
It ran for a while with no bugs, and then wsl crashed :(

Other Tests

Applied the plan from the config mentioned in #34847 and it failed with an error (expected behavior)

Config File:

resource "aws_dynamodb_table" "siam-test-table" {
  name                        = "siam-test-table"
  billing_mode                = "PAY_PER_REQUEST"
  hash_key                    = "key"
  read_capacity               = 0
  write_capacity              = 0
  stream_enabled              = false
  tags_all                    = {}
  deletion_protection_enabled = false

  attribute {
    name = "key"
    type = "S"
  }


  import_table {
    input_compression_type = "GZIP"
    input_format           = "DYNAMODB_JSON"
    s3_bucket_source {
      bucket       = "<BUCKET WITH ONE FILE NOT OF THE GZIP FORMAT>"
      bucket_owner = "<ACCOUNT NUMBER>"
      key_prefix   = ""
    }
  }
}

Console Log

terraform apply plan.out
╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - hashicorp/aws in /home/hsiam261/go/bin
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with
│ published releases.
╵
aws_dynamodb_table.siam-test-table: Creating...
aws_dynamodb_table.siam-test-table: Still creating... [10s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [20s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [30s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [40s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [50s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [1m0s elapsed]
aws_dynamodb_table.siam-test-table: Still creating... [1m10s elapsed]
╷
│ Error: creating Amazon DynamoDB Table: ImportArn "arn:aws:dynamodb:us-east-1:XXXXXXXXXXXX:table/siam-test-table/import/01702410536894-304cc257" : unexpected state 'FAILED', wanted target 'COMPLETED'. last error: %!s(<nil>)
│ 
│   with aws_dynamodb_table.siam-test-table,
│   on main.tf line 1, in resource "aws_dynamodb_table" "siam-test-table":
│    1: resource "aws_dynamodb_table" "siam-test-table" {
│ 
╵

if import fails i.e if import doesn't lead to completed status with
timeout period or fails due to some error before that, we will throw an
exception.
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/S Managed by automation to categorize the size of a PR. service/dynamodb Issues and PRs that pertain to the dynamodb service. labels Dec 10, 2023
@terraform-aws-provider terraform-aws-provider bot added the needs-triage Waiting for first response or review from a maintainer. label Dec 10, 2023
@hsiam261 hsiam261 changed the title Wait for import to become complete during dynamodb table creation Wait for import to complete during dynamodb table creation Dec 11, 2023
@johnsonaj johnsonaj self-assigned this Dec 11, 2023
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Dec 11, 2023
@justinretzolk justinretzolk added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 12, 2023
Otherwise the table would get created but state won't manage it which
would then require manual intervention to fix.
Copy link
Contributor

@johnsonaj johnsonaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

% make testacc TESTARGS="-run=estAccDynamoDBTable_importTable\|estAccDynamoDBTable_basic\|estAccDynamoDBTable_disappears" PKG=dynamodb

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/dynamodb/... -v -count 1 -parallel 20  -run=estAccDynamoDBTable_importTable\|estAccDynamoDBTable_basic\|estAccDynamoDBTable_disappears -timeout 360m
=== RUN   TestAccDynamoDBTable_basic
=== PAUSE TestAccDynamoDBTable_basic
=== RUN   TestAccDynamoDBTable_disappears
=== PAUSE TestAccDynamoDBTable_disappears
=== RUN   TestAccDynamoDBTable_importTable
=== PAUSE TestAccDynamoDBTable_importTable
=== CONT  TestAccDynamoDBTable_basic
=== CONT  TestAccDynamoDBTable_importTable
=== CONT  TestAccDynamoDBTable_disappears
--- PASS: TestAccDynamoDBTable_disappears (35.23s)
--- PASS: TestAccDynamoDBTable_basic (38.93s)
--- PASS: TestAccDynamoDBTable_importTable (165.57s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/dynamodb	169.036s

@johnsonaj johnsonaj merged commit d605b97 into hashicorp:main Dec 15, 2023
29 of 33 checks passed
@github-actions github-actions bot added this to the v5.32.0 milestone Dec 15, 2023
@github-actions github-actions bot removed the bug Addresses a defect in current functionality. label Jan 12, 2024
Copy link

This functionality has been released in v5.32.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@justinretzolk justinretzolk added the bug Addresses a defect in current functionality. label Feb 10, 2024
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. service/dynamodb Issues and PRs that pertain to the dynamodb service. size/S Managed by automation to categorize the size of a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Terraform doesn't throw an error when dynamodb imports fail during creation
3 participants