Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDB item creation using a aws_dynamodb_table_item resource #517

Closed
hashibot opened this issue Jun 13, 2017 · 15 comments · Fixed by #3238
Closed

DynamoDB item creation using a aws_dynamodb_table_item resource #517

hashibot opened this issue Jun 13, 2017 · 15 comments · Fixed by #3238
Labels
new-resource Introduces a new resource. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Milestone

Comments

@hashibot
Copy link

This issue was originally opened by @drieselliott as hashicorp/terraform#11773. It was migrated here as part of the provider split. The original body of the issue is below.


Hi,

It would be nice to have a 'aws_dynamodb_table_item' resource that allows you to write configuration entries to DynamoDB tables when provisioning an environment. It should support both the DynamoDB JSON and the normal JSON format.

resource "aws_dynamodb_table_item" "configuration_item" {
    table = "${aws_dynamodb_table.configuration_table.name}"
    format = "json"
    item = <<EOF
       //Some JSON content
EOF
}

table: The table name
format: Either 'json' or 'dynamodb' for 'normal json' or 'dynamodb json'
item: The item to insert into the table (either as inline content or template)

This is very useful in serverless applications that use DynamoDb as their configuration store and Terraform to provision everything. Otherwise you need to rely on a separate non-terraform script to upload your configuration entries.

Thanks for considering this.

Dries

@hashibot hashibot added the enhancement Requests to existing resources that expand the functionality or scope. label Jun 13, 2017
@galadriel2143
Copy link
Contributor

galadriel2143 commented Aug 10, 2017

Started a PR to implement this feature. Functionality is complete, but needs tests (not sure if someone can help with this but would be appreciated) and there are a few questions I would have before it gets merged back in.

  • Behavior on update: Should it force attributes back to the ones specified in the resource definition of they don't match? I think this would be fine based on how terraform handles attributes on other resources.
  • Behavior on externally removed item: Recreate, leave alone, or hard fail? Recreating when it's missing seems to fall in line with other resources.
  • Output values: Currently I'm returning the range attribute value, hash attribute value, and the "query key", which contains both and can be used in a DDB query to get the object if necessary. Is there anything else that should be returned, or maybe we shouldn't return these at all...
  • Query setup: I have the range_key and hash_key names specified on the resource. These probably could be determined by querying the table metadata, but that would add more HTTP calls, and specifying them on the resource theoretically offers some protection from the key schema changing in an unexpected way.
  • Value comparison: Doing a deep equal on the item returned from DynamoDB (type map[string]*dynamodb.AttributeValue). It seems to work correctly, but are there any potential problems with this? I'm only projecting the attributes that exist in the item definition, so that way any additional attributes don't constantly trigger false positives.

Example:

resource "aws_dynamodb_table_item" "configuration_item" {
    table_name = "testy"
    hash_key = "hello"
    range_key = "getsorty"
    item = <<EOF
{
    "hello": { "S": "helloworld"},
    "getsorty": { "N": "100" },
    "othervalue": { "N": "900" },
}
EOF
}

output "query_key" {
    value = "${aws_dynamodb_table_item.configuration_item.query_key}"
}

output "table_name" {
    value = "${aws_dynamodb_table_item.configuration_item.table_name}"
}

Computed outputs:

    range_value = "{\"N\": \"100\" }"
    hash_value = "{\"S\": \"helloworld\" }"
    query_key = <<EOF
    {
        "hello": { "S": "helloworld" },
        "getsorty": {"N": "100" }
    }
    EOF

One possible usage of outputs:

terraform output query_key | xargs -d'\n' aws dynamodb --table-name "$(terraform output table_name)" --key

@radeksimko radeksimko added new-resource Introduces a new resource. and removed enhancement Requests to existing resources that expand the functionality or scope. labels Aug 10, 2017
@galadriel2143
Copy link
Contributor

galadriel2143 commented Aug 15, 2017

@drieselliott @radeksimko Do you have any comments on this?

galadriel2143 added a commit to galadriel2143/terraform-provider-aws that referenced this issue Aug 17, 2017
@jantman
Copy link

jantman commented Oct 20, 2017

Is there any movement on this? This would be an extremely useful feature for us...

@ccgagnon
Copy link

+1 It would be very useful feature for us too.

@galadriel2143
Copy link
Contributor

@jantman @ccagnon Are either of you familiar with Terraform's test suite, and/or want to take a crack at it? Do you see anything that needs to be improved with my code? Please let me know.

@galadriel2143
Copy link
Contributor

@ccgagnon

@galadriel2143
Copy link
Contributor

Our team is very eager to have this merged in as well. Currently we're using the resources from a separate provider plugin.

@jantman
Copy link

jantman commented Oct 23, 2017

Unfortunately I'm not familiar with Terraform's test suite, or even Go in general...

@galadriel2143
Copy link
Contributor

@jantman I wasn't very familiar with Go either before I wrote this. Gogland is a pretty useful IDE if you need some assistance. I wouldn't recommend developing on Windows though.

@ccgagnon
Copy link

@galadriel2143 I am not familiar with Terraform's test suite, but I know go. I can take a look and try it.

@nikhil-p
Copy link

Any update on this? Need this feature soon.

@queeno
Copy link

queeno commented Dec 14, 2017

I would like to get this feature in terraform asap as well

@tskotov
Copy link

tskotov commented Dec 19, 2017

+1 for this resource.

@radeksimko radeksimko added the service/dynamodb Issues and PRs that pertain to the dynamodb service. label Jan 22, 2018
@bflad bflad added this to the v1.9.0 milestone Feb 9, 2018
@bflad
Copy link
Contributor

bflad commented Feb 9, 2018

This has been released in terraform-provider-aws version 1.9.0. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

jocgir added a commit to coveord/terraform-provider-aws that referenced this issue Feb 12, 2018
* commit '5293a0e3b1366ee16d8742b9b2354781a79bfbd9': (224 commits)
  v1.9.0
  Update CHANGELOG for hashicorp#1101 and hashicorp#3283
  docs/resource/aws_sns_platform_application: Add note about platform_credential and platform_principal hashing
  resource/aws_sns_platform_application: Refactor ID parsing to its own function, use testing.T in testAccAwsSnsPlatformApplicationPlatformFromEnv
  Add lambda example (hashicorp#3168)
  Update CHANGELOG for hashicorp#3157
  docs/data-source/aws_region: Remove now deprecated current argument
  data-source/aws_region: Refactor logic into findRegionByEc2Endpoint and findRegionByName functions
  Update CHANGELOG for hashicorp#3301
  Update CHANGELOG for hashicorp#2559 and hashicorp#3240
  Update CHANGELOG.md
  resource/aws_kinesis_stream: Retry deletion on LimitExceededException (hashicorp#3108)
  Update CHANGELOG.md
  resource/aws_dynamodb_table_item: Cleanup + add missing bits
  Added dynamodb_table_item resource hashicorp#517
  Update CHANGELOG.md
  New Resource: aws_cloud9_environment_ec2
  Update CHANGELOG.md
  Fixed markdown typo in docs
  resource/aws_kinesis_firehose_delivery_stream: Prevent crashes on empty CloudWatchLoggingOptions and fix extended_s3_configuration kms_key_arn
  ...

# Conflicts:
#	aws/validators.go
jocgir added a commit to coveord/terraform-provider-aws that referenced this issue Feb 12, 2018
…parameters-features

* commit '5293a0e3b1366ee16d8742b9b2354781a79bfbd9': (752 commits)
  v1.9.0
  Update CHANGELOG for hashicorp#1101 and hashicorp#3283
  docs/resource/aws_sns_platform_application: Add note about platform_credential and platform_principal hashing
  resource/aws_sns_platform_application: Refactor ID parsing to its own function, use testing.T in testAccAwsSnsPlatformApplicationPlatformFromEnv
  Add lambda example (hashicorp#3168)
  Update CHANGELOG for hashicorp#3157
  docs/data-source/aws_region: Remove now deprecated current argument
  data-source/aws_region: Refactor logic into findRegionByEc2Endpoint and findRegionByName functions
  Update CHANGELOG for hashicorp#3301
  Update CHANGELOG for hashicorp#2559 and hashicorp#3240
  Update CHANGELOG.md
  resource/aws_kinesis_stream: Retry deletion on LimitExceededException (hashicorp#3108)
  Update CHANGELOG.md
  resource/aws_dynamodb_table_item: Cleanup + add missing bits
  Added dynamodb_table_item resource hashicorp#517
  Update CHANGELOG.md
  New Resource: aws_cloud9_environment_ec2
  Update CHANGELOG.md
  Fixed markdown typo in docs
  resource/aws_kinesis_firehose_delivery_stream: Prevent crashes on empty CloudWatchLoggingOptions and fix extended_s3_configuration kms_key_arn
  ...

# Conflicts:
#	aws/resource_aws_ssm_parameter_test.go
@ghost
Copy link

ghost commented Nov 7, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Nov 7, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
new-resource Introduces a new resource. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants