Support for IOT Rule Action "dynamoDBv2"

[tobiasschaber]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

There is already a dynamodb IOT topic rule ("aws_iot_topic_rule"), but the dynamodb version 2 rule is not yet suported. It would be nice to have that also.

The version 2.0 rule action is called something like "Split message into multiple columns of a database table (DynamoDB Version 2)" and it's available via AWS Management console.

New or Affected Resource(s)

• aws_iot_topic_rule, dynamodb object

Potential Terraform Configuration

table_name: my_target_table
iam_role: my_iam_role

References

• https://docs.aws.amazon.com/iot/latest/developerguide/iot-rule-actions.html
  See "dynamoDBv2" rule

[flmag]

Hi,

Also need dynamodbv2 for some projects, any updates ?

Regards,
Florian

[gmarkey]

I've hacked together support for this as I needed it in the meantime. I do not recommend using this patch if you want your statefile to be valid once support for DDBv2 rules is properly added.

https://gist.github.com/gmarkey/1fe21a6d16b35d8b73c2600b73684f74

[abdelhakim-atmani]

Hi,

Any update ?

[legokichi]

here is my workaround

locals {
    prefix = "test-iot-action-dynamodb2"
    topic = "${local.prefix}/statuslog"
    rule = "${replace("${local.prefix}-rule", "-", "_")}"
}

resource "aws_dynamodb_table" "dynamodb_table" {
    name           = "${local.prefix}-table"

    read_capacity  = 2
    write_capacity = 2
    hash_key       = "DeviceId"
    range_key      = "TimeStamp"

    attribute = [{
        name = "DeviceId"
        type = "S"
    }, {
        name = "TimeStamp"
        type = "N"
    }]
}

resource "aws_iam_role" "iot_rule" {
    name = "${local.prefix}-iot_rule"
    assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "iot.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
EOF
}

data "template_file" "iot_rule" {
    template = <<EOF
{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Action": "dynamodb:PutItem",
        "Resource": "$${table_arn}"
    }
}
EOF
    vars {
        table_arn = "${aws_dynamodb_table.dynamodb_table.arn}"
    }
}

resource "aws_iam_policy" "iot_rule" {
    name = "${local.prefix}-iot_rule"
    policy = "${data.template_file.iot_rule.rendered}"
}

resource "aws_iam_role_policy_attachment" "iot_rule" {
    role = "${aws_iam_role.iot_rule.name}"
    policy_arn = "${aws_iam_policy.iot_rule.arn}"
}

# resource "aws_iot_topic_rule" "iot_rule" {
#     # MUST be ^[a-zA-Z0-9_]+$
#     name = "${replace("${local.rule}", "-", "_")}"
#     sql = "SELECT * FROM '${local.topic}'"
#     sql_version = "2016-03-23"

#     dynamodbv2 {
#         role_arn = "${aws_iam_role.iot_rule.arn}"
#         table_name = "${aws_dynamodb_table.dynamodb_table.arn}"
#     }
# }


data "template_file" "iot_rule_patch" {
    template = <<EOF
{
    "sql": "SELECT * FROM '${local.topic}'",
    "awsIotSqlVersion": "2016-03-23",
    "actions": [
        {
            "dynamoDBv2": {
                "roleArn": "$${role_arn}",
                "putItem": {
                    "tableName": "$${table_name}"
                }
            }
        }
    ]
}
EOF
    vars {
        role_arn = "${aws_iam_role.iot_rule.arn}"
        table_name = "${aws_dynamodb_table.dynamodb_table.arn}"
    }
}

resource "null_resource" "iot_rule_patch" {
    provisioner "local-exec" {
        command = "(aws iot delete-topic-rule --rule-name ${local.rule} || : ) && aws iot create-topic-rule --rule-name ${local.rule} --topic-rule-payload '${data.template_file.iot_rule_patch.rendered}'"
    }
    depends_on = [
        "aws_iam_role_policy_attachment.iot_rule"
    ]
}

[Martinnew]

Thanks for your workaround!
To make it work I had to give the dynamodb_table name, instead the arn to the template_file.

Now I ran into another problem: I simply cannot use the '+' wildcard character within the sql statement:
'An error occurred (SqlParseException) when calling the CreateTopicRule operation: Unexpected token'

The solution for me was to not load the template_file directly into the cli call, but store it in a local_file instead. When calling create-topic-rule I then reference the locally stored file.
I really don't get why this would make a difference. If you have any ideas, please let me know.

Following is my code working with wildcard characters within the sql statement:

data "template_file" "iot_rule_patch" {
  template = <<EOF
{
    "sql": "SELECT CONCAT(\"(\",topic(3),\":\",topic(5),\")\") AS idTuple, timestamp, temperature, humidity, pressure, batteryLevel FROM '$${iot_rule_data_topic}/Gateways/+/Sensors/+/data/'",
    "awsIotSqlVersion": "2016-03-23",
    "actions": [
        {
            "dynamoDBv2": {
                "roleArn": "$${role_arn}",
                "putItem": {
                    "tableName": "$${table_name}"
                }
            }
        }
    ]
}
EOF
  vars {
    role_arn = "${aws_iam_role.iot_rule_role.arn}"
    table_name = "${aws_dynamodb_table.table_raw_data.name}"
    iot_rule_data_topic = "${local.service_name}"
  }
}

resource "local_file" "iot_rule_payload" {
  filename = "./iot_rule_payload"
  content = "${data.template_file.iot_rule_patch.rendered}"
}


resource "null_resource" "iot_rule_patch" {
  provisioner "local-exec" {
    command = "aws iot delete-topic-rule --rule-name ${local.iot_rule_name} || :"
  }
  provisioner "local-exec" {
    command = "aws iot create-topic-rule --rule-name ${local.iot_rule_name} --topic-rule-payload file://iot_rule_payload"
  }
  depends_on = [
    "aws_iam_role_policy.iot_rule_role_policy"
  ]

}

[bflad]

Support for this functionality has been merged and will release with version 2.62.0 of the Terraform AWS Provider, later this week. Thanks to @edwardbrowncross for the implementation. 👍

[ghost]

This has been released in version 2.62.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!