MLPAB-1090 - Stream DB Events to Eventbridge Bus

[andrewpearce-digital]

Purpose

Forward database events for processing

Fixes MLPAB-1090

Approach

• use deletion protection instead of lifecycle rules and prevent prod deletion
• create a dynamodb table for reduced fees
• create an event bus for sending events across accounts
• create an event bridge pipe to send events from dynamodb stream to the event bus
• add rule to forward to a remote event bus

Put an item with the aws cli

aws-vault exec <profile> -- aws dynamodb put-item \
    --table-name <environment-name>-reduced-fees \
    --item '{"PK": {"S": "eff"}}'

Note:
There's a bug with the aws provider causing an error when creating a pipe. hashicorp/terraform-provider-aws#32018

Learning

• https://aws.amazon.com/blogs/database/dynamodb-streams-use-cases-and-design-patterns/
• https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-dynamodb.html
• https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-filtering.html
• https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-bus-perms.html
• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_archive
• https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-filtering.html

The total entry size must be less than 256KB. You can calculate the entry size before you send the events.

• https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-putevent-size.html

[github-actions]

PR Environment Terraform Plan Summary

Plan: 0 to add, 0 to change, 11 to destroy

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (20a6414) 92.58% compared to head (e4b1055) 92.58%.

❗ Current head e4b1055 differs from pull request most recent head c13fc51. Consider uploading reports for the commit c13fc51 to get more accurate results

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #547   +/-   ##
=======================================
  Coverage   92.58%   92.58%           
=======================================
  Files         134      134           
  Lines        6203     6203           
=======================================
  Hits         5743     5743           
  Misses        391      391           
  Partials       69       69           

Flag	Coverage Δ
unittests	92.58% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.