Outpost event metadata on a queue destination

[leggetter]

Related to #293

Upon receipt of an event on a queue destinations (at time of writing that's RabbitMQ and AWS SQS) there is no way to determine the event topic based on the payload or metadata.

For RabbitMQ, the topic is used as the routing key, so it's possible to configure a RabbitMQ exchange to route events based on the topic/routing key.

However, SQS has no concept of topics, and we do not presently include the topic value anywhere.

For webhooks, we include the topic in a header: {"x-outpost" | CUSTOM_HEADER}-topic. Should we include the topic within the metadata within a similar naming structure for queue destinations by default?

[alexbouchardd]

The default HTTP headers equivalant keys/values are supposed to be present as default metadata

[leggetter]

• Outpost adds three pieces of metadata to all messages
  • event-id (note: when it's a JSON payload, should this be eventId or event_id for more conventional dot-notation access? metadata.event_id vs. metadata["event-id"])
  • timestamp
  • topic
• How metadata is applied will differ based on the destination type.
  • Webhooks:
    • Custom prefix configured at the webhooks-only level.
    • The custom prefix is applied to all metadata, whether it's the three Outpost attributes or metadata on the published event (Note: I think augmenting published metadata names will cause confusion)
  • SQS
    • Outpost will add a metadata message attribute to all messages with a JSON string payload containing the Outpost metadata (note: this is like saying metadata.metadata, so is there a better attribute name?).
    • We'll take this approach because SQS only allows 10 message attributes, and we don't want to use up three (3) of them with Outpost metadata.
    • No prefix will be applied. Instead, we'll document that metadata is reserved.
  • RabbitMQ
    • Outpost metadata will be added as individual headers without prefixes. We'll document that the headers are reserved.