RabbitMQ Production Readiness

We are firm believers in Murphy's law. Here's a checklist to ensure your RabbitMQ runs flawlessly in production:

• Tune the RabbitMQ Broker opts carefully
  • Refer to all the settings in the RabbitMQ Message Broker wiki
  • In :settings, :automatically-recover is set to true by default. Never set it to false
  • Use a RabbitMQ cluster for higher resiliency & reliability
  • When using cluster, replicate messages by using quorum queues via :queue-type config
    • Ensure if replication-factor isn't greater than number of nodes, else it'll be set to cluster size
  • When initializing Producer, configure the channel-count based on expected enqueue-rate
    • Goose defaults to 5 channels, if not configured
• Use multiple queues for different functions to improve CPU utilisation of nodes
  • A single RabbitMQ queue is bounded to a single core
  • We don't recommend a different queue per function, just enough to utilise all CPUs & balance load amongst them
• When consuming messages, Goose uses acknowledgement timeouts to reliably execute a Job
  • If a Job continues running beyond the timeout, RabbitMQ redelivers the Job to a different thread for execution
  • Set acknowledgement timeout to 2x of p99 Job execution times to avoid double executions
• Tune system parameters
  • File limits
  • Disk space limits
  • Memory management
  • Enable TCP Keepalives. Guide on how-to enable them
    • Some proxies may terminate "idle" TCP connections
    • In :settings, set :requested-heartbeat timeout so client sends heartbeat at about 1/2 the interval
    • Alternatively, you can set heartbeat timeout on server side as well
• Use RabbitMQ's Management Plugin
• Refer to RabbitMQ's official Production checklist, utilization, monitoring & planning for upgrades guides once

---

Previous: Production Readiness        Next: Redis Production Readiness