Level 2 · 25 min
RabbitMQ
RabbitMQ is a traditional message broker implementing AMQP. Unlike Kafka's append-only log, RabbitMQ routes messages through exchanges to queues based on binding rules. Messages are typically consumed once and deleted. It excels at task queues, RPC patterns, and complex routing scenarios.
Exchanges and Routing
Producers send messages to exchanges, never directly to queues. Exchange types determine routing: Direct (routes to queues with an exact binding key match — point-to-point, O(1) lookup via hash table), Topic (routing key patterns with `*` for exactly one word and `#` for zero or more words — evaluated against a trie, amortized O(binding count) lookup), Fanout (broadcasts to all bound queues — O(bound queue count), no routing key inspection), Headers (routes by AMQP message headers using x-match: all or any — rarely used, poor performance). The routing key is a dot-delimited string in the message (e.g., `payment.credit.error`); bindings define which patterns an exchange sends to which queue. Exchange durability: a durable exchange survives broker restart but only if the queue is also durable — non-durable queues bound to a durable exchange are gone after restart. Alternate exchange: a fallback exchange for unroutable messages — declare with `alternate-exchange` argument to capture messages that match no bindings instead of silently dropping them.
Queues and Bindings
An e-commerce platform ran 6 nodes with classic mirrored queues for order processing. One node failed during peak traffic. Mirroring sync paused for 45 seconds — during which the surviving mirrors were out of sync. On failover, 3,400 messages delivered to the failed node were lost because they hadn't yet been mirrored. Root cause: classic mirrored queues use asynchronous mirror sync that can lag. Fix: migrate to quorum queues (RabbitMQ 3.8+). Quorum queues use the Raft consensus protocol — a message is not confirmed to the producer until a quorum (majority) of nodes has durably written it to their WAL. No async lag, no data loss on failover. Quorum queues require `x-queue-type: quorum` and `durable: true`. They do not support per-message TTL, priority queues, or lazy mode. Stream queues (RabbitMQ 3.9+): append-only log model similar to Kafka — multiple consumers at different offsets, long retention, messages not deleted on consume. Use stream queues when you need fan-out to independent consumers or replayability. Queue arguments: `x-message-ttl` (per-message TTL in ms, classic queues only), `x-max-length` (cap by message count), `x-max-length-bytes`, `x-dead-letter-exchange` (DLX for rejected/expired/overflow), `x-dead-letter-routing-key` (routing key when dead-lettering), `x-queue-type: quorum`. Key insight from Designing Data-Intensive Applications: "A database and a message queue have some superficial similarity — both store data for some time — but they have very different access patterns, which means different performance characteristics, and thus very different implementations." RabbitMQ implements the AMQP model where messages are routed, acknowledged, and deleted — optimized for task dispatching and RPC patterns. Kafka implements the log model where records are appended and retained — optimized for event streaming and replay. The architectural choice between them is driven by this distinction: if consumers need to replay past events or multiple independent consumers need the same messages, use a log-based broker. If you need complex routing, per-message TTL, or priority queues, use RabbitMQ. Quorum queues (RabbitMQ 3.8+) bring Raft-based consensus to RabbitMQ, providing durability guarantees comparable to Kafka with `acks=all` and `min.insync.replicas=2`.
Acknowledgments and Dead Letter Queue
Consumer acknowledgments control message lifecycle. Auto-ack (`autoAck=true`): message deleted immediately on delivery — fastest but loses messages if consumer crashes before processing. Manual ack: consumer sends `basicAck(deliveryTag, false)` after successful processing — at-least-once. `basicNack(deliveryTag, false, requeue=true)` sends back to the head of the queue; `requeue=false` routes to the Dead Letter Exchange (DLX). Beware the infinite requeue loop: if processing fails for a poison message and you always requeue, the message bounces forever, consuming CPU and blocking other messages behind it. Solution: check the `x-death` header (count of previous deaths) — after N retries, nack with requeue=false to route to DLQ. Dead Letter Exchange: a normal exchange you declare and bind a DLQ queue to. Configure on the source queue with `x-dead-letter-exchange`. The DLX receives messages with original headers plus `x-death` metadata (original queue, death reason, time, count). Key production configs: `prefetch_count` (QoS) — limits how many unacked messages a channel holds, preventing slow consumers from starving others. `channel.basicQos(10)` means at most 10 unacked messages per consumer at a time. Without this, RabbitMQ pushes all messages to one consumer's TCP buffer even if it's slow.
Code example
// Quorum queue with DLX, prefetch, manual ack
Map<String, Object> queueArgs = Map.of(
"x-queue-type", "quorum",
"x-dead-letter-exchange", "orders.dlx",
"x-dead-letter-routing-key", "orders.dead",
"x-delivery-limit", 3 // quorum queues: auto-DLQ after 3 delivery attempts
);
channel.queueDeclare("orders.queue", true, false, false, queueArgs);
channel.basicQos(10); // at most 10 unacked messages per consumer
channel.basicConsume("orders.queue", false, (tag, delivery) -> {
try {
processOrder(delivery.getBody());
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
} catch (NonRetryableException e) {
// requeue=false → goes to DLX
channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, false);
} catch (RetryableException e) {
channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, true);
}
}, tag -> log.warn("Consumer cancelled: {}", tag));