Managing Message Reliability

1. At-Least-Once Delivery Pattern

AspectDetail
GuaranteeMessage delivered ≥1 time; possibly duplicates
MechanismProducer retries on broker failure; consumer acks after processing
RequiredIdempotent consumer
Default InKafka, RabbitMQ, SQS

2. At-Most-Once Delivery Pattern

AspectDetail
GuaranteeMessage delivered ≤1 time; may be lost
MechanismProducer fire-and-forget; consumer auto-ack before processing
Use CaseMetrics, telemetry where loss tolerable

3. Exactly-Once Delivery Pattern

AspectDetail
GuaranteeEach message processed exactly once (effectively)
Achieved ViaAt-least-once + idempotent consumer (most common)
Native SupportKafka transactions (EOS), Pulsar, Flink checkpointing
CostHigher latency, more coordination
Note: True end-to-end exactly-once across systems is impossible (Two Generals). Implement effectively-once via idempotency.

4. Idempotent Consumer Pattern

StrategyDetail
Dedup TableRecord event ID with side-effect in same tx
Natural IdempotencyOperation safe to repeat (e.g., SET x = 5)
Conditional UpdateUPDATE WHERE version = expected
UpsertINSERT ... ON CONFLICT DO NOTHING

Example: Idempotent Insert

INSERT INTO processed_events (event_id, processed_at)
VALUES ($1, now())
ON CONFLICT (event_id) DO NOTHING;
-- If 0 rows affected → duplicate; skip side effect

5. Message Acknowledgment Pattern

ModeDetail
Auto-AckBroker marks delivered on send; risk of loss
Manual AckConsumer explicitly acks after success
Negative Ack (NACK)Reject + requeue or send to DLQ
Batch AckAck range of offsets (Kafka commit)

6. Dead Letter Queue Pattern

AspectDetail
PurposePark messages that fail repeatedly
TriggerMax retries exceeded, deserialization fail, validation fail
ActionAlert, manual inspection, replay after fix
RetentionLonger than main queue (days/weeks)

Example: RabbitMQ DLX Configuration

Map<String, Object> args = Map.of(
    "x-dead-letter-exchange", "orders.dlx",
    "x-dead-letter-routing-key", "orders.failed",
    "x-message-ttl", 60000);
channel.queueDeclare("orders", true, false, false, args);

7. Poison Message Handling Pattern

StepAction
1. DetectFailure on every retry → poison
2. QuarantineMove to DLQ; do not block other messages
3. AlertNotify on-call with payload + error
4. DiagnoseInspect payload; fix code or data
5. ReplayRe-publish to main queue once safe

8. Message Deduplication Pattern

MethodDetail
Producer-SideIdempotent producer (Kafka enable.idempotence=true)
Broker-SideSQS FIFO dedup window (5 min); Kafka EOS
Consumer-SideInbox/dedup table
End-to-EndCombine producer + consumer dedup

9. Inbox Pattern

AspectDetail
PurposeAtomically dedupe + persist incoming events
MechanismInsert event ID into inbox table in same tx as side-effect
DuplicateInsert fails (PK conflict) → skip processing
CleanupPeriodic delete of old inbox rows

Example: Inbox Atomic Processing

BEGIN;
INSERT INTO inbox(event_id, received_at) VALUES ($1, now()); -- fails on duplicate
UPDATE accounts SET balance = balance + $2 WHERE id = $3;
COMMIT;

10. Priority Queue Pattern

ApproachDetail
Multiple QueuesSeparate queue per priority; consumers prefer high
Native PriorityRabbitMQ x-max-priority; ActiveMQ priority
FairnessUse weighted polling to avoid starving low priority
Note on KafkaNo native priority; use multiple topics or partitioning

Related Cheat Sheets

High Level Design

Designing Service Communication Patterns

Advanced Java

Working with Serialization

Low Level Design

Designing Interfaces and Abstractions

API Gateway

Implementing Authorization