Implementing Message Queue Logic

1. Designing Message Producer

Example: Kafka producer

@Service @RequiredArgsConstructor
public class OrderProducer {
    private final KafkaTemplate<String, OrderPlaced> kafka;
    public void publish(OrderPlaced e) {
        kafka.send("orders", e.orderId().toString(), e);
    }
}

2. Designing Message Consumer

Example: @KafkaListener

@KafkaListener(topics = "orders", groupId = "billing")
public void onOrder(OrderPlaced e, Acknowledgment ack) {
    billing.charge(e);
    ack.acknowledge();   // manual commit after success
}

3. Implementing Message Acknowledgment

ModeDetail
AutoFast but at-most-once risk
Manual after successAt-least-once
Batch ackHigher throughput
Negative ack (nack)Trigger redelivery / DLQ

4. Implementing Dead Letter Queues

AspectDetail
When to sendAfter max retries
IncludeOriginal message + error + attempts
AlertingOn DLQ depth growth
Replay toolingMove from DLQ back to main

5. Implementing Message Retry Logic

Example: Spring Retry on consumer

@RetryableTopic(
    attempts = "5",
    backoff = @Backoff(delay = 1000, multiplier = 2.0),
    dltStrategy = DltStrategy.FAIL_ON_ERROR)
@KafkaListener(topics = "orders")
public void onOrder(OrderPlaced e) { billing.charge(e); }

6. Designing Message Format

FormatTrade-off
JSONHuman-readable, larger
AvroSchema registry, compact
ProtobufFast, compact, typed
MessagePackBinary JSON-like

7. Implementing Message Routing

PatternDetail
Topic-basedKafka topics
Direct exchangeRabbitMQ routing key match
FanoutBroadcast to all queues
Header-basedMatch on metadata
Content-basedInspect payload

8. Implementing Topic Subscription

PatternDetail
Single consumer groupLoad-balanced delivery
Multiple groupsEach gets full stream
Partition assignmentRange / round-robin / sticky
WildcardsTopic pattern matching

9. Handling Message Ordering

SystemHow
KafkaPer-partition order; partition by key
RabbitMQSingle queue + single consumer
SQS FIFOMessageGroupId for ordering
Trade-offOrdering reduces parallelism

10. Implementing Message Persistence

SystemDetail
KafkaDurable log; configurable retention
RabbitMQDurable queues + persistent messages
Replication factor≥ 3 for production
AcksProducer acks=all for safety

11. Implementing Producer Retry Logic

SettingDetail
retriesMax attempts on transient errors
retry.backoff.msSpacing
enable.idempotencePrevents duplicates on retry (Kafka)
max.in.flightSet to 1 for strict order with retries

12. Implementing Consumer Group Management

ConceptDetail
Group IDIdentifies logical consumer
Offset commitPer group per partition
RebalanceOn member join/leave
Static membershipReduce rebalance churn
Lag monitoringCritical health metric