Message acknoledgement

When a producer sends a message to a broker, it must determine when to consider the message successfully delivered.

Network issues or transient broker errors can cause messages to be lost in transit.

Without confirmation, the producer cannot know if the message reached the broker.

This leads to the need for message acknowledgments.

---

Unsafe acknowledgment mode:

• Message is considered sent when added to the producer's socket buffer.
• No confirmation is sent from the broker.
• There is no guarantee the broker received or stored the message.
• Often referred to as "fire and forget".

Safe acknowledgment mode:

• Producer waits for an acknowledgment from the broker.
• Acknowledgment may occur:
  • When the message is received and stored in memory.
  • When stored successfully on disk.
  • When replicated across multiple machines.
• The broker defines what "safe" means.

---

Acknowledgments introduce latency.

Time is needed for:

• Persisting messages to disk.
• Replicating messages across machines.

This creates a trade-off between:

• Throughput (how many messages can be sent per second).
• Durability (how reliably messages are stored).

Unsafe mode maximizes throughput when message loss is not critical.

Safe mode ensures higher durability at the cost of reduced throughput.

---

When a message is sent:

• The producer starts a timer.

If no acknowledgment is received within a specified timeout:

• The producer assumes the message was lost.
• The producer retries sending the message.

This retry process continues until an acknowledgment is received.

---

TCP uses a similar retry mechanism:

• The sender maintains a timer from packet transmission.
• If no acknowledgment is received, the packet is retransmitted.

Both TCP and message systems may result in duplicate messages.

TCP solves this with sequence numbers.

A broker can use a similar approach to eliminate duplicates.

---