Page cache

Even though embedded databases are viable for storing messages, many messaging systems do not use them.

Databases are optimized for long-term storage and operations like create, update, delete, and range queries.

A messaging system only needs to store and delete messages quickly—no need for complex queries or aggregations.

Thus, a full-featured database is overkill and unnecessary.

A log file with indexes is sufficient for messaging systems.

This design has been discussed previously.

However, additional concepts merit attention.

This video explores those concepts.

To increase disk throughput, we must maximize both write and read performance.

Two primary inefficiencies occur during intensive disk operations:

• Too many small I/O operations
• Excessive byte copying

To reduce small I/O operations, avoid writing messages one by one.

Instead, buffer multiple messages in memory, then write them as a single batch.

Batching reduces overhead: instead of incurring processing costs per message, the cost is paid once per batch.

This principle applies beyond messaging.

For example, in network communication, sending one message per request incurs connection, thread, and setup costs.

Combining messages into a single request reduces these costs—paying them only once.

Where and how should messages be buffered before writing to disk?

The answer lies in leveraging the operating system’s page cache.

A page cache is a disk cache in Linux.

A page or virtual page is a fixed-length block of virtual memory—smallest unit in memory management.

When data is written to a file system, it first goes to the page cache.

Data remains in the page cache until it must be written to disk.

Flashing to disk occurs in two cases:

• When an application explicitly requests it
• When the OS decides to do so based on its own flushing policy

Flushing is handled by background threads.

When reading a file from disk, file blocks are written to the page cache.

If the file is read again later, the blocks are served directly from the page cache—no disk access required.

Most physical memory not directly assigned to applications is used by the OS for the page cache.

When Linux needs more memory for applications, unused page cache areas are automatically freed.

When storing messages:

• Open a segment file
• Write messages to it
• Data enters the page cache

The OS enhances throughput using two mechanisms:

1. Batching consecutive small writes into larger physical writes
2. Attempting to reduce redundant disk reads by serving data from memory

Re-sequence writes to reduce physical movement of the disk head. This improves I/O efficiency by organizing write operations in a more predictable order.

The page cache can be flushed to disk in two ways:

• Every few seconds
• When a certain number of messages accumulate in the cache

We can implement message batching using a local in-memory cache. The page cache offers an OS-native alternative to custom in-memory caching.

Linux divides virtual memory into two distinct areas:

• Kernel space: where the operating system kernel runs
• User space: where applications execute

When an application retrieves a message from a segment file:

1. The OS reads data from disk into the page cache (kernel space)
2. The application reads data from kernel space into a user space buffer
3. The application writes data back into kernel space via a socket buffer
4. The OS copies data from the socket buffer to the network interface card buffer for transmission

This process results in four data copies, contributing to the excessive byte-copying problem.

Zero-copy read avoids user space entirely. The OS transfers data directly from the page cache to the network interface.

This enables message consumption rates approaching the network connection’s bandwidth limit.

Zero-copy read only works when no message manipulation occurs in user space.

If filtering or routing logic is required, data must be copied into user space.

A message may be lost if:

• The broker accepts it and adds it to the page cache
• Sends an acknowledgment immediately
• The broker crashes before writing the message to disk

The message remains in memory and is never persisted.

This loss risk is not unique to page caches. Any in-memory buffer is susceptible to data loss upon system failure.

Flushing the page cache after every message write reduces throughput due to high I/O overhead.

Replication of messages can mitigate data loss risks.

Apache Kafka relies heavily on page cache for both read and write operations.

This design underpins Kafka’s ability to guarantee message durability.