Metadata cache

The deduplication cache from the previous video is simple: it stores deduplication IDs and lets them expire after a set time.

A more typical use case involves a backend store. Data is stored both in the cache for faster reads and in the backend store for persistence.

The core challenge is maintaining consistency between the cache and the backend store.

This means ensuring the cache copy of data is as up to date as the data in the backend store.

Several patterns exist for achieving this. They can be classified by who manages synchronization.

First, the application manages both the cache and the backend store.

In this pattern, the application queries the cache first.

If data is found, it's a cache hit — the application uses the cached value and does not access the backend store.

If data is not found, it's a cache miss — the application queries the backend store and stores the result in the cache.

This pattern is known as cache aside.

Cache aside keeps the cache on the side, with no direct interaction between the cache and the data store.

It is widely used but has notable drawbacks.

Every cache miss results in three round trips: one to check the cache, one to fetch from the backend, and one to store in the cache.

This makes it unsuitable for latency-critical systems.

When data in the backend store changes, the cache has no knowledge of this change.

This issue is partially mitigated by setting expiration times on cache entries.

While this does not fully solve the problem, it works well in practice for many applications.

Another issue is cache stampede, also known as dogpiling.

When a cache entry expires, multiple threads simultaneously attempt to query the backend store.

This causes a surge in load on the backend store.

This is not a problem for small systems but can become severe in large-scale systems.

This behavior occurs when the application manages both stores.

Setting expiration times helps maintain synchronization between the cache and the backend store.

Next, the cache takes responsibility for synchronization.

In this model, the application no longer queries the backend store directly.

All read and write operations go through the cache.

This leads to two key patterns: read through and write through.

In the read through pattern, the cache loads data from the backend store when a cache miss occurs.

In the write through pattern, any modification to data in the cache is written back to the backend store.

Both patterns ensure synchronous updates between the cache and the backend store.

The application waits for the operation to complete before returning a response.

A key advantage of these patterns is simplified application code.

The application only needs to interact with a single data source: the cache.

This reduces complexity in data access logic.

The read through pattern also protects the backend store from traffic spikes caused by cache stampede.

When a cache entry expires, the read through pattern handles the backend fetch directly.

This prevents multiple threads from simultaneously hitting the backend store.

The write through pattern ensures that every change in the cache is immediately reflected in the backend store.

This guarantees data consistency across both stores.

Both patterns eliminate the need for application-level logic to manage cache and store synchronization.

They provide a more robust and scalable solution for systems with high read and write loads.

The choice between patterns depends on system requirements, latency needs, and failure tolerance.

Each pattern has trade-offs in performance, complexity, and reliability.

The read through pattern is better for systems where backend store stability is critical.

The write through pattern is better for systems requiring strong consistency and immediate data propagation.

These patterns form the foundation of metadata cache design in production systems.

Ensures that even if many requests for the same object arrive, only one read operation accesses the data store.

Requests for the same object are queued. Once the data store response is fetched, it is sent to all clients on the waiting list.

This technique is called request coalescing.

All requests go through the cache, which may store large amounts of data rarely accessed.

This is less problematic in large caches but can become an issue in small caches.

The cache becomes a single point of failure.

If the cache is unavailable, the entire system fails.

Writes data synchronously to both the cache and the backend store.

The application waits for both writes to complete.

If either write fails, the application receives an exception.

The application may retry or lock the exception.

This improves data persistence reliability.

Waiting for both writes reduces write performance.

Writes data to the cache first, then asynchronously writes to the data store.

This increases write throughput and reduces latency.

If the cache crashes before data is propagated to the data store, data is lost.

Clients can configure queues and topics with parameters like maximum message size and retention time.

This metadata is stored persistently on disk using an embedded database.

Metadata is also kept in memory for faster access.

Validation of message size occurs every time a message arrives at the broker.

Fast access to metadata is critical.

The metadata cache uses read through and write through patterns.

Read through: retrieves data from the cache if available, otherwise from persistent storage.

Write through: writes data to both cache and persistent storage on every update.