Request routing

A chart does not require a separate physical machine. While one-to-one mappings between charts and machines are common, multiple charts can reside on a single machine.

Such configurations are called virtual charts. For example, three charts can run on each of two machines, resulting in a many-to-one relationship between charts and servers.

This setup simplifies chart rebalancing—though that topic will be covered in the next video.

When I refer to a 'chart' in this lesson, I mean a specific chart instance running on a physical machine.

When I say 'chart machine,' 'chart server,' or 'chart node,' I mean the physical machine hosting that chart instance.

When a user sends a request to a first-home service—whether to read or write data for a key—each application server in the service cluster must perform two actions:

1. Identify the chart that stores data for the specified key.
2. Forward the request to the physical machine hosting that chart.

To identify the correct chart, the application server uses one of three partitioning strategies: lookup, range, or hash.

To determine which machine hosts a chart, the application server requires a mapping between a chart name and the IP address of its host machine.

Every client must obtain and maintain an up-to-date version of this mapping.

One initial idea is to store the mapping in a file and deploy it to every client.

This method is simple to implement and may work well in small-scale systems with few charts.

The approach lacks flexibility due to several dynamic changes:

• The number of charts can increase.
• IP addresses change when a machine fails and is replaced.
• Shard-to-IP assignments shift during chart rebalancing.

Each time the mapping changes, every client must be redeployed.

With many first-home services, this results in a large number of redeployment operations.

Instead of deploying the file to every client, store it in shared storage—such as a 3-storage service.

Each application server runs a demand process that pulls the file from this shared location at regular intervals (e.g., every minute or several minutes).

This reduces deployment overhead: only one location needs to be updated.

The update propagation time is significantly faster than full redeployment.

This approach increases client complexity.

Each client must run and maintain the demand process.

The process must remain active at all times.

If it fails or stops, changes to the mapping will not be reflected in time.

What if we remove the mapping entirely from the client?

If clients do not maintain the chart-to-IP mapping, they do not need to be updated when configurations change.

This eliminates the need for client-side deployment or maintenance.

The next step in this design would be to explore how clients can discover chart machines without relying on stored mappings.

This approach reduces client complexity and improves system resilience.

It shifts the responsibility of mapping updates to the infrastructure layer.

This design choice will be expanded upon in the next section.

There are two distinct approaches for routing requests in a distributed system.

Proxy Option

A reverse proxy sits between clients and chart servers.

• Clients send requests to a proxy machine.
• The proxy uses a mapping to determine which chart server handles the request.
• The proxy then forwards the request to the appropriate chart server.

The proxy maintains full knowledge of chart server locations.

Peer-to-Peer Option

Chart servers discover each other and manage routing directly.

• Clients connect to a random chart server.
• The server checks if it owns the requested chart.
• If not, it forwards the request to the correct server.
• The server waits for a response and returns it to the client.

Both models keep clients lightweight.

Clients must know the list of IP addresses for either:

• Proxy servers (in proxy option), or
• Chart servers (in peer-to-peer option).

This list changes infrequently.

• For small server counts: share the list via DNS.
• For large server counts: use a load balancer between clients and servers.

Two primary methods exist:

• Static IP List

  • A fixed list of chart server IPs.
  • Deployed only to a limited number of proxy or chart machines.

• Service Registry

  • A dedicated service allows chart machines to register and report status.
  • Updates propagate when servers are added or removed.
  • Proxy and chart machines retrieve metadata from this service.

• Gossip Protocol

  • Chart servers exchange state information continuously.
  • Enables dynamic discovery and health monitoring.

The concept of chart discovery mirrors service discovery.

A chart is an instance of a service.

Thus, the underlying principles are identical.