Service discovery

The host discovery problem—where one host finds another using a hostname—is a specific case of service discovery.

Service discovery is a fundamental problem in distributed systems, especially in microservices architectures.

When all instances of service A must identify the IP addresses and port numbers of all service B instances to send requests, the system must solve the service discovery problem.

There are two primary patterns: server-side discovery and client-side discovery.

In server-side discovery, a standalone component—such as a reverse proxy or load balancer—sits between services.

This component maintains knowledge of all service B instances.

Service A instances use a single IP address or hostname for the load balancer.

They send requests to the load balancer, which forwards them to appropriate service B instances.

Service B instances do not handle discovery; the load balancer manages it.

In client-side discovery, service B instances register themselves in a shared service registry.

Each service A instance reads the registry to obtain a list of available service B instances.

It then selects one instance to send a request to.

This pattern is the focus of this video.

We begin with the service registry component.

The service registry is not DNS.

Although DNS can be used for client-side discovery, it is not optimal.

In large, dynamic service clusters, instances frequently come and go.

Some may become unavailable due to network partitions.

Others are added quickly to scale.

DNS suffers from propagation delays and caching, making it unsuitable for such environments.

Yet, like DNS, the service registry must be reliable and scalable.

The service registry consists of multiple servers.

Service instances in the cluster share information using TCP or HTTP.

When a service instance wants to register, it sends a registration request to one of the registry servers.

During registration, it provides the service name, its IP address, and port number.

The registry server stores this data and replicates it across the cluster.

Data replication ensures any registry node can respond to read requests.

Thus, when service A queries the registry for service B, any node can return the latest list of service B instances.

Service B instances must send heartbeat requests to the registry.

These occur every few seconds to confirm they are alive.

If multiple consecutive heartbeats fail, the registry marks the instance as down.

The registry removes the instance from the list.

This change is propagated to all service A instances.

They stop sending requests to the failed instance.

Service A instances may cache registry results.

Why? Because the registry itself may become unavailable.

For example, during a network failure.

With caching, service A instances can still retrieve service B instance data locally.

This allows them to continue operations.

It mirrors how local DNS resolvers cache DNS data.

We now examine the service registry in greater detail.

There are two primary approaches for services to retrieve information from a service registry.

Embed a service registry client directly into the service application.

• The client handles communication with the registry and caches responses.
• This approach is easy to implement and maintain.
• Requires language-specific clients:
  • Java applications need a Java client.
  • Go applications require a Go client.
• The integration occurs at the code level, making it language-dependent.

Run a separate demon process on each machine in the cluster.

• The demon connects to the service registry, retrieves data, and stores it locally (e.g., as a file on disk).
• This data can be used to configure a load balancer running on the same machine.
• The service application sends requests to the load balancer, which forwards them to appropriate service instances.

Advantages of this approach:

• Language agnostic: The demon can be written in any programming language, regardless of the service application.
• Less error-prone: The demon runs in its own memory space. Bugs in the demon do not affect the service application.
• Isolated failures: Issues in the demon’s registry communication are confined to the demon process.
• Shared registry data: Multiple applications on the same machine share the same cached registry information.

Both Pinterest and Airbnb use this decoupled demon approach for service discovery.

More details are available in the engineering blocks.