Network protocols in real-life systems

The choice of network protocol for publishing and retrieving messages depends on the system's accessibility.

If the messaging system is publicly accessible from the Internet, HTTP is the preferred protocol.

Cloud messaging systems like AWS SQS, SNS, and Kinesis use HTTP for both publishing and retrieving messages.

To publish a message, an HTTP request is sent to the system.

To retrieve a message, another HTTP request is sent.

If the messaging system is restricted to a private network and not exposed to the Internet, TCP is optimal.

TCP maximizes throughput due to its efficiency and lower overhead.

Kafka, for example, uses a custom binary protocol over TCP.

Producers and consumers establish socket connections to the broker and maintain persistent connections.

It is important to note that Kafka can be accessed from the Internet.

This is possible by installing Kafka on EC2 instances and opening TCP ports to the world.

Alternatively, Kafka REST Proxy provides a RESTful interface to a Kafka cluster.

We will discuss proxies in later sections of the course.

When designing distributed systems, the key decision is whether the system serves external clients, internal clients, or both.

Supporting multiple protocols is acceptable.

Internal clients can use TCP for minimal latency.

External clients use HTTP for accessibility and standardization.

When designing a service called by various clients—including other services—the protocol choice depends on exposure.

If the service is publicly available, HTTP is the clear choice.

HTTP is also widely used for internal services today.

Many services, whether internal or external, follow REST design principles.

RESTful services are inherently tied to the HTTP protocol.

If the service is internal-only and performance is critical, an RPC service over TCP—or even UDP—should be used.

Apache Thrift can help build such a service.

We will explore REST and RPC in later course sections.

For storing and querying data, the protocol choice depends on access patterns.

If the database is a cloud service like DynamoDB, it uses a REST-based API accessible over HTTP.

Often, databases are not publicly accessible.

They are accessed by trusted clients within secure, internal environments.

In such cases, application servers establish direct connections to the database server.

TCP is the natural choice for these connections.

This applies to both SQL databases like Oracle and MySQL, and NoSQL databases like Cassandra.

Data stored in a database should be replicated across multiple locations.

This ensures high availability and fault tolerance.

Replication supports continuous access even if one site fails.

It is a core requirement in distributed systems.

Replication must be managed carefully to maintain consistency and performance.

The protocol used for replication may differ from the protocol used for client access.

For example, replication might use TCP for internal communication between nodes.

Client access might use HTTP or a custom protocol.

The system design must account for both access and replication protocols.

Each component must be evaluated independently for protocol suitability.

The decision should be based on performance, security, and access constraints.

No single protocol fits all scenarios.

The right choice emerges from understanding the system’s environment and usage.

Internal vs. external access is the primary factor.

Performance, security, and scalability must all be considered.

When a database server fails, data must remain accessible on other machines through replication.

For fast, reliable replication, TCP is the preferred choice.

In leader-based replication, each follower establishes a TCP connection with the leader.

The leader uses this connection to send data updates to followers for every request.

Some databases, like CouchDB, use HTTP for replication.

However, HTTP is not ideal for internal cluster communication.

Distributed systems need to determine which nodes are active.

Protocols like GoCyp help identify active members of a cluster.

Databases use this information to route requests to the correct node.

In a cluster, each machine runs both an application and a cache process.

Each cache process stores a portion of cached data.

When an application performs a cache lookup, it sends a request to the machine holding the requested key.

Cache processes on all machines must communicate to share data.

This design reduces hardware costs.

It avoids the need for a separate cache cluster.

It is operationally simpler than maintaining a dedicated cache system.

To prevent overload, each machine tracks request counts.

Every second, a machine records how many requests it has processed.

This data is shared with other machines in the cluster.

Each machine then computes the total number of requests processed across the cluster.

On receiving a new request, each machine uses this total to decide whether to accept or reject it.

This ensures the cluster stays within a defined request limit.

Machines in a cluster communicate to vote and elect a leader.

Many system design problems rely on having a leader.

This topic will be covered in a separate discussion.

For now, focus only on network protocols used in leader election.

Internal cluster communication does not require public APIs.

Speed and reliability are the top priorities.

The choice is between TCP and UDP.

Both UDP and TCP can solve the membership problem.

UDP is acceptable because lost gossip messages are typically recovered in the next round.

TCP is also viable.

Reliable TCP connections help nodes achieve synchronized views of the cluster state faster.

Two nodes establish a TCP connection and exchange multiple messages over it.

This enables more consistent and synchronized cluster state.

TCP’s reliability supports faster convergence of cluster membership views.

This makes TCP superior for maintaining synchronized cluster awareness.

In a distributed cache, both UDP and TCP are viable options for node communication.

Using UDP may result in occasional data loss, leading to a higher percentage of cache misses.

If the service can tolerate such misses, UDP is a valid choice.

I have worked extensively with TCP implementations in real systems.

These have performed reliably under practical conditions.

When nodes use UDP, sporadic data loss causes under-reporting of processed requests.

This leads to less accurate rate limiting.

TCP provides more accurate rate limiting by ensuring reliable data exchange.

However, TCP introduces slower data exchange between nodes.

Both UDP and TCP can be used for leader election.

For small clusters, both protocols are acceptable.

As cluster size grows—reaching thousands of machines—TCP connection overhead increases.

UDP becomes more efficient in such large-scale deployments.

The video you are watching is delivered via one of these:

• UDP
• TCP sockets
• HTTP

Consider which protocol is used in this delivery.

Reflect on this before the next segment begins.