Bulkhead

The bulkhead pattern partitions system resources into isolated groups to limit the impact of failures.

When a group of resources is exhausted, only that group is affected.

Other parts of the system continue to function normally.

Think of a ship with bulkheads—internal walls that divide compartments.

If one compartment fills with water, the ship remains afloat.

The Titanic sank because water overflowed bulkheads, flooding the entire hull.

Instead of one shared thread pool for all consumers,

create multiple smaller thread pools.

Each pool serves a subset of consumers.

If one consumer slows down and uses all threads in a pool,

other pools continue delivering messages to remaining consumers.

For example, an order service must:

• Check inventory

• Process payment

The order service should not be overwhelmed by a degraded dependency.

The bulkhead pattern prevents this.

1. Limit connections per dependency using separate connection pools.

2. Limit concurrent requests using semaphores.

3. Use dedicated thread pools per dependency.

Set up a separate connection pool for each dependency.

Define a maximum number of connections per pool.

When all connections are in use, new requests are blocked.

This ensures only a limited number of threads call any one dependency.

Use semaphores to track concurrent requests.

When the limit is reached, new requests are immediately rejected.

This prevents resource exhaustion.

Create a separate thread pool for each dependency.

Each thread pool has a fixed size.

Worker threads route requests to the appropriate pool.

Only a limited number of threads can call any given dependency.

Method 1 is simplest.

It’s a configuration setting in the client.

But it relies on client trust.

Client libraries are constantly changing.

They may have bugs.

Performance may degrade.

Connection handling may change.

Therefore, service teams treat client libraries as black boxes.

They reduce reliance on client behavior.

They provide protection against client library failures.

Dedicated thread pools offer the strongest isolation.

But they introduce computational overhead.

Less overhead than method 3.

But less flexible.

For example, it does not allow request timeouts.

A worker thread remains blocked until the network call times out.

This can lead to unresponsive behavior in degraded conditions.

The bucket pattern divides system resources into bounded groups to isolate failures and limit impact.

Each group must have a defined, finite size to prevent uncontrolled resource consumption.

Critical components must be constrained, including:

• Queue capacities
• Thread pool sizes
• Message payload sizes
• Workload execution times

The system must shape incoming traffic by imposing request rate limits.

Identifying appropriate limits across all dimensions is complex and often requires deep system understanding.

Limits must be reviewed and updated regularly to reflect changing workloads and conditions.

The system must undergo regular load testing and stress testing.

Failure scenarios must be tested to ensure resilience under adverse conditions.

These testing and operational practices will be discussed in a separate section on testing and operational excellence in distributed systems.