System overload

System overload occurs when client requests exceed a service’s capacity to process them.

In push-based messaging, consumers may not be ready to process incoming messages, leading to performance degradation or failure.

In pull-based systems, brokers can be overwhelmed by excessive request volume.

Overload causes slow response times, request timeouts, and increased retry loops that worsen the problem.

Without protection, services degrade under load and may fail entirely.

System overload happens when the number of incoming requests exceeds a service’s processing capacity.

It can occur in both push and pull-based architectures.

Clients may send requests aggressively or in large volume.

Examples include a few clients making many requests or millions of users making concurrent calls.

• Request processing takes too long
• Clients exceed timeout thresholds
• Clients retry requests
• Retries increase load, worsening overload
• Service performance degrades
• Service may fail completely

• Request batching: combine multiple client requests into one batch
• Rate limiting: cap the number of requests per client or per time period
• Load shedding: drop non-critical requests during overload
• Client-side throttling: limit requests from individual clients
• Circuit breaking: stop processing requests when service is overloaded

• A messaging broker sending messages faster than consumers can process them
• A web service overwhelmed by millions of concurrent requests
• A client repeatedly retrying after timeout, increasing load