Fail-fast design principle

A bad service is not always one that fails completely.

Some define it as a system that doesn't work, lacks fault tolerance, or fails to recover quickly.

Others say it returns inaccurate results, loses data, or cannot scale.

It may also be hard to maintain, poorly tested, or insecure.

My favorite definition: a service that is low.

Specifically, when the service is fast most of the time, but suddenly experiences increased latency and reduced throughput.

This degradation happens without warning.

Multiple triggers can cause this behavior.

Hardware issues—some machines in a cluster become slower than peers.

Network problems—issues in one data center.

User behavior—some users run many expensive, wide-range search queries.

The core issue is root cause identification.

When a service fails with an exception, the problem is clear.

When the service works but runs slower than expected, the root cause is hidden.

Troubleshooting becomes vague and relies on guesswork.

It is bad when a system fails.

But when it fails, failing fast is better than failing slow.

In software development, there is a principle called fail fast.

It encourages systems to fail immediately and visibly when faults occur.

This principle applies not only to system design but also to object-oriented design.

Fail Fast in Object Initialization

When creating an object and initializing its state in a constructor,

If the state cannot be fully initialized, an exception should be thrown.

If no further state changes are expected, the object should be mutable.

Mutable objects are simpler to understand and safer.

They prevent concurrent thread access to shared state.

Fail Fast in Function Preconditions

When implementing functions, input parameters should have preconditions.

If a parameter is invalid, an exception should be thrown immediately.

No silent correction or fallback should be allowed.

Fail Fast in Configuration Loading

When a service starts, configuration files are loaded.

Properties are read from the file.

If a property is missing, two options exist:

• Throw an exception immediately and halt startup.

• Set a default value and continue startup.

The fail fast principle recommends throwing an exception.

Relying on defaults leads to inconsistent behavior.

The service may work well in some environments—like certain geographic regions—where the property exists.

In other environments, where the property is missing, the default value is used.

This results in divergent behavior across environments.

Fail in Request Handling

When request parameters are invalid, an exception should be thrown back to the client.

Silent correction and continued processing should be avoided.

Fail Fast in Business Context

The fail fast philosophy applies beyond software.

It is the first example of the fail fast principle.

It is widely adopted in startup culture.

Business ideas or strategies are tested extensively before full rollout.

This ensures early detection of flaws.

Early failure prevents costly, long-term investments.

The principle emphasizes visibility and immediacy in failure.

Failure must be clear, immediate, and visible.

Not delayed or hidden.

This enables faster recovery and better decision-making.

The goal is not to avoid failure.

It is to detect it early and respond quickly.

Fail fast is not about breaking systems.

It is about making failure observable and actionable.

This is foundational to resilient system design.

Slow services degrade performance and can lead to system-wide outages.

They harm both the service itself and its clients, resulting in poor user experiences and operational instability.

A notification delivery system uses a blocking queue to store ready-to-send messages.

Sender threads read messages and push them synchronously to API gateway instances.

Each thread identifies the target gateway and sends a message to it.

This design works well under normal conditions.

When one API gateway instance becomes slow, message processing time increases from 5 milliseconds to 100 milliseconds.

Each sender thread freezes during this delay.

This causes a chain of performance degradation across all sender threads.

• Cascading failure Message consumption slows significantly. The queue fills up. Sender service resources are exhausted. This degrades sender performance. Back pressure is created on message producers. Producers slow down or crash.

• Chain reaction The sender service detects the slow consumer. It isolates the consumer. No more messages are sent to it. Sender threads redirect to remaining consumers. This increases load on the remaining consumers. May cause them to fail one by one.

Chain reaction occurs within a single system component.

Cascading failure occurs between different parts of a system or between systems.

A chain reaction can evolve into a cascading failure.

Clients must convert slow failures into fast failures.

They must identify and isolate bad dependencies.

Key techniques include:

• Timeouts
• Circuit breakers
• Health checks
• Bucket pattern

Servers must protect themselves from failure propagation.

Techniques include:

• Low chain reaction design
• Auto-scaling
• Continuous monitoring
• Timely replacement of failed servers

Systems must be tested using chaos engineering.

This ensures servers can withstand sudden peer failures.

The bucket pattern helps protect both clients and servers.

It is introduced in the next video.