When to retry

An operation is idempotent if, regardless of how many times it is executed, the result remains the same.
This property is critical when designing retry logic.

For example, the operation ax = 5 is idempotent because the variable x does not change with repeated execution.
In contrast, x++ is not idempotent—each execution increases x, leading to cumulative changes.

When calling APIs, we assess whether an operation is idempotent.
An API is idempotent if multiple identical requests produce the same outcome as a single request.
Read-only APIs are typically idempotent.
APIs that create resources, however, are generally not idempotent.

Consider a service A calling service B to retrieve read-only data.
No matter how many times service A sends the request, the result remains consistent.
This consistency is essential for reliable retry behavior.

If a service creates a resource, such as a video or token, each request may result in a new resource.
For instance, a service "A" may generate a unique video token.
Service "B" then stores this token and serves it to clients.

Service "B" maintains a unique string identifier—64 ASCII characters—representing the video.
When service "B" first processes a request, it generates the video and stores it.
Subsequent requests for the same video simply retrieve the stored version.

This behavior demonstrates idempotency: the output remains consistent across repeated calls.
Services that perform idempotent operations typically store state for longer durations.
For example, an API from service A may store results for 10–15 minutes.
A payment processing API might store state for 24 hours.

To design effective retry logic, we must determine whether a service is idempotent.
We must identify all APIs in our system and classify them as idempotent or not.
This classification must be accurate across all client interactions.

We must also evaluate failure scenarios.
If a request fails, can we safely retry it?
If the operation is not idempotent, retrying may cause unintended side effects.

For instance, if a request fails and we retry it, and the operation creates a resource, we risk creating multiple instances.
If the operation is idempotent, retrying is safe.

A failure may occur due to missing credentials or network issues.
If credentials are not available, the request fails and cannot be retried.

If a request fails and no state is stored, we cannot determine whether it was successful.
We cannot safely retry such requests.

If a request fails and the system has no state, we must avoid retrying.

In some cases, a system may store state to allow retry.
For example, a system may store a request ID and its status.

When a request fails, we must check whether it was processed.
If it was, we can skip retry.
If not, we may need to retry—only if the operation is idempotent.

We must ensure that any retry mechanism respects the idempotency of the operation.
Failure to do so may result in duplicate resources or inconsistent data.

The system must track request state to determine whether a retry is safe.
This state must be stored and accessible across retries.

A retry is only safe when the operation is idempotent and the system has sufficient state.
Without this, retrying may lead to data corruption or unintended behavior.

We must implement retry logic that respects both idempotency and state visibility.

A retry should only be considered when we know that the endpoint is critical.

If a service fails and no recovery is possible, retrying is not justified.

Failure without recovery mechanisms cannot be addressed by retrying.

A failure may be retried if it occurs before a service call and if the service is capable of handling the failure state.

Typically, we should not retry a failure if the service has not yet responded.

Retrying is only acceptable when we know the endpoint is important.

The failure must be fully unblocked from the service before retrying.

If the endpoint is not critical, the failure should not be retried.

The failure must be fully unblocked from all involved services.

This means the failure must be completely resolved before any retry is attempted.

If the endpoint is not important, no retry is necessary.

Retrying without full unblocking leads to instability.

The system must ensure that all services involved in the request are fully unblocked.

Failure to unblock the service before retrying undermines reliability.

Retrying should only occur when the endpoint is known to be essential.

Without this knowledge, retrying introduces risk.

The failure must be fully unblocked from the service before any retry is allowed.

This applies to all services involved in the request path.

No retry should occur if the endpoint is not critical.

The condition for retrying is clear: endpoint importance must be known.

Failure must be fully unblocked from the service.

Only then can a retry be safely executed.

Failure without unblocking is not a valid condition for retry.

Retrying without full unblocking violates system stability.

The system must ensure complete unblocking before retry.

This rule applies uniformly across all services.

Failure to meet this condition results in unreliable behavior.