Scalability

Scalability is a system's ability to handle increasing load.

Load can be measured in multiple ways:

• Number of requests per second
• Volume of incoming or outgoing data
• Number of concurrent connections
• Other performance metrics

Vertical scaling (scaling up) adds more resources to a single machine.

Examples include increasing CPU, memory, or storage.

Horizontal scaling (scaling out) adds more machines to the system.

A system with horizontal scaling is often described as having a shared nothing architecture.

Scalability is difficult to achieve.

Designers must consider scalability early in the process.

Vertical scaling has limits.

Even the most powerful machines may not handle excessive load.

High-end machines are often expensive.

The second or third most powerful machine may still be unaffordable.

Horizontal scaling offers nearly unlimited capacity.

New machines can be added as load increases.

However, it introduces complexity.

Service discovery: clients must locate all service machines.

Load balancing: requests must be distributed evenly across machines.

Request routing: clients may need to reach the same machine for state persistence.

Maintenance: managing multiple machines adds operational overhead.

Some systems begin with a single machine.

They scale vertically until cost or performance limits are reached.

Then, they transition to horizontal scaling.

A relational database is a typical example.

It starts as a single instance.

It scales vertically to a limit.

Later, it can be horizontally scaled using sharding (for writes) and replication (for reads).

Many assume horizontal scaling is the default for all systems.

This is not true.

Scalability needs must be clarified with stakeholders.

In single-machine systems, availability is often more critical than scalability.

A single machine will eventually fail.

Thus, systems must be designed with redundancy.

High availability can be achieved with a single machine using active-passive setups.

This is a form of fault tolerance.

Vertical and horizontal scaling can be used together.

Start with a horizontally scalable system.

Keep the number of machines small for easier maintenance.

As load grows, replace machines with more powerful ones.

When vertical scaling becomes too costly, shift to horizontal scaling.

Elasticity is a system's ability to acquire resources as needed.

It enables dynamic resource allocation.

Elasticity supports scalability by adjusting capacity in real time.

It is often paired with scalability in modern system design.

Elasticity and scalability are distinct concepts.

Elasticity refers to the ability to dynamically adjust resources in response to short-term workload changes.

A typical request pattern follows a normal distribution: traffic rises in the morning, peaks, then declines.

At night, the system requires N machines.

During the day, peak traffic demands 2N machines.

An elastic system starts with N machines at night.

As traffic increases, it adds machines throughout the day.

After the peak, it gradually reduces the number of machines.

This short-term adjustment is elasticity.

Elasticity adapts to immediate workload shifts.

It reduces operational costs by avoiding over-provisioning.

A non-elastic system maintains 2N machines continuously.

This leads to higher costs due to idle capacity.

Elasticity saves money by aligning resources with real-time demand.

Scalability addresses annual or long-term growth.

As the system gains popularity, traffic increases over time.

The system must scale up to meet growing demand.

This scaling occurs in incremental steps over extended periods.

Scalability is a strategic, long-term requirement.

It ensures the system can grow with user demand.

Elasticity depends on scalability.

If a system cannot scale, it cannot quickly provision or deprovision resources.

Thus, scalability is a prerequisite for elasticity.

Scaling can be manual or automatic.

Automatic scaling is known as autoscaling.

Autoscaling is a cornerstone of cloud computing.

We will explore its high-level design in the course.

Autoscaling enables dynamic, real-time resource adjustments.

It supports both elasticity and long-term scalability.

The system must support both short-term and long-term resource management.

Without scalability, elasticity is not feasible.

Autoscaling enables efficient, responsive infrastructure management.