Why this matters
Distributed systems often scale beyond single-machine limits. Understanding compute architecture is essential for designing systems that handle large-scale, compute-intensive workloads.
Batch computing enables efficient execution of tasks that require significant computational resources.
This pattern is especially useful for data-intensive applications like machine learning, big data analysis, and scientific simulations.
Thread pools and virtual machine pools
Thread pools in single-machine systems allow parallel processing of tasks.
A similar model exists in distributed systems using pools of virtual machines.
Each virtual machine acts as a worker processing tasks in parallel.
This model is known as the competing consumers pattern.
When the model works
The model works well when tasks are:
- Homogeneous
- Arrive at a steady rate
- Are independent
- Take little time to complete
When to use batch computing
Use batch computing when tasks are:
- Heterogeneous
- Require large compute resources
- Run periodically
- Are tightly coupled
- Take long to execute
Job queue and coordinator
A job queue stores job definitions.
Each job definition includes:
- Code to execute
- Required CPU
- Required memory
A coordinator service:
- Picks jobs from the queue
- Provisions virtual machines
- Assigns jobs to VMs
- Runs and monitors jobs
- Restarts failed jobs
- Tears down VMs when no jobs remain
Public cloud services
Cloud platforms offer batch computing services:
- AWS Batch
- Azure Batch
These allow running both small and large jobs.
Big compute applications
Big compute refers to high-performance computing applications.
Common use cases include:
- Big data analysis
- Machine learning model training
- Weather forecasting
- Climate simulation
- Drug discovery
- Genomic sequence analysis
Embarrassingly parallel problems
These problems can be divided into independent tasks.
Each task can be executed by a separate processor.
Examples include:
- Analyzing trading transactions
- Processing clickstream events
- Parsing application logs
- Training models for thousands of customers
Netflix uses this pattern to generate personalized homepages.
Questions
- What are the key differences between homogeneous and heterogeneous tasks?
- How does a job definition specify resource requirements?
- Why is the coordinator service critical in batch computing?
- What makes a problem embarrassingly parallel?
- How does batch computing differ from map reduce?
- What are three real-world applications of big compute?
- How does the coordinator handle job failures?
- When should batch computing be preferred over real-time processing?
Key takeaways
- Thread pools have a distributed equivalent using virtual machine pools.
- The competing consumers model works best for homogeneous, short-running tasks.
- Batch computing is ideal for heterogeneous, long-running, compute-intensive tasks.
- Job queues store job definitions with code and resource requirements.
- A coordinator provisions, assigns, and monitors jobs.
- AWS Batch and Azure Batch implement batch computing.
- Big compute handles large-scale, parallel workloads.
- Embarrassingly parallel problems are ideal for batch computing.
- Examples include ML model training, weather forecasting, and genomic analysis.