Home › Scenarios › Multi-region failover
System design scenario
Design multi-region failover
Advanced
Two regions share the load behind a global load balancer. At t = 15 s, us-east goes dark for 20 s. Health checks move its traffic to us-west, and us-west has to survive the double load.
▶ Simulate this designThe goal
Keep p99 < 200 ms and error rate < 1.00% at 300,000 req/s for 25s.
- Peak load
- 300,000 requests per second
- Traffic mix
- 90% reads, 10% writes
- p99 latency
- under 200 ms
- Error rate
- under 1.00%
- Chaos
- us-east outage at 15 s for 20 s
The starting architecture
The simulator opens with this design. It is deliberately too weak for the goal: run it and watch which component saturates first.
- Global LB (Load Balancer, 2 instances): Global load balancer with health checks: shifts traffic away from a region that goes dark.
- us-east LB (Load Balancer, 2 instances): Regional load balancer in us-east.
- us-east API (Service, 40 instances): API fleet in us-east. Must be able to absorb all traffic if us-west fails.
- us-east DB (Database): Regional database with read replicas in us-east.
- us-west LB (Load Balancer, 2 instances): Regional load balancer in us-west.
- us-west API (Service, 40 instances): API fleet in us-west. Must be able to absorb all traffic if us-east fails.
- us-west DB (Database): Regional database with read replicas in us-west.
How to approach it
The hints unlock one at a time in the app. Try the design yourself before reading them.
Show the hints
- Run it and watch us-west when us-east fails: failover only works if the survivor has room.
- N+1 at the region level: each region must carry 100% of the load on its own.
- Remember the database: replicas must absorb the doubled read traffic too.
What the simulator shows you
SimLoad models every component as a queue with real capacity, latency, timeouts and retries, so you see utilisation, p50/p99 latency, errors and monthly cost change as you scale. Insights name the bottleneck in plain language, and the capacity planner sizes the fix. A reference solution that meets the goal is verified by automated tests.
▶ Try it in the simulator