Load balancing distributes incoming work across multiple servers, containers, queues, or services so a system stays responsive and reliable. This cheat sheet covers the main strategies used in web systems, distributed services, and cloud infrastructure. College students need it to compare algorithms, reason about tradeoffs, and connect theory with production architecture.
It is especially useful when studying scalability, fault tolerance, and performance engineering.
The core ideas are to measure load, select a target, and adapt when servers fail or traffic changes. Common policies include round robin, weighted round robin, least connections, random choice, consistent hashing, and latency-aware routing. Important formulas include utilization rho = lambda / mu, response time for an M/M/1 queue R = 1 / (mu - lambda), and weighted traffic share p_i = w_i / sum(w).
Good load balancing also depends on health checks, session affinity, backpressure, and avoiding single points of failure.
Key Facts
- Round robin sends request k to server index k mod n, which is simple but ignores differences in server capacity and current load.
- Weighted round robin assigns each server a traffic share p_i = w_i / sum(w), so stronger servers receive more requests.
- Least connections chooses the server with the smallest active connection count c_i, which works well when requests have uneven durations.
- Randomized load balancing with two choices selects two random servers and sends the request to the less loaded one, often written as choose min(load(a), load(b)).
- Consistent hashing maps both keys and servers onto a hash ring so only about 1 / n of keys move when one of n servers is added or removed.
- Queue utilization is rho = lambda / mu, where lambda is arrival rate and mu is service rate, and stable queues require rho < 1.
- For an M/M/1 queue, average response time is R = 1 / (mu - lambda), so response time grows sharply as lambda approaches mu.
- A load balancer must use health checks and remove unhealthy targets, because routing to failed servers increases errors even if the algorithm is otherwise balanced.
Vocabulary
- Load Balancer
- A component that distributes incoming requests or tasks across multiple backend servers or workers.
- Round Robin
- A load balancing policy that sends requests to servers in a fixed repeating order.
- Weighted Routing
- A routing method that sends a larger fraction of traffic to servers with higher assigned weights.
- Consistent Hashing
- A hashing technique that minimizes key movement when servers are added to or removed from a distributed system.
- Session Affinity
- A policy that keeps requests from the same user or session on the same backend server when state locality matters.
- Health Check
- A repeated test used by a load balancer to decide whether a backend server is available and safe to receive traffic.
Common Mistakes to Avoid
- Using round robin for unequal servers is wrong when machines have different capacities, because weaker servers can become overloaded while stronger servers remain underused.
- Ignoring request duration is wrong for long-lived connections, because equal request counts do not mean equal active workload.
- Forgetting health checks is wrong because a load balancer can keep sending traffic to crashed or degraded servers, causing avoidable failures.
- Using sticky sessions without a fallback plan is risky because one overloaded or failed server can trap many users on a bad target.
- Assuming higher utilization is always better is wrong because as rho approaches 1, queueing delay increases rapidly and response times can become unstable.
Practice Questions
- 1 A weighted load balancer has servers A, B, and C with weights 2, 3, and 5. What fraction of traffic should each server receive?
- 2 A service receives lambda = 80 requests per second and one server can process mu = 100 requests per second. Compute utilization rho and the M/M/1 average response time R.
- 3 Using round robin with 4 servers numbered 0 through 3, which server receives request k = 14 if the rule is server index = k mod n?
- 4 A video streaming service has users with long sessions and servers with changing connection counts. Explain why least connections may be a better strategy than simple round robin.