Load Balancing
Spread traffic across healthy servers so no single machine becomes the bottleneck.
A load balancer sits in front of a group of servers and decides which one handles each request. It is what makes horizontal scaling and zero-downtime deployments possible.
It also removes unhealthy servers from rotation using health checks, so users are routed around failures automatically.
Diagram
How it works
- The client resolves the domain to the load balancer’s address.
- The load balancer terminates TLS (at layer 7) and picks a backend using its algorithm.
- Health checks run every few seconds; failing servers are removed from the pool.
- The backend responds through the load balancer, which can add headers such as
X-Forwarded-For.
Key concepts
- Round robin
- Each server in turn. Simple and fair when servers and requests are similar.
- Weighted round robin
- Bigger servers receive a larger share of requests.
- Least connections
- Send to the server with the fewest active connections. Good for long or uneven requests.
- IP / consistent hash
- The same client or key goes to the same server — useful for caches and sticky workloads.
- Layer 4 vs Layer 7
- L4 routes by IP and port (fast). L7 reads HTTP — paths, headers, cookies — for smarter routing and TLS termination.
Trade-offs
Benefits
- Horizontal scaling and high availability
- Rolling and blue-green deployments
- Central place for TLS, compression and basic protection
Costs
- The load balancer itself must be redundant
- Sticky sessions undermine even distribution
- Adds a network hop
Use it when
- Any service running more than one instance
- Routing different paths to different services (L7)
Common mistakes
- Health checks that only test “process is running”, not “can serve requests”
- Sticky sessions hiding a stateful application
- Forgetting connection draining, so deploys cut off in-flight requests
Interview questions
Q1When would you pick least-connections over round robin?
When request durations vary a lot, such as file uploads or long-polling. Round robin could pile slow requests onto one server.
Q2How do you avoid the load balancer becoming a single point of failure?
Run it as a redundant pair or use a managed cloud load balancer that is highly available across zones, with DNS or anycast in front.
Q3What is the difference between a liveness and a readiness check?
Liveness asks “is the process alive or should it be restarted?”. Readiness asks “can it serve traffic now?”. Load balancers should use readiness.
Related lessons
Scalability & Performance
Vertical vs Horizontal Scaling
Make one machine bigger, or add more machines — the first decision in every scaling story.
APIs & SecurityAPI Gateway
A single front door that routes, secures and shapes traffic to your back-end services.
Cloud & DeliveryDeployment Strategies
Rolling, blue-green, canary and feature flags — release without downtime and roll back fast.