Load balancer

A load balancer sits in front of several servers and distributes incoming requests between them, checking which are healthy and sending traffic only to those.

It does two jobs that are easy to conflate. It spreads load, which is how a system serves more traffic than one machine can. And it removes the single point of failure, which is how a system survives a machine dying — including on purpose, during a deployment.

The consequence for the application is that any request may land on any server, so nothing important can be kept in one server's memory. Sessions, uploads and caches move to shared storage, which is a design decision best made before the second server exists rather than after.