Share traffic across replaceable workers
Picture
A load balancer chooses a healthy worker for each request. More workers can share traffic, but they need the state required to serve any user.
Predict
Does adding app workers automatically fix a saturated database?
Walk through the small example
No. More callers can increase pressure on the same bottleneck.
Mental model and runnable Python model
A stateless application worker keeps no exclusive durable session state needed for future requests. A load balancer can then route to interchangeable healthy workers. L4 and L7 balancing see different protocol information; choose based on routing requirements, not labels alone. Health checks should reflect readiness without causing unnecessary cascading removal. Drain in-flight work during deployment and plan behavior when capacity drops. Autoscaling needs measured signals and startup time; it is not instantaneous spare capacity.
What the function promises: RoundRobin selects from an explicit current healthy-worker list. It is a routing model only and performs no network calls or health probes.
class RoundRobin:
def __init__(self):
self.next_index = 0
def choose(self, healthy):
if not healthy:
raise RuntimeError("No healthy workers")
chosen = healthy[self.next_index % len(healthy)]
self.next_index += 1
return chosen
router = RoundRobin()
assert [router.choose(["A", "B"]) for _ in range(4)] == ["A","B","A","B"]
assert router.choose(["B"]) == "B"
Work and storage: O(1) selection given a prepared healthy list. Real balancing includes connection and health-state management.
When to recognize it: Round-robin is simple, but readiness checks, connection draining, slow workers, and uneven request costs affect real routing.
Check before applying it: A load balancer spreads traffic; it does not remove shared-state or downstream bottlenecks.
Change one rule
One request takes a millisecond and another takes a minute. Does equal request count mean equal load?
See what changes
No. Consider active work, queueing, or resource-aware routing and separate workloads when necessary. Measure the resource that is actually constrained.
Connect
Follow one request across the network: Follow the request into a chosen healthy worker.
Split ownership without creating a hot corner: Split key ownership only when shared storage or work also needs distribution.