Split ownership without creating a hot corner
Picture
Replication copies the same data. Partitioning gives different keys to different owners so storage and work can be spread out.
Predict
Does consistent hashing automatically solve a single hot key?
Walk through the small example
No. One key can still concentrate work on its owner.
Mental model and runnable Python model
Choose partition keys from common queries and write distribution. Range partitioning preserves locality but can create moving hot ranges; hashing spreads keys but complicates ordered range queries. Consistent hashing reduces which key ranges change owners when nodes change, while virtual nodes can improve distribution statistically. Neither solves a hot key or cross-shard transaction by itself. Plan resharding, ownership metadata, replication per shard, and operational visibility. Python’s process-randomized hash is not a stable distributed routing choice.
What the function promises: The model uses SHA-256 tokens and deterministic virtual nodes to select an owner. It does not migrate data or coordinate membership changes.
from hashlib import sha256
from bisect import bisect_left
def stable_hash(value):
return int.from_bytes(sha256(value.encode()).digest()[:8], "big")
class HashRing:
def __init__(self, servers, virtual_nodes=16):
if not servers or virtual_nodes < 1:
raise ValueError("Need servers and virtual nodes")
self.tokens = sorted((stable_hash(f"{server}:{i}"), server)
for server in set(servers)
for i in range(virtual_nodes))
self.positions = [position for position, _ in self.tokens]
def owner(self, key):
index = bisect_left(self.positions, stable_hash(key))
return self.tokens[index % len(self.tokens)][1]
ring = HashRing(["A", "B"])
assert ring.owner("note-7") in {"A", "B"}
assert ring.owner("note-7") == HashRing(["B", "A"]).owner("note-7")
assert HashRing(["only"]).owner("anything") == "only"
Work and storage: O(Sv log(Sv)) ring construction and O(log(Sv)) lookup for S servers and v virtual nodes each.
When to recognize it: Consistent hashing assigns a key to the next token on a ring. Virtual nodes spread each server across several ranges.
Check before applying it: Balanced key ownership is not balanced traffic, and routing changes are not data migration.
Change one rule
Most requests target one celebrity’s timeline. Will more virtual nodes divide that one key’s load?
See what changes
No. Consider replication for reads, caching, splitting the logical workload, or a special hot-key path while preserving its consistency requirements.
Connect
Copies, lag, leaders, and agreement: Replicas copy a shard; sharding decides which keys that shard owns.
Estimate the bottleneck before adding servers: Measure the hottest partition, not just average cluster load.