Reservoir sampling: choose from a stream
Picture
Items arrive one by one and cannot all be stored. We still want each final item equally likely.
Predict
Must the total stream length be known beforehand?
Walk through the small example
No. Replacement probability maintains equal inclusion probability after every prefix length.
Mental model and complete Python solution
The reservoir makes a promise after every prefix: each item seen so far is equally likely to be stored. The next replacement probability preserves that promise.
What the function promises: Return one item from a nonempty iterable using an injected random generator supporting randrange. Raise ValueError for empty input.
from random import Random
def reservoir_one(items, random_source):
chosen = None
count = 0
for count, item in enumerate(items, 1):
if random_source.randrange(count) == 0:
chosen = item
if count == 0: raise ValueError("Empty stream")
return chosen
assert reservoir_one([10], Random(1)) == 10
assert reservoir_one(range(20), Random(7)) in range(20)
Work and storage: O(n) time, O(1) sample storage, and one random draw per item.
When to recognize it: Replacement probability maintains equal inclusion probability after every prefix length.
Check before applying it: Randomness quality matters, and weighted sampling needs a different rule.
Change one rule
Need k uniform items without replacement.
See what changes
Keep a reservoir of size k; item i>k replaces a uniformly selected reservoir slot with probability k/i.
Connect
invariants: After i items, each has probability exactly 1/i.
choices: The random choice probability changes with the prefix length.