Design a social feed
Picture
When someone posts, push its ID into followers’ inboxes—or wait until a follower reads and merge followed authors’ posts.
Predict
Is fan-out-on-write always the best choice regardless of follower count?
Walk through the small example
No. Large fan-out and read frequency change the write-versus-read tradeoff.
Mental model and runnable Python model
Define whether the feed is chronological or ranked, which relationships grant visibility, and how quickly new posts/deletions must appear. Push materializes per-user inboxes and speeds reads at the cost of write amplification. Pull keeps author streams and merges at read time, trading cheaper writes for read work. A hybrid handles hot authors separately. Store post bodies separately from timeline references, use stable ordering/cursors, and perform visibility checks at the appropriate read boundary. Queue lag affects freshness and should be observable.
Requirements and API
Define chronological versus ranked order, follows, privacy, blocking, deletion latency, and freshness. POST /posts writes content; GET /feed uses a stable cursor. The user’s relationship and permission context must be trusted server-side.
Data and write flow
Store posts, follow edges, and optional per-user timeline references. A committed post emits an event; fan-out workers write idempotent (user_id, post_id) references. High-fan-out authors may bypass push and join at read time.
Read and ranking
Fetch inbox candidates, merge pull-only author streams, deduplicate, hydrate posts, check visibility, then rank or sort under a documented cursor model. Ranking changes may require snapshot/version semantics to avoid confusing page boundaries.
Failure and scale
Monitor fan-out backlog age, not just queue length. Bound retry work, tolerate duplicate jobs, and define behavior for partial timeline population. Cache hot content separately from personalized permissions; celebrity traffic needs a deliberate special path.
What the function promises: The code merges already-authorized, descending (timestamp,post_id) author streams into one chronological page. It models pull-on-read candidate assembly, not ranking or a complete social network.
from heapq import merge
def timeline(posts_by_author, followed, limit):
if not 1 <= limit <= 100:
raise ValueError("Invalid page size")
streams = [posts_by_author.get(author, [])
for author in sorted(set(followed))]
result, seen = [], set()
for post in merge(*streams, reverse=True):
if post[1] not in seen:
seen.add(post[1])
result.append(post)
if len(result) == limit:
break
return result
posts = {"Ana": [(9, "a2"), (3, "a1")], "Bo": [(8, "b1")]}
assert timeline(posts, ["Ana", "Bo"], 3) == [(9,"a2"), (8,"b1"), (3,"a1")]
assert timeline(posts, ["Ana", "Ana"], 2) == posts["Ana"]
Work and storage: Heap merge takes O(consumed candidates × log(number of streams+1)), plus stream-head storage and dedup state. Real candidate retrieval adds storage/network work.
When to recognize it: Push ordinary authors’ posts. Pull high-fan-out authors at read time, then merge candidates into one ordered page.
Check before applying it: Feed materialization must not turn stale timeline references into permanent authorization.
Change one rule
An author becomes private after their post ID was pushed into many inboxes. Is deletion from every inbox the only protection?
See what changes
No. Enforce current visibility during hydration or serving, and propagate cleanup asynchronously. The exact stale-visibility tolerance must be explicit.
Connect
Let work wait without losing its identity: Fan-out workers turn one post into many timeline references.
Design a URL shortener: Compare a personalized timeline with a simple read-heavy key lookup.