cconnected.
Understand the pattern. Find your own way.Find your starting point
Your learning feed/Idea 170 of 218
Ideas this builds on
Low-level design170 / 218

Iterator: reveal items one at a time

Picture

A page has two notes. Ask for the next page only when those notes have been consumed.

Predict

Does returning an iterator automatically make a remote dataset a consistent snapshot?

Walk through the small example

No. Snapshot semantics need support from the data source and query contract.

Mental model and complete Python solution

Iterator separates traversal from representation. page_items requests a page, yields its items, and follows the returned cursor. The generator stores only the current page and cursor rather than collecting all data. Its fetch collaborator can be replaced in tests. The contract requires cursors to advance and eventually terminate; otherwise malformed pagination could loop forever.

What the function promises: fetch(cursor) returns (items, next_cursor), with None marking the end. The sample uses a fixed in-memory dataset and never performs network calls.

def page_items(fetch):
    cursor = None
    while True:
        items, following = fetch(cursor)
        yield from items
        if following is None:
            return
        if following == cursor:
            raise ValueError("Pagination cursor did not advance")
        cursor = following

pages = {None: (["a", "b"], "second"),
         "second": (["c"], None)}
calls = []
def fetch(cursor):
    calls.append(cursor)
    return pages[cursor]

items = page_items(fetch)
assert next(items) == "a" and calls == [None]
assert list(items) == ["b", "c"]
assert calls == [None, "second"]

Work and storage: O(total items + page-fetch work) time and O(page size) local storage, excluding the data source.

When to recognize it: A cursor can move through a stable order, but pagination alone does not promise a snapshot while concurrent writes occur.

Check before applying it: Lazy iteration does not define ordering, snapshot consistency, or resource cleanup by itself.

Change one rule

A client stops iterating after the first result. Must every remaining page be fetched?

See what changes

No. Lazy iteration fetches only as needed. If a real fetch holds connections or streams open, also provide explicit cleanup or a context manager.

Connect

loops: A loop can consume one item at a time without knowing storage.

Make an API safe to call again: Remote cursors add ordering, failure, and consistency requirements.