Separate metadata from large bytes
Picture
Store a video as an object. Keep its owner, status, size, and object key in a database that can answer application queries.
Predict
Does possession of an object hash automatically grant permission to read it?
Walk through the small example
No. Integrity or identity is not authorization.
Mental model and runnable Python model
Use object storage for large opaque bytes and a metadata database for ownership, lifecycle, searchable attributes, and completion state. CDN caching reduces origin traffic and geographic latency for suitable objects, with explicit cache headers and invalidation/versioning. Multipart upload separates large transfers into retryable pieces; abandoned sessions and orphaned objects need cleanup. Signed URLs are bearer capabilities that should be scoped and time-limited, not exposed as permanent public identifiers for private assets. The course itself uses D1 notebook records and R2 assets as a concrete separation, without claiming those services are interchangeable with every storage workload.
What the function promises: The model stores bytes in an in-memory content-addressed dictionary and creates a separate metadata record. It performs no upload or authorization.
from hashlib import sha256
def store_object(objects, metadata, owner, data):
key = sha256(data).hexdigest()
objects.setdefault(key, bytes(data))
record = {"owner": owner, "key": key, "size": len(data), "status": "ready"}
metadata.append(record)
return record
objects, metadata = {}, []
a = store_object(objects, metadata, "Ana", b"hello")
b = store_object(objects, metadata, "Bo", b"hello")
assert a["key"] == b["key"] and len(objects) == 1
assert a["owner"] != b["owner"]
Work and storage: O(byte count) hashing and storage. Identical-content deduplication may create privacy/accounting considerations beyond this example.
When to recognize it: Cache suitable bytes near readers. Private assets still need access checks or bounded signed access; public caching rules must not leak private content.
Check before applying it: Object identity, cacheability, and authorization are separate concerns.
Change one rule
Metadata commits but the byte upload failed. Should the object appear ready?
See what changes
No. Use pending/uploading/verified/ready states and expose only completed objects. Define orphan cleanup for the opposite failure order too.
Connect
Choose storage from reads and writes: Keep searchable ownership and lifecycle facts in metadata storage.
Design resumable file uploads: Multipart uploads coordinate byte completion with metadata readiness.