System Design

Object Storage

Design an object storage service like Amazon S3 or Google Cloud Storage.

Functional requirements

• Clients create buckets and PUT, GET, HEAD, and DELETE objects by key within a bucket (e.g., photos/2026/09/cat.jpg), with user metadata and a content type.
• Clients list a bucket's objects by key prefix with a delimiter to emulate directories, paginated, in lexicographic key order.
• Objects up to 5 TB are uploaded via multipart upload: initiate, upload parts in any order and in parallel, then complete or abort.
• Overwriting a key replaces the object atomically (readers see the old or the new object, never a mix), and buckets can enable versioning to keep prior versions.
• Access is private by default with per-bucket and per-object permissions and time-limited pre-signed URLs, and every object carries a checksum verified on upload and download.

Non-functional requirements

• Durability of 99.999999999% (11 nines) per object per year: an acknowledged PUT survives the simultaneous loss of two storage nodes or one entire rack without data loss.
• Strong read-after-write for a single key: a GET after a successful PUT, overwrite, or DELETE never returns the prior state, and never a partially written object; bucket listings may lag a completed write by a few seconds.
• Availability of 99.99% for reads and 99.9% for writes; p99 time-to-first-byte under 100ms for objects up to 1 MB and at least 100 MB/s sustained per connection for large objects.
• Lost or corrupted copies are detected and re-created in the background within hours, and bit-rotted data is never returned: checksums are verified on every read.
• Every request is authenticated and authorized, data is encrypted in transit and at rest, and tenants are isolated so no bucket operation can read another tenant's data.

Scaling & constraints

• 500 PB across 10 trillion objects, so the average object is 50 KB while the median is ~5 KB; 90% of bytes are in objects over 100 MB.
• ~5 million requests/sec at peak with a GET:PUT:LIST mix of ~80:15:5; ~150 GB/s ingress and ~400 GB/s egress.
• Metadata per object is ~500 bytes (key, size, checksum, version, placement, ACL pointer, timestamps): ~5 PB of index, growing ~40% per year.
• Key skew: 1% of buckets hold 60% of objects, the largest bucket holds 200 billion objects, and LIST traffic on one hot bucket can reach 10,000/sec.
• 100,000 storage nodes across 3 zones and ~1,000 racks; ~2% annual disk failure rate (~50 disk failures per day), and a rack loses power a few times a year.
• A single object can go viral at 500,000 GET/sec for minutes; multipart parts are 5 MB to 5 GB, with ~10,000 uploads in flight at any time, ~5% of them abandoned.

Out of scope

• Cross-region replication and geo-redundant buckets.
• Lifecycle policies, storage tiering, and archival classes.
• Static website hosting, CDN integration, and event notifications.
• Billing, quotas, and IAM policy language beyond bucket and object permissions.