System Design

Package Registry

Design a package registry that serves library downloads, like npm or PyPI.

Functional requirements

• Publish a new version of a package by uploading a tarball plus its metadata (e.g., lodash@4.17.21); once published, a version is immutable and is never overwritten.
• Resolve a package name and a version range (e.g., lodash@^4.17.0) to one concrete version, returning its download URL and integrity hash.
• Serve package tarballs for download at high volume, with the bytes verifiable by the client against the hash recorded at publish time.
• Yank a bad version so it is skipped by new range resolutions, while a build that already pins that exact version can still download it.
• Track and expose a download count per package, like the weekly download number shown on a package page.

Non-functional requirements

• Metadata resolution is on the critical path of every install: p99 under 100ms for a name plus version-range lookup.
• Tarball downloads go through a CDN with p99 time to first byte under 200ms on a cache hit, and reads must keep working while publishing is degraded.
• Availability over strict consistency for reads: a new version may take a few seconds to become visible everywhere, but a version that already resolves must never point at different bytes.
• Publishing is strongly consistent for uniqueness: a name and version can be published exactly once, a retry must not create a second artifact, and only an authenticated owner may publish or yank.
• Published tarballs and metadata are durable for the life of the registry, since losing one permanently breaks every build pinned to it; download counts are best effort and may undercount.

Scaling & constraints

• 3M package names holding 40M published versions in total, growing by 40,000 new versions per day.
• 2B tarball downloads per week, about 3,300 per second on average and 10,000 per second at weekday peak.
• Read:write ratio is roughly 7,000:1 (tarball downloads vs. the 40,000 publishes a day), with at least one metadata resolution on top of every download.
• The average tarball is 250KB and the p99 tarball is 10MB; the metadata document for one package name averages 20KB.
• Traffic is heavily skewed: the top 1% of packages take about 80% of downloads, and one popular package can be requested 2,000 times per second during CI peaks.
• Every version is retained forever and is never deleted, so total tarball storage grows by about 35% per year at the current publish rate, and a CI client retries a failed download up to 3 times.

Out of scope

• Account signup, token issuance, and organization permission models; assume a publish or yank request arrives with an already verified owner identity.
• Malware and typosquatting scanning of published packages.
• Full transitive dependency solving and lockfile generation on the client.
• Private registries, mirroring, and billing.