System Design

Session Store

Design the login session store for a large web application like GitHub.

Functional requirements

• Create a session on successful login and return an opaque session ID for the client to store as a cookie.
• Validate a session ID on any authenticated request, returning the user ID and roles, or a clear expired or invalid answer.
• Extend a session's expiry on activity (sliding renewal), up to a fixed absolute lifetime after which the user must log in again.
• Invalidate one session on logout, and invalidate every session belonging to a user on password change or 'log out everywhere'.
• Let a user see their active sessions with device and last-seen time and revoke any one of them (e.g., the security page on GitHub).

Non-functional requirements

• Validation runs before essentially every authenticated request: p99 under 5ms measured inside the data center.
• Revocation cannot be eventually consistent: after logout or a password change the session must stop validating everywhere within 1 second, while sliding-expiry updates may be several minutes stale.
• Availability of the read path is critical, since a store outage signs out every user at once; validation must survive the loss of any single node and fail closed on an unknown session.
• Sessions are soft state and a total loss only forces users to log in again, but losing every session at once is a customer-visible outage, so a node restart must not empty the store.
• Session IDs carry at least 128 bits of cryptographic randomness, travel only over HTTPS cookies marked HttpOnly, Secure and SameSite, and are stored hashed rather than raw.

Scaling & constraints

• 50 million registered users, about 5 million sign-ins a day creating new sessions, and roughly 20 million unexpired sessions live at any moment (5 million a day times a mean session life of about 4 days), spread over the 3 to 4 devices an active user keeps signed in.
• Peak load is 200,000 validations per second from roughly 1 million sessions in active use at once (about one authenticated request every 5 seconds each) against 200 sign-ins per second, a 1000:1 read to write ratio, served by 40 app servers; the identity service on the login path has p99 latency of 800ms.
• A session record (session ID, user ID, issued-at and expires-at, device fingerprint, and a short list of roles) is about 400 bytes.
• Sessions expire 3 days after the last request with a hard 14-day absolute lifetime, and session volume grows about 25% per year.
• Traffic is skewed: the busiest 1% of the sessions in active use produce 20% of validations, and one shared session (a wall display, a kiosk) can reach 20,000 validations per second on a single key.
• Revocations (logout, password change, log out everywhere) run about 30,000 per day, spiking to 200 per second in a forced password reset.

Out of scope

• Password storage, credential verification, and multi-factor authentication; the identity service is a black box with a stated latency.
• OAuth and single sign-on federation with third-party identity providers.
• Authorization policy and permission modeling beyond returning a role list.
• Bot detection, account takeover scoring, and CAPTCHA flows.