Caching in System Design Interviews: What to Say When

When an interviewer asks about caching, they want to see that you can name a specific place to put the cache, pick a read/write strategy, and reason about what happens when the cache is stale, cold, or full. The strong answer sounds like: "I'll put a cache-aside layer in front of the read path, key by user ID, set a 5 minute TTL, and use LRU eviction; on a cache miss I read from the database and backfill." The weak answer is "we can add caching to make it faster." This guide covers exactly what to say and when, so caching becomes a scoring signal instead of a throwaway line.

Caching is one of the highest-leverage topics in a system design round because it forces you to talk about latency budgets, consistency trade-offs, and failure modes all at once. It also comes up in nearly every design prompt: news feeds, URL shorteners, rate limiters, and product catalogs all lean on caches. If you want targeted reps, our system design and coding practice runs the same follow-ups a real interviewer would.

Start with why: the latency gap

The reason caching exists is the enormous speed gap between memory, disk, and network. Reading from RAM takes on the order of 100 nanoseconds; reading from a local SSD is roughly 100 microseconds (about 1,000x slower); a network round trip to another datacenter is measured in milliseconds. That gap is why an in-memory cache sitting next to your service can turn a 50ms database query into a sub-millisecond lookup.

Say this out loud early. It tells the interviewer you cache because of a measured latency budget, not out of habit. A clean framing: "Our p99 read latency target is under 100ms, the primary datastore gives us 30 to 50ms per query under load, and reads outnumber writes maybe 100 to 1. That read-heavy skew is exactly what a cache is good for." Now every later decision has a reason behind it.

Where the cache lives

Before you pick a strategy, name the layer. Interviewers want to know you understand there are several distinct places to cache, each with different trade-offs.

Client / browser cache: cheapest, but you control invalidation poorly. Good for static assets.
CDN / edge cache: serves static content and cacheable API responses close to the user. Bring this up for anything with global reads (images, video, public pages).
Application / in-memory cache: a local map inside the service process. Fastest, but not shared across instances and lost on restart.
Distributed cache: a shared tier like Redis or Memcached that all app servers hit. This is the workhorse answer for most designs.
Database cache: the buffer pool and query cache inside the DB itself. Usually you mention it but don't rely on it.

For most interviews the interesting decision is the distributed tier. The default pairing to name is Redis or Memcached. Memcached is a simpler, multithreaded, pure key-value store; Redis adds rich data structures (sorted sets, hashes, streams), persistence options, and replication. If the prompt needs leaderboards, rate limiting counters, or pub/sub, say Redis and justify it with the data structure you need. If it is a plain string cache and you want raw simplicity, Memcached is a defensible call.

Read strategies: cache-aside is your default

There are three read patterns worth knowing, but cache-aside (also called lazy loading) is the one to lead with because it is the most common in production.

Cache-aside. The application checks the cache first. On a hit, return it. On a miss, read the database, write the result back into the cache, and return. The cache never talks to the database directly. This keeps the cache optional: if it goes down, you still serve reads (slower) from the DB. The downside is the first request for any key always misses, and stale data can linger until the TTL expires.

Read-through. The cache itself loads from the database on a miss, so the application only ever talks to the cache. Cleaner application code, but you need a cache that supports it and you couple more tightly to the cache being up.

Write strategies pair with these:

Strategy	What happens on write	Trade-off
Write-through	Write to cache and DB synchronously	Consistent cache, higher write latency
Write-back (write-behind)	Write to cache, flush to DB async	Fast writes, risk of data loss on crash
Write-around	Write to DB only, cache filled on read	Avoids caching write-once data, more read misses

The judgment call to voice: "I'll use cache-aside with write-around for this catalog because items are written rarely and read constantly, so I don't want to pollute the cache on every write." That one sentence shows you matched the strategy to the access pattern instead of reciting definitions.

Eviction and TTL: what to say when the cache fills up

A cache has finite memory, so you must answer: when it is full, what gets thrown out? The standard policies are LRU (evict least recently used), LFU (evict least frequently used), and FIFO. LRU is the sensible default and the one to name first. Reach for LFU when you have a stable set of hot keys that should survive bursts of one-off requests that would otherwise flush them under LRU. Redis exposes both as configurable eviction policies (for example allkeys-lru and allkeys-lfu), plus a noeviction mode that returns errors on write once memory is full, which is a real gotcha worth mentioning.

Separately from eviction, set a TTL (time to live) on entries so stale data self-corrects even if nothing evicts it. Pick a number and justify it: "5 minute TTL on the profile cache because we tolerate 5 minutes of staleness there, but 30 seconds on inventory counts because overselling is expensive." Interviewers reward a specific TTL tied to a business tolerance far more than "we'll expire it eventually."

This is also where a coding round can intersect with system design. Implementing an LRU cache from scratch (a hash map plus a doubly linked list for O(1) get and put) is a classic coding question, and memory-management problems like DevInterview's Design Memory Allocator exercise the same muscle of tracking, allocating, and freeing fixed capacity. If you can code the eviction logic, explaining it at the system level comes easily.

Invalidation and the failure modes that separate candidates

Phil Karlton's line that cache invalidation is one of the two hard problems in computer science exists for a reason. The moments where candidates win or lose points are the failure modes, so raise them before the interviewer has to.

Cold start / cache miss storms. When the cache is empty (fresh deploy, flush), every request hits the DB at once. Mitigate with cache warming or gradual rollout.
Thundering herd / cache stampede. A single hot key expires and thousands of concurrent requests all miss and hit the DB simultaneously. Mitigate with a mutex or single-flight lock so only one request recomputes, or with staggered/jittered TTLs.
Stale reads. With cache-aside plus TTL, you accept bounded staleness. If you cannot, use write-through or explicit invalidation on write. State the consistency model you are choosing.
Hot keys. One celebrity user or viral item overwhelms a single cache shard. Mention replication of hot keys or client-side caching for the top N.
Cache penetration. Requests for keys that do not exist bypass the cache every time and hammer the DB. Cache the negative result or use a Bloom filter.

Naming even two of these unprompted signals senior-level thinking. For more on fielding the deeper probes that follow, see our guide on handling follow-up questions.

A script you can adapt

Put it together and your caching segment should sound roughly like this:

"Reads dominate here about 100 to 1, and my read latency target is under 100ms while the DB gives 40ms, so I'll add a Redis distributed cache using cache-aside. Keys are user:{id}:feed, values are the serialized feed, TTL is 60 seconds. Eviction is allkeys-lru since we have more keys than memory. Writes use write-around and invalidate the specific key. My two risks are stampede on hot keys, which I'll solve with a single-flight lock, and cold start after deploy, which I'll handle by warming the top accounts."

That is 30 seconds of speech that touches placement, strategy, keying, TTL, eviction, invalidation, and two failure modes. It is the difference between "mentioned caching" and "designed a cache."

FAQ

Do I need to memorize exact latency numbers?

Know the orders of magnitude: RAM in nanoseconds, SSD in microseconds, cross-datacenter network in milliseconds. You do not need precise figures, but being able to say "memory is roughly 1,000x faster than SSD" justifies why the cache exists and anchors your latency math.

Redis or Memcached, which should I say?

Default to Redis unless the prompt is a trivial string cache. Redis gives you data structures, persistence, and replication that most designs eventually need. Choose Memcached only when you specifically want a simpler, multithreaded, pure key-value store and can say why.

When is cache-aside the wrong choice?

When you cannot tolerate any staleness or when the first-request miss latency is unacceptable. In those cases prefer write-through for consistency or read-through with cache warming to avoid cold misses. Always state the consistency trade-off you are accepting.

How do I bring up caching without being told to?

Once you establish a read-heavy access pattern and a latency target, propose the cache as the fix for that specific bottleneck. Tie it to numbers you already stated so it reads as a deliberate decision, not a reflex.