30-second elevator pitch: "Real-time updates require efficient push channels. I choose based on latency needs: polling for 2-5 second updates, long polling or SSE for near real-time, WebSockets when I need bidirectional low latency. The second hop is Pub/Sub or consistent hashing to fan out updates from the source to servers."

The Problem

Consider a collaborative document editor like Google Docs. When one user types a character, all other users viewing the document need to see that change within milliseconds. In apps like this you cannot have every user constantly polling the server for updates every few milliseconds without crushing your infrastructure.

The core challenge is establishing efficient, persistent communication channels between clients and servers. Standard HTTP follows a request-response model: clients ask for data, servers respond, then the connection closes. This works great for traditional web browsing but breaks down when you need servers to proactively push updates to clients.

These problems are often solved once by a specialized team, so many candidates have never built them. Do not worry. This pattern covers what you need to make great decisions in your interview.

9 problems that use this pattern: Ticketmaster, Uber, WhatsApp, Robinhood, Google Docs, Strava, Online Auction, FB Live Comments, and similar collaborative or live systems.

What You Will Learn

Hop 1: Client-Server Connection

- Networking 101 (OSI layers, TCP/UDP, L4 vs L7 load balancers)

- Simple polling (baseline)

- Long polling (latency trade-off)

- Server-Sent Events (SSE) - chunked streaming, EventSource, last event ID

- WebSockets - full-duplex, HTTP upgrade, L4 requirement, deployment

- WebRTC - peer-to-peer, STUN/TURN, signaling

>

Hop 2: Source to Server

- Pull polling

- Push via consistent hashing (ZooKeeper)

- Push via Pub/Sub (Kafka, Redis)

>

Deep Dives (what interviewers ask next)

- Connection failures and reconnection

- Celebrity problem (millions of followers)

- Message ordering across distributed servers

The Solution: Two Hops

Real-time systems require two distinct pieces: how updates get from the server to the client (Hop 1), and how updates get from the source to the server (Hop 2). The diagram below illustrates both hops: clients connect via polling, long polling, SSE, or WebSockets to endpoint servers; those servers receive updates from the source via pull polling, consistent hashing, or Pub/Sub.

What interviewers want to hear: "I start with the simplest approach that meets the latency requirement. Many problems do not need WebSockets - polling every 2-5 seconds is fine. If we need sub-second updates, I use long polling or SSE for one-way, WebSockets for bidirectional. The second hop is Pub/Sub or consistent hashing to distribute updates from the source."

Networking 101

Before diving into protocols, a quick primer on how networking works. Networks use a layered architecture (the OSI model). Three layers matter most in system design interviews:

Network Layer (Layer 3) - IP handles routing and addressing. It breaks data into packets and provides best-effort delivery. No guarantees: packets can be lost, duplicated, or reordered.

Transport Layer (Layer 4) - TCP and UDP provide end-to-end communication. TCP is connection-oriented: you establish a connection first, then data is delivered correctly and in order. TCP connections take time to establish and resources to maintain. UDP is connectionless: you can send to any IP without setup, but there are no delivery or ordering guarantees.

Application Layer (Layer 7) - HTTP, DNS, WebSockets, WebRTC. These build on TCP to provide application-level abstractions.

Request lifecycle. When you type a URL and press Enter: DNS resolves the domain to an IP. The client initiates a TCP connection via a three-way handshake: SYN, SYN-ACK, ACK. Once connected, the client sends an HTTP GET. The server responds. The connection closes via a four-way teardown: FIN, ACK, FIN, ACK. Each round trip adds latency. The TCP connection represents state that both sides must maintain.

With load balancers. In practice, clients talk to load balancers, which distribute to backend servers. Two types matter:

Layer 4 load balancers operate at TCP/UDP. They route by IP and port only, without inspecting packet content. Effectively, it is as if the client has a direct TCP connection to one backend. L4 maintains persistent TCP connections. Fast and efficient. Good for protocols that need persistent connections like WebSockets.

Layer 7 load balancers operate at the application layer. They understand HTTP. They can route by URL, headers, cookies. They terminate incoming connections and create new ones to backends. More CPU-intensive. The underlying TCP connection to your server is just a transport; L7 does not guarantee the same backend for every request. L7 is better for HTTP-based solutions like long polling.

The choice between L4 and L7 comes up when discussing real-time features. L4 is better for WebSockets. L7 offers flexibility for HTTP-based approaches.

Hop 1: Client-Server Connection Protocols

We break down each protocol with its trade-offs. As a motivating example: a chat app where users need new messages sent to their room.

Simple Polling: The Baseline

The simplest approach: the client makes an HTTP request at a regular interval. The server responds with the current state. For chat, the client polls for "what messages have I not received yet?"

How it works (plain English): The client calls fetch to /api/updates every N seconds (e.g., 2 seconds). It parses the JSON response and updates the UI. No special setup.

Advantages: Simple to implement. Stateless. No special infrastructure. Works with any standard networking. Quick to explain in an interview.

Disadvantages: Higher latency (up to polling interval plus request processing). Limited update frequency. More bandwidth. Resource-intensive with many clients (new connections, etc.).

When to use: If updates every 2-5 seconds are acceptable, simple polling is a great baseline. Also good when the update window is short. Propose it to your interviewer before jumping to WebSockets. Say: "I will start with simple polling to focus on X, but we can switch if we need more sophistication."

Interview tip: Use HTTP keep-alive. Setting a keep-alive longer than the polling interval means you establish the TCP connection once, minimizing setup and teardown overhead.

Long Polling: The Easy Solution

The client makes a request and the server holds it open until new data is available. When data arrives, the server responds, the client immediately makes a new request, and the cycle repeats. For chat, the client requests the next message; if none exists, the server holds the request until one is sent.

How it works (plain English): The client enters a loop. It fetches /api/updates. The server blocks until new data or a timeout. The client receives the response, processes it, and immediately fetches again. On error, it waits briefly and retries.

Latency trade-off: Assume 100ms round-trip latency. Two updates occur 10ms apart. With long polling, the first update arrives ~100ms after it occurred. The second can arrive up to ~290ms later: 90ms for the first response to return, 100ms for the second request to reach the server, 100ms for the response. High-frequency updates suffer from this call-back latency.

Advantages: Builds on standard HTTP. Works everywhere HTTP works. Easy to implement. No special infrastructure. Stateless server-side.

Disadvantages: Higher latency than SSE or WebSockets. More HTTP overhead. Resource-intensive with many clients. Not suitable for frequent updates. Makes monitoring harder (long-hanging requests). Browsers limit concurrent connections per domain.

When to use: Near real-time with simple implementation. Good when updates are infrequent. Ideal for long async processes like payment processing: long-poll for payment status until the job finishes, then show the success page.

Interview tip: Keep timeouts consistent across the stack. If your load balancer times out at 30 seconds, do not configure long-poll for 60 seconds. 15-30 seconds is common and minimizes fuss.

Server-Sent Events (SSE): The Efficient One-Way Street

SSE extends long polling by letting the server send a stream of data. Normal HTTP uses Content-Length. SSE uses Transfer-Encoding: chunked - the client does not know how many chunks or how big they are until they arrive. The server sends a chunk, keeps the connection open, and sends more as needed. Perfect when servers push to clients but clients rarely need to send back.

How it works (plain English): The client creates an EventSource pointing to /api/updates. When the server has new data, it writes chunks in a specific format. The client's onmessage handler parses the JSON and updates the UI. On the server, you set headers for text/event-stream, no-cache, keep-alive. You write data as formatted lines. When the client disconnects, you remove listeners and clean up.

Reconnection: The SSE standard includes a "last event ID." If the client loses the connection, it reconnects and sends the last event ID. The server can send all events that occurred while the client was disconnected. The browser EventSource handles this automatically.

Advantages: Built into browsers. Automatic reconnection. Works over HTTP. More efficient than long polling (less connection churn). Simple to implement.

Disadvantages: One-way only. Some proxies and load balancers do not support streaming and buffer the response, which blocks the stream and is hard to debug. Browsers limit concurrent connections per domain. Long-lived requests complicate monitoring.

When to use: Great upgrade from long polling when you need higher update frequency without call-back latency. Popular for AI chat apps that stream tokens as they are generated. Be aware that some infrastructure does not support streaming.

WebSockets: The Full-Duplex Champion

WebSockets provide true bidirectional communication. If you have high-frequency reads and writes, WebSockets are the default choice. They build on HTTP through an "upgrade" handshake: an existing TCP connection switches from HTTP to WebSocket. You can reuse cookies and headers for authentication.

How it works (plain English): The client opens a WebSocket to ws://api.example.com/socket. The connection upgrades from HTTP. Both sides send and receive messages (opaque blobs - JSON, Protobuf, whatever). The client's onmessage handler parses and updates. On close, the client implements reconnection (e.g., retry after 1 second). On the server, you create a WebSocket server, accept connections, and forward messages from your data source to each client.

Infrastructure challenges: WebSockets need persistent connections. L7 load balancers often do not preserve connection stickiness. L4 load balancers do - the same TCP connection hits the same backend. For WebSockets, prefer L4.

Deployments: When servers restart, you either sever old connections and have clients reconnect, or have new servers take over. Prefer severing and reconnecting - simpler. You must handle clients that reconnect and may have missed updates.

Load balancing: Long-running connections mean you "stick" to each allocation. A load balancer cannot send a new request to a different server without breaking the WebSocket. Many architectures terminate WebSockets into a WebSocket service early. That service handles connection management and scaling; the rest of the system stays stateless. The WebSocket tier changes less often, so fewer deployments churn connections.

Advantages: Full-duplex. Lower latency than HTTP (no per-message headers). Efficient for frequent messages. Wide browser support.

Disadvantages: More complex. Requires special infrastructure. Stateful connections complicate load balancing and scaling. Must handle reconnection.

When to use: High-frequency bidirectional communication. A common pattern is SSE for updates plus HTTP POST for writes - only use WebSockets when you need both directions at high frequency. Defer to SSE unless you have a clear need for bidirectional.

Interview tip: Discuss how you manage connections, handle reconnections, and handle server restarts. For scaling, use "least connections" load balancing. Offload heavy processing to other services so WebSocket servers stay lightweight.

WebRTC: The Peer-to-Peer Solution

WebRTC enables direct peer-to-peer communication between browsers. Ideal for video/audio calls and some data sharing like document editors. Clients talk to a central "signaling server" that tracks peers and their connection info. Once a client has another peer's info, it tries to establish a direct connection. Most clients block inbound connections (NAT). WebRTC addresses this with:

STUN - Session Traversal Utilities for NAT. Techniques like "hole punching" let peers establish publicly routable addresses and ports. Peers create open ports and share them via the signaling server.

TURN - Traversal Using Relays around NAT. A relay service bounces traffic through a central server to reach the peer when STUN fails.

How it works (plain English): A client creates an RTCPeerConnection, gets the local media stream, adds tracks to the connection, creates an offer, sets it as the local description, and sends the offer to the signaling server. The signaling server is lightweight; most traffic flows peer-to-peer.

The signaling server itself is a real-time system - it needs WebSockets, SSE, or similar so clients can find peers and exchange connection info.

Advantages: Direct peer communication. Lower latency. Reduced server costs. Native audio/video support.

Disadvantages: Complex setup. Requires signaling server. NAT/firewall issues. Connection setup delay.

When to use: Video/audio calls, screen sharing, gaming. Can reduce server load when clients talk frequently to each other - some collaborative document editors use WebRTC for presence and pointer sharing so participants stream cursor position directly to peers instead of routing through the server.

When NOT to use: Overkill for most real-time update use cases. Only use when the problem clearly requires peer-to-peer.

Protocol Choice Flowchart

Not latency sensitive: Simple polling. Start here unless you have a specific need.

Do not need bidirectional: SSE. Lightweight, works for many use cases.

Need frequent bidirectional: WebSocket.

Audio/video or peer-to-peer collaboration: WebRTC.

Hop 2: Source to Server

Updates must get from wherever they are produced (other users, drivers, services) to the servers that hold client connections. Three patterns:

Pulling with Simple Polling

A pull-based model. The server (or clients via the server) polls a database for updates. For chat, clients poll for messages with timestamp greater than the last received. The poll itself is the trigger, even though the update occurred earlier.

Advantages: Dead simple. State lives in the database. No special infrastructure. Quick to explain.

Disadvantages: High latency. Excess database load when updates are infrequent but polling is frequent.

When to use: When responsiveness matters but real-time is not the main concern. If you need true real-time, this is not the best approach. Watch out: a million clients polling every 10 seconds is 100k TPS of read volume.

Pushing via Consistent Hashing

With long polling, SSE, or WebSockets, clients have persistent connections to specific servers. To send a message to User C, you must know which server they are on. Hashing solves this: hash the user ID and assign them to one server. A central service (e.g., ZooKeeper or Etcd) tracks server count N and addresses. User u goes to server (u mod N). Clients either connect directly to the right server (if they know the mapping) or connect to any server and get redirected. Once connected, that server holds the connection in a map. When an update arrives for User C, you hash their ID, send to the right server, which looks up the connection and forwards the message.

Consistent hashing improves scaling. With simple modulo, changing N forces almost all users to reconnect. Consistent hashing maps servers and users onto a hash ring. Each user connects to the next server clockwise. When you add or remove servers, only users in the affected ring segments move. Minimal connection churn.

Advantages: Predictable server assignment. Minimal disruption when scaling. Works well with stateful connections. Easy to add/remove servers (with consistent hashing).

Disadvantages: Complex to implement. Requires coordination service (ZooKeeper, etcd). All servers need routing info. Connection state is lost if a server fails.

When to use: When you have persistent connections and significant per-connection state (e.g., Google Docs - document loading, operations, sync). Consistent hashing keeps state on one server and minimizes churn during scaling. If you are just passing small messages without much state, Pub/Sub is simpler.

Interview tip: Discuss how scaling works. In practice: (1) Signal the start of a scaling event and record both old and new server assignments. (2) Slowly disconnect clients from old servers and have them reconnect to newly assigned servers. (3) During the transition, send messages to both old and new servers until clients have moved. (4) When done, signal the end of the scaling event and update the coordination service (ZooKeeper, etcd) with the new assignments. The mechanics of discovering initial server assignments matter: clients can connect directly if they know the hash and server list, or connect to any server and get redirected. Redirects add a round-trip; a central lookup can become a bottleneck during scaling.

Pushing via Pub/Sub

A pub/sub service (Kafka, Redis) collects updates and broadcasts to interested subscribers. Endpoint servers hold lightweight connections to clients and subscribe to topics. They forward updates from the pub/sub service to the right clients. Clients can connect to any endpoint server; the server registers them with pub/sub for the relevant topics. When an update is published, the pub/sub service broadcasts to subscribers; endpoint servers forward to clients.

Advantages: Easy load management (e.g., "least connections" load balancer). Efficient broadcast to many clients. Minimal state on endpoint servers.

Disadvantages: Do not know subscriber connection status or when they disconnect. Pub/sub can become a bottleneck and single point of failure. Extra hop adds latency (usually under 10ms). Many-to-many connections between pub/sub hosts and endpoint servers.

When to use: Broadcasting to many clients with little per-client state. Redis cluster can scale pub/sub by sharding subscriptions. Use "least connections" for inbound load balancing.

Choosing Your Hop 2 Approach

The right Hop 2 strategy depends on your scale and how much state each connection carries. Pull polling works at small scale with few servers. Consistent hashing shines when connections carry heavy state. Pub/Sub is the default for lightweight fan-out at scale.

When to Use in Interviews

Real-time updates appear in most system design interviews involving user interaction or live data. Proactively identify where immediate updates matter. Strong candidates say early: "Messages need to be delivered instantly - I will address that with WebSockets" or "Character-level changes need sub-second propagation."

Common scenarios:

Chat Applications - Messages must appear instantly. WebSockets for bidirectional; pub/sub to distribute. Consider message ordering, typing indicators, presence.

Live Comments - High-volume social interaction during events. Millions commenting creates fan-out problems. Hierarchical aggregation and batching prevent overload.

Collaborative Document Editing - Character-level changes, instant propagation. WebSockets plus operational transforms or CRDTs for conflict resolution. Cursors and selection add real-time complexity.

Live Dashboards and Analytics - Constantly changing metrics. SSE works well for one-way flow. Consider aggregation intervals and what "real-time enough" means.

Gaming and Interactive Apps - Lowest latency. WebRTC for peer-to-peer; WebSockets for server coordination.

When NOT to use: If simple polling suffices, use it. Do not over-engineer. Avoid real-time patterns when the problem does not require them.

Common Deep Dives

"How do you handle connection failures and reconnection?"

Networks are unreliable. Mobile users lose connections. WiFi drops. Servers restart. You need quick disconnect detection and recovery without data loss. WebSocket connections do not always signal breaks - a client may think it is connected while the server has cleaned up.

Heartbeat mechanisms detect "zombie" connections. The client or server sends periodic PING frames; the other side responds with PONG. If no PONG arrives within the timeout window, the connection is declared dead and resources are cleaned up. Without heartbeats, zombie connections leak memory, file descriptors, and pub/sub subscriptions indefinitely.

For recovery, track what each client has received. On reconnect, send everything they missed. Maintain a per-user message queue or use sequence numbers. Redis streams are a popular option for this.

"What happens when a single user has millions of followers who all need the same update?"

The celebrity problem. Naive fan-out crashes the system. Use caching and hierarchical distribution. Cache the update once. Regional servers pull and push to local clients. Reduces load on any single component. See hierarchical aggregation patterns in scaling writes.

"How do you maintain message ordering across distributed servers?"

Messages sent milliseconds apart can arrive out of order if they take different paths or hit different servers. Vector clocks or logical timestamps help. Each server maintains a clock; messages carry timestamp info so recipients can order them.

For strict ordering, funnel related messages through a single server or partition. Trades scalability for consistency. For most product-style questions, a single server or partition is enough. If all messages go to one host, stamping with timestamps gives you a total order.

Summary

Two hops: Client-server (polling, long polling, SSE, WebSockets, WebRTC) and source-server (pull polling, consistent hashing, Pub/Sub).

Start simple: Polling often suffices. Many problems do not need sub-second latency.

Protocol choice: Polling for 2-5 second updates; long polling or SSE for near real-time; WebSockets for bidirectional high-frequency; WebRTC for video/audio or peer-to-peer.

L4 vs L7: WebSockets need L4 load balancers for persistent connections. L7 works for HTTP-based approaches.

Hop 2: Pub/Sub for lightweight broadcast; consistent hashing when connection state is heavy.

Deep dives: Reconnection and missed updates; celebrity fan-out; message ordering.

{{SUBSCRIBE}}

{{BUTTON:Read More Articles|https://newsletter.systemdesignlaws.xyz}}

30-second elevator pitch: "Multi-step processes fail halfway - payment charged but inventory not reserved. I use workflow engines like Temporal or AWS Step Functions so each step is durable, failures trigger retries or compensation, and the system picks up exactly where it left off after a crash."

What You Will Learn

Approaches

- Single-server orchestration (simple, no durability)

- Single-server with state persistence (checkpoints, pub/sub callbacks)

- Event sourcing (Kafka log, workers react to events: OrderPlaced, PaymentCharged, InventoryReserved)

- Durable execution (Temporal: workflows, activities, deterministic replay, signals)

- Managed workflows (Step Functions: declarative state machines, JSON)

>

Implementations

- Temporal, AWS Step Functions, Durable Functions, Airflow

>

Deep Dives

- Workflow versioning and migrations when you add new steps

- Keeping workflow state size manageable

- Handling external events and signals (user signs document in 5 days)

- Ensuring activities run exactly once (idempotency)

The Problem

Consider e-commerce order fulfillment: charge payment, reserve inventory, create shipping label, wait for a human to pick up the item, send confirmation, and wait for pickup. Each step calls different services. Any step can fail or timeout. Your server might crash after charging payment but before reserving inventory. Now you have money but no reserved item.

Distributed systems make even simple sequences of steps surprisingly hard. Manually adding retries, state checkpoints, and compensation logic to each step makes the system brittle. You interweave system-level concerns (crashes, retries, failures) with business-level concerns (what happens if we cannot find the item?). Workflow systems and durable execution solve this by design.

2 problems that use this pattern: Uber, Payment System.

The Solution: From Simple to Durable

The escalation from simple (single-server orchestration) to durable (event sourcing, Temporal, Step Functions). Each step adds reliability: state persistence, fault tolerance, automatic retries, and compensation.

What interviewers want to hear: "For simple flows I start with single-server orchestration. When I need reliability, I use event sourcing or a workflow engine. Temporal is my default - it gives durable execution, automatic retries, and compensation without building it myself."

Single-Server Orchestration

The simplest approach: one service calls each step in sequence. Your API server receives the order request, calls payment, then inventory, then shipping, and returns the result.

Without state - Fine for low-stakes flows. Crashes mean lost progress. No way to handle long waits (for example, a user signing a document in 5 days) without blocking.

With state - Add database checkpoints between steps and pub/sub for callbacks. You can scale out API servers and resume from state. But this quickly becomes complex: you are manually building a state machine. Who picks up dropped work when multiple servers run? Compensation is still unsolved - if inventory reservation fails after payment, you need to refund.

The architecture becomes a tangled mess of state management, error handling, and compensation logic scattered across your application.

Choreography vs Orchestration

Before diving into specific solutions, understand the two fundamental patterns for coordinating multi-step processes.

Choreography means each service reacts to events independently. No central coordinator. Services emit events and other services listen. Simple to start, but hard to reason about as complexity grows. Debugging "why did the order not ship?" requires tracing events across multiple services.

Orchestration means a central coordinator calls each service in sequence, deciding what to do next based on results. Easier to understand and debug but creates a single point of coordination.

Interview tip: Most workflow engines use orchestration. Mention choreography as an alternative when services are truly independent and the flow is simple.

Event Sourcing

Event sourcing stores a sequence of events that represent what happened instead of the current state. The event log both records history and orchestrates next steps.

Here is the flow: the API emits "OrderPlaced" when an order request arrives. The payment worker sees OrderPlaced, charges payment, and emits "PaymentCharged" or "PaymentFailed". The inventory worker sees PaymentCharged, reserves stock, and emits "InventoryReserved" or "InventoryFailed". The shipping worker continues from there.

Benefits:

Fault tolerance - If a worker crashes, another picks up the event

Scalability - Add more workers for higher load

Observability - Complete audit trail of all events

Flexibility - Possible to add new steps or modify workflows

Note: You are building significant infrastructure: event stores, message queues, worker orchestration. For complex business processes this becomes its own distributed systems project.

The Saga Pattern and Compensation

When a step fails after previous steps have completed, you need compensating actions to undo the work. This is the saga pattern.

Each forward step must have a defined reverse action. If shipping fails after payment was charged and inventory was reserved, the saga runs compensating actions in reverse order: release inventory, then refund payment.

Key rule: Compensating actions must be idempotent. A refund should be safe to retry if it fails halfway through.

Workflows: Durable Execution vs Managed

What we really want is a workflow: a reliable, long-running process that survives failures and continues where it left off.

Durable Execution Engines (Temporal)

Durable execution means long-running code that can move between machines and survive crashes. Instead of losing progress when a server restarts, the engine resumes from the last successful step on a new host.

You write a function that represents the workflow. It looks like normal sequential code: call processPayment, then if it succeeds call reserveInventory, then shipOrder and sendConfirmationEmail; if inventory fails, call refundPayment.

Workflows and activities:

Workflows - The high-level flow. Deterministic: given the same inputs and history, they always make the same decisions. This enables replay-based recovery.

Activities - The individual steps (charge payment, reserve inventory). Must be idempotent: same inputs yield the same result.

Each activity run is recorded in a history database. If a workflow runner crashes, another replays the workflow, uses the history for prior activity results, and does not re-run completed activities.

Managed Workflow Systems (Step Functions)

Managed workflow systems use a declarative approach. You define the workflow as a state machine or DAG in JSON, YAML, or a specialized DSL. The engine handles orchestration.

AWS Step Functions defines workflows as state machines in JSON. You declare states: ProcessPayment (Task, calls a Lambda), then CheckPaymentResult (Choice, branches on success/failure), then ReserveInventory or PaymentFailed.

Interview tip: Both approaches can work for similar purposes. Default to Temporal unless the company is AWS-centric.

Handling External Events and Signals

Your workflow needs to wait for a customer to sign documents. They might take 5 minutes or 5 days. How do you handle this efficiently?

Workflows excel at waiting without consuming resources. Use signals for external events. The workflow sends the document for signing via an activity, then waits for a "document_signed" signal with a timeout. If the signal arrives, process it. If it times out, send a reminder.

External systems send signals through the workflow engine API. No polling, no resource consumption. This handles human tasks, webhook callbacks, and external integrations.

Reliability Deep Dives

Retry with Backoff

When an activity fails (network timeout, service unavailable), the workflow engine automatically retries with exponential backoff. Each retry waits longer: 1s, 2s, 4s, 8s. Add jitter to prevent thundering herd when many workflows retry simultaneously.

Exactly-Once Semantics

Most workflow systems ensure an activity runs "exactly once" for a specific definition of "run". If the activity finishes successfully but fails to acknowledge the engine, the engine may retry.

Make activities idempotent - The activity can be called multiple times with the same inputs and produce the same result. Store an idempotency key in a database and check if it exists before performing the irreversible action.

Dead Letter Queue

Messages that repeatedly fail processing go to a dead letter queue for inspection. This prevents poison messages from blocking the entire queue.

Workflow Versioning

You have 10,000 running workflows. You need to add a new compliance check. Workflow versioning lets old workflows continue with old logic while new workflows use updated logic.

Real-World Interview Scenarios

Payment System Workflow

Payment systems have lots of state, strong need for graceful failure handling. You do not want a user charged for a product they did not receive.

Uber Ride Workflow

When a user requests a driver, the driver must accept. The workflow waits for human completion at multiple points: driver acceptance, pickup arrival, and ride completion.

When to Use in Interviews

When NOT to use:

Simple async - Resize an image, send an email. Use a message queue. Workflows are overkill for single-step operations.

Synchronous operations - Client waits for response, or latency is critical. Workflows are for truly async, multi-step processes.

High-frequency, low-value - Workflows add overhead. For millions of simple operations, cost and complexity are not justified.

Interview tip: Start simple. Only introduce workflows when you identify specific problems they solve: partial failure handling, long-running processes, complex orchestration, or audit requirements.

Summary

Simple flows - Single-server orchestration. Fine for low-stakes, no long waits.

State persistence - Add checkpoints and pub/sub. Quickly becomes complex.

Event sourcing - Kafka log, workers emit events (OrderPlaced, PaymentCharged, InventoryReserved). Fault tolerant, scalable, full audit trail.

Durable execution (Temporal) - Workflows and activities, deterministic replay, signals for external events. Survives crashes, automatic retries.

Managed workflows (Step Functions) - Declarative state machines in JSON. Good for AWS-heavy environments, less expressive.

Listen for - State machines, partial failures, long waits, compensation logic.

In interviews, demonstrate you know when to introduce workflows and when simpler solutions suffice. Be ready to discuss versioning, state size, external events, and idempotency.

30-second elevator pitch: "Contention happens when two users act on the same data at once, like two people buying the last concert ticket. I fix it by making operations atomic, using locks when conflicts are common, or optimistic concurrency when they are rare. For distributed systems, I use sagas or two-phase commit. I always exhaust single-database solutions first."

What You Will Learn

Single-node solutions (start here)

- Race conditions and the double-booking problem

- Atomicity and transactions (the READ COMMITTED gotcha)

- Pessimistic locking (FOR UPDATE, row locks, lock granularity)

- Isolation levels (READ UNCOMMITTED through SERIALIZABLE)

- MVCC (multi-version concurrency control)

- Optimistic concurrency control (version column, compare-and-swap, ABA problem)

>

Multiple nodes

- Two-phase commit (prepare phase, blocking, coordinator log)

- Distributed locks (Redis TTL, ZooKeeper/etcd, fencing tokens, reservations for UX)

- Saga pattern (compensation, orchestrator, temporary inconsistency)

>

Choosing the right approach - Decision tree from single DB to distributed

>

Deep Dives (what interviewers ask next)

- Deadlocks and ordered locking

- ABA problem with optimistic concurrency

- Write skew anomaly

- Optimistic vs pessimistic tradeoffs

- Hot partition and queue-based serialization

- Interview scenario pattern matching

The Problem: Race Conditions

Consider buying concert tickets online. There is 1 seat left for a popular concert. Alice and Bob both click "Buy Now" at the same moment. Without coordination, both read "1 seat available," both proceed to payment, and both get confirmation emails for the exact same seat. One gets kicked out at the door.

The race condition happens because both read the same initial state before either write takes effect. There is a gap between "check the current state" and "update based on that state" where the world can change. In that tiny window (microseconds in memory, milliseconds over a network), things break. This is fundamentally an isolation problem, not an atomicity problem. Each transaction individually succeeds; they interfere because they see the same data concurrently.

The Double-Booking Problem

The core issue is read-then-write without coordination. Both users read the same snapshot, both make decisions based on that stale read, and both write conflicting updates.

3 problems that use this pattern: Ticketmaster, Rate Limiter, Online Auction.

The Solution: Escalation Path

Contention handling follows an escalation path: single-database solutions first, then distributed coordination only when necessary. The diagram below shows the escalation from atomicity (single DB) through locks and optimistic concurrency, then to distributed coordination (2PC, sagas, distributed locks) when multiple databases are involved.

What interviewers want to hear: "I start with atomic transactions. If contention is high, I use pessimistic locking. If it is low, optimistic concurrency is faster. For distributed systems, I prefer sagas over 2PC when I can tolerate eventual consistency. I do everything possible to keep contended data in a single database."

Single-node Solutions

When all your data exists in a single database, contention solutions are straightforward. Nine times out of ten, keeping contended data together avoids distributed coordination entirely.

Atomicity

Atomicity means a group of operations either all succeed or all fail. No partial completion. Transactions provide this: you start a transaction, perform operations (debit one account, credit another), then COMMIT to save or ROLLBACK to undo everything. If anything fails mid-way, the database rolls back all changes in that transaction.

For a concert ticket purchase, you need two operations: decrement the seat count and create a ticket record. Wrap both in one transaction. If the decrement fails (sold out) or the insert fails (constraint violation), both roll back. No orphaned ticket without a seat.

Many databases support transactions: relational (PostgreSQL, MySQL), NoSQL (MongoDB multi-document, DynamoDB), and distributed SQL (CockroachDB, Spanner). Concepts apply regardless of engine; isolation guarantees vary.

Interview tip: Always mention transactions when multiple related writes must stay consistent. And always check affected row count when your update includes a predicate. Zero rows means someone else won the race.

Pessimistic Locking

Pessimistic locking assumes conflicts will happen and prevents them up front. The name comes from being pessimistic about conflicts. You acquire an exclusive row lock before reading, so no other transaction can read or modify that row until you commit.

The pattern: first run a SELECT with FOR UPDATE on the concert row (this acquires the lock), then decrement seats only if available_seats is greater than zero, check that the update affected one row, then insert the ticket. The second transaction blocks at the SELECT until the first commits. Only one person checks and updates at a time.

A lock is a mechanism that prevents other database connections from accessing the same data until the lock is released. PostgreSQL and MySQL handle thousands of concurrent connections; locks ensure only one connection modifies a specific row at a time.

Without lock - Both read seats = 1, both decrement. Result: -1 seats, double sale. With FOR UPDATE - First transaction locks row; second waits. Only one sale.

The available_seats greater than zero guard matters. Locking serializes access; the predicate enforces correctness. Without it, two transactions running back-to-back could decrement seats to -1.

Lock Granularity

The scope of your lock determines how much concurrency you sacrifice. Locking an entire table kills throughput. Locking a single row is precise and fast. The right granularity depends on what your operation touches.

Isolation Levels

Isolation levels control how much concurrent transactions see of each other's changes. Instead of explicit FOR UPDATE, you can raise the isolation level and let the database detect conflicts.

Most databases support four standard levels (different options, not a progression):

READ UNCOMMITTED

The weakest isolation level. Transactions can see uncommitted changes from other transactions. This is called a "dirty read" because you might read data that gets rolled back and never actually existed.

Rarely used in production. The tiny performance gain is not worth the data corruption risk. Mentioned in interviews to show you understand the spectrum.

READ COMMITTED

The default in PostgreSQL. Only committed data is visible to reads. This prevents dirty reads but still allows the double-booking race condition because two transactions can read the same committed state and both proceed.

REPEATABLE READ

The default in MySQL. Once you read a value within a transaction, re-reading it returns the same result even if another transaction committed a change. The database takes a snapshot at transaction start.

Prevents non-repeatable reads but still allows phantom reads (new rows appearing in range queries).

SERIALIZABLE

The strongest isolation level. The database makes transactions appear to run one after another, even though they run concurrently. It detects conflicts between concurrent transactions and aborts one.

MVCC: How Databases Make It Work

Under the hood, modern databases use Multi-Version Concurrency Control to implement isolation levels without readers blocking writers. The database keeps multiple versions of each row and serves the right version to each transaction based on its snapshot.

MVCC is why PostgreSQL and MySQL achieve high concurrency. Readers never block writers, writers never block readers. Only writer-writer conflicts need locks. Understanding MVCC helps you reason about what your isolation level actually guarantees.

Optimistic Concurrency Control

Optimistic concurrency control (OCC) assumes conflicts are rare. You let transactions proceed and detect conflicts after they occur. When conflicts are infrequent, this eliminates locking overhead.

Version column pattern: Add a version number to your data. On each update, increment the version and include "expected version = X" in your WHERE clause. When Alice updates first, her version goes from 42 to 43. When Bob tries to update with "WHERE version = 42," zero rows match because someone else changed the record. The application must check the affected row count. If zero rows, roll back and do not insert the ticket. The database will not raise an error for a WHERE that matches nothing.

Business value as version: You can use existing data as the version when it changes monotonically. For concerts, available_seats itself can serve: "UPDATE ... WHERE available_seats = 1" means only one transaction will match. Again, check affected row count. If zero rows, roll back.

Compare-And-Swap

The version check pattern is a form of Compare-And-Swap (CAS), a fundamental primitive in concurrent systems. Read a value, compare it with what you expect, and swap atomically only if it matches. This pattern shows up everywhere from SQL to DynamoDB to CPU hardware.

This approach makes sense when conflicts are uncommon. For most e-commerce, the chance of two people buying the exact same item at the exact same moment is low. Occasional retries beat locking overhead.

The ABA Problem

Example: a bank balance goes $100 to $50 to $100. Your check would pass. A dedicated, monotonically increasing version column is safest. Use business values as version only when they change in one direction (e.g. auction bid amounts that only go up).

Multiple Nodes

When you need to coordinate updates across multiple databases, things get complex. If your system needs strong consistency under high contention, do everything possible to keep the contended data in a single database first.

Consider a bank transfer where Alice and Bob have accounts in different databases (e.g. after sharding). Database A debits Alice; Database B credits Bob. Both must succeed or both must fail. If A debits but B fails, money disappears.

Two-Phase Commit (2PC)

Two-phase commit coordinates across participants. Your transfer service acts as the coordinator. Phase one (prepare): ask all participants to prepare the transaction. Each database does the work except the final commit, verifying funds, placing holds, making the prepared state durable. Phase two (commit or abort): if all prepared successfully, tell them to commit; otherwise abort.

Critically, the coordinator must write to a persistent log before sending commit or abort. The log records which participants are involved and the transaction state. Without it, coordinator crashes leave participants unsure whether to commit or abort prepared transactions.

Distributed Locks

For simpler coordination, distributed locks ensure only one process works on a resource at a time across the system. Acquire locks on both Alice's and Bob's account IDs before starting the transfer.

Redis with TTL is the most common implementation. Use SET with NX (only set if not exists) and expiration. Redis removes the lock after TTL. Fast, simple; Redis is a single point of failure. The NX flag is critical because without it, a second process could overwrite an existing lock.

ZooKeeper and etcd provide consensus-based locks with strong consistency. ZooKeeper uses ephemeral nodes that disappear when the client session ends. These systems maintain consistency through Raft (etcd) and ZAB (ZooKeeper) protocols. More robust under partitions, but more operational complexity.

Reservations for UX: Distributed locks improve user experience by shrinking the contention window. For Ticketmaster, when a user selects a seat, move it to "reserved" with a 10-minute TTL instead of "sold." The window shrinks from the full purchase flow (minutes) to the reservation step (milliseconds). Same pattern: Uber's "pending_request" driver status, e-commerce cart "holds," meeting room temporary reservations.

Saga Pattern

The saga pattern breaks the operation into steps. Each step is a committed transaction. If a later step fails, run compensating transactions to undo earlier steps.

For a bank transfer: Step 1 is debit Alice in Database A, commit. Step 2 is credit Bob in Database B, commit. Step 3 is send notifications. If Step 2 fails (Bob's account does not exist), compensate Step 1 by crediting Alice back. No long-running transactions holding locks across network calls.

Compensating Transactions in Detail

Every forward step in a saga must have a defined compensation before implementation. Compensations are semantic undos, not database rollbacks. The forward step already committed, so the compensation is a new action that reverses the business effect.

Sagas need a durable orchestrator to track which steps completed. If the coordinator crashes after Step 1, it must resume and either complete remaining steps or run compensations. Workflow engines (Temporal, Cadence) or a state machine backed by a database handle this.

Choosing the Right Approach

Start with: can you keep all contended data in a single database? If yes, use pessimistic or optimistic based on conflict frequency.

Optimistic vs Pessimistic: The Core Tradeoff

The choice between optimistic and pessimistic concurrency control comes down to contention level. Under low contention, optimistic wins because no locks are held and throughput is higher. Under high contention, pessimistic wins because blocked threads waste less work than retried transactions.

Approach selection:

Pessimistic locking - Use when: high contention, critical consistency, single database. Avoid when: low contention, high throughput needs. Latency: low (single DB query). Complexity: low.

SERIALIZABLE isolation - Use when: need automatic conflict detection, cannot identify specific locks. Avoid when: performance critical, high contention. Latency: medium (conflict detection overhead). Complexity: low.

Optimistic concurrency - Use when: low contention, high read/write ratio, performance critical. Avoid when: high contention, cannot tolerate retries. Latency: low when no conflicts. Complexity: medium.

Distributed transactions (2PC) - Use when: must have atomicity across systems, can tolerate complexity. Avoid when: high availability requirements, performance critical. Latency: high (network coordination). Complexity: very high.

Distributed locks - Use when: user-facing flows, need reservations, simpler than 2PC. Avoid when: no alternatives, purely technical coordination. Latency: low (simple status updates). Complexity: medium.

When to Use in Interviews

Do not wait for the interviewer to ask about contention. When you see multiple processes competing for the same resource, call it out and suggest coordination.

Common Interview Scenarios

Online Auction - Optimistic concurrency. Use current high bid as "version" (bids only go up, no ABA). Accept new bids only if higher than expected current bid.

Ticketmaster / Event Booking - Temporary reservations beat pure locking. Reserve seats with 10-minute TTL when selected. Avoids users losing seats after filling payment info.

Banking / Payment Systems - Distributed transactions. Start with sagas for resilience; mention 2PC if interviewer pushes strict consistency.

Ride Sharing Dispatch - Temporary status reservations. Set driver to "pending_request" when sending requests. Use cache with TTL or DB status fields with cleanup jobs.

Flash Sale / Inventory - Mix of optimistic concurrency (version column for inventory) and cart "holds" (distributed locks with TTL).

Yelp / Review Systems - Optimistic concurrency. Dedicated version column for restaurant average rating; update only if version matches to avoid corrupted calculations when reviews arrive simultaneously.

When NOT to Overcomplicate

Do not reach for distributed locks (Redis, etc.) when a simple database transaction with row locking or OCC is enough.

Low contention (e.g. admin-only product description edits) - Basic optimistic concurrency with retry.

Single-user operations (todo lists, private docs, preferences) - No coordination needed.

Read-heavy with occasional writes - Simple OCC for rare write conflicts.

Common Deep Dives

How do you prevent deadlocks with pessimistic locking?

Example: Alice transfers $100 to Bob while Bob transfers $50 to Alice. Transaction A locks Alice first, then tries Bob. Transaction B locks Bob first, then tries Alice. Both wait forever.

Deadlocks happen when transactions acquire locks in different orders. Use ordered locking: always acquire locks in a consistent order (e.g. sort by user ID). For a transfer between users 123 and 456, always lock 123 first regardless of who initiated. The exact scheme does not matter as long as it is globally consistent.

As a fallback: transaction timeouts and database deadlock detection. Set timeouts so deadlocked transactions are killed and can retry with correct ordering.

What if your coordinator crashes during a distributed transaction?

Classic 2PC failure. Databases hold prepared transactions, waiting for commit or abort. They hold locks and block other operations.

Production systems use coordinator failover and recovery. A new coordinator reads persistent logs, finds in-flight transactions, and completes them. Design for coordinator high availability.

Sagas are more resilient: no locks across network calls. Coordinator failure pauses progress rather than leaving participants blocked.

How do you handle the ABA problem with optimistic concurrency?

ABA: a value goes from A to B and back to A. Your optimistic check sees the same value and assumes nothing changed.

Example: a restaurant has 4.0 stars and 100 reviews. Two new reviews (5 and 3 stars) arrive. Both see 4.0 and compute new averages. The math might yield 4.0 again with 102 reviews. Using average as "version," both could pass the check and corrupt the count.

Use a dedicated version column that increments on every update. Update only if version matches what you read. Do not use business values like review_count as version unless they only increase; if reviews can be deleted, count can go 100 to 99 to 100 (ABA).

What is write skew and how do you prevent it?

Write skew is a subtle anomaly where two transactions read the same data but write to different rows, violating a constraint that spans multiple rows.

REPEATABLE READ misses write skew because each transaction consistently sees its own snapshot and they write different rows. There is no row-level conflict for the database to detect. Only SERIALIZABLE isolation or explicit locking of all rows involved in the constraint can prevent it.

What about performance when everyone wants the same resource?

Hot partition or celebrity problem: everyone hammers one resource. Sharding does not help (you cannot split one Taylor Swift concert). Load balancing just spreads requests to servers that all hit the same DB row. Read replicas do not help because the bottleneck is writes.

First: can you change the problem? Maybe 10 identical items with separate auctions. Maybe eventual consistency for social follows.

For strong consistency on a hot resource, use queue-based serialization. Put all requests for that resource into a dedicated queue processed by a single worker. Operations become sequential; contention disappears. The queue absorbs spikes. Tradeoff: higher latency for users.

Summary

Atomicity - Transactions for single-DB consistency. Check affected row count; READ COMMITTED alone does not prevent races.

Pessimistic - Lock first (FOR UPDATE) when conflicts are common. Order locks to avoid deadlocks. Keep lock scope narrow.

Optimistic - Version check when conflicts are rare. Dedicated version column avoids ABA. CAS pattern works across SQL, NoSQL, and caches.

Isolation levels - SERIALIZABLE for automatic conflict detection; explicit locks for efficiency. Understand MVCC to reason about what you see.

Distributed - 2PC for strong consistency (coordinator log, blocking tradeoff); sagas for resilience and temporary inconsistency.

Distributed locks - Redis, ZooKeeper/etcd. Use reservations for UX; use fencing tokens when TTL can expire early.

Good system designers keep data together as long as possible and pick the right coordination pattern for their consistency requirements.

30-second elevator pitch: "Caching stores frequently accessed data in memory so reads skip the database. I start with cache-aside using Redis as the default, add CDN for global static assets, and choose invalidation strategy based on freshness requirements. Cache-aside handles most scenarios, and I layer additional strategies only when the problem demands it."

The Problem

Reading a user profile from PostgreSQL takes 5-50ms. Reading the same profile from Redis takes ~1ms. That is a 50x improvement. Databases hit disk. Caches serve from memory.

Consider a social media feed. When you open the app, it fetches dozens of posts, each with user info, like counts, and comment previews. That is 100+ read operations just to render a feed. Now multiply by millions of concurrent users. Your database cannot handle that load -- but a cache can, because the same popular content gets requested millions of times.

This imbalance is everywhere. For every tweet posted, thousands read it. For every product listed on Amazon, hundreds browse it. Caching exploits this skewed access pattern: a small set of data accounts for the majority of reads.

14 problems that use this pattern: Distributed Cache, Rate Limiter, URL Shortener, Instagram, Facebook News Feed, YouTube, Ticketmaster, Yelp, Facebook Post Search, Local Delivery, News Aggregator, Metrics Monitoring, Top-K Problem, Notification System.

What You Will Learn

Where to Cache (pick the right layer)

- External caching (Redis, Memcached)

- CDN edge caching

- Client-side caching

- In-process caching

>

Cache Architectures (how reads and writes flow)

- Cache-aside (lazy loading) -- the default

- Write-through, write-behind, read-through

- Pattern comparison and trade-offs

>

Invalidation and Consistency (keeping cache fresh)

- TTL, event-driven, versioned keys

- Consistency models

- Cache warming

>

Deep Dives (what interviewers ask next)

- Cache stampede and thundering herd

- Hot key fanout and request coalescing

- Eviction policies (LRU, LFU, FIFO)

- Redis vs Memcached

- Distributed cache topology

The Solution

Caching follows a progression: start with external cache (Redis), add CDN for global reach, and layer in-process caching only for ultra-hot keys. Each layer adds complexity. Start simple.

What interviewers want to hear: "I would start with cache-aside using Redis for our hot read paths. If we need global distribution, I would add a CDN. The key trade-off is freshness vs complexity -- TTL gives us simplicity, write-through gives us consistency."

Where to Cache

External Caching (Redis, Memcached)

A standalone cache service shared across application servers. Every app instance talks to the same cache. This is the default caching layer in system design interviews.

How it works: Application checks cache first. On hit, return immediately (~1ms). On miss, fetch from DB (5-50ms), store in cache with TTL, return.

Use when: High read traffic, shared hot data across instances. This is your baseline answer.

CDN (Content Delivery Network)

A CDN is a network of geographically distributed edge servers that cache content close to users. Cloudflare, Fastly, and Akamai operate hundreds of edge locations worldwide.

Without a CDN, a user in Tokyo requesting an image from a Virginia server experiences 250-300ms of network latency. With a CDN, the Tokyo edge server serves the cached image in 20-40ms.

Introduce when: Static assets at global scale -- images, video thumbnails, CSS/JS bundles. CDNs reduce origin load by 90%+. Do not cache user-specific private data at the edge.

Client-Side Caching

Client-side caching stores data on the user's device -- browser localStorage, sessionStorage, IndexedDB, or native app storage. Zero network latency because the data is already local.

Strava keeps run data on-device for offline use, then syncs when connectivity returns. This reduces API calls and enables offline access.

Mention as: An optimization layer. Limited backend control over invalidation makes it secondary to server-side caching.

In-Process Caching

In-process caching stores data in the application's local memory. No network hop. Faster than Redis for small, frequently accessed data -- feature flags, configuration, rate-limit counters.

The limitation: each app instance maintains its own separate cache. Updates on one instance do not propagate to others. This means short TTLs or broadcast invalidation are necessary.

Mention after: You have already introduced external cache. In-process is a supplementary optimization, not a primary caching strategy.

Cache Architectures

How you read and write cached data determines performance and consistency. Know these four patterns.

Cache-Aside (Lazy Loading)

The default pattern for interviews. Application manages both cache and database.

1. App checks cache

2. Hit -- return immediately (~1ms)

3. Miss -- fetch from DB, store in cache with TTL, return

Cache-aside only stores data when it is requested. This keeps the cache lean -- only hot data stays cached. The trade-off is cold start latency: the first request for any key hits the database.

Write-Through

Write-through writes to cache and database synchronously. Both must succeed before returning to the caller. The cache is always fresh -- reads never see stale data.

The cost: every write has 2x latency (cache write + DB write). Cache gets polluted with data that may never be read. Redis does not support write-through natively -- you need application logic.

Use when: Reads must always return the latest data and you can accept slower writes. Financial balances, inventory counts.

Write-Behind (Write-Back)

Write-behind writes to cache first (fast return), then asynchronously batches writes to the database. Writes are fast because the caller only waits for the cache write.

The risk: if the cache dies before flushing to the database, queued writes are lost. This is eventual consistency with a data loss window.

Use when: Write throughput matters more than durability. Analytics ingestion, view counters, metrics.

Read-Through

Read-through puts the cache in front of the database as a data access layer. The application asks the cache, and on a miss, the cache fetches from the database and populates itself. The application never touches the database directly.

Use when: You want to simplify application code by making the cache the single data access point.

Invalidation and Consistency

Cache invalidation is the process of removing or updating cached data when the source changes. Getting this wrong means users see stale data -- or worse, the system acts on outdated information.

TTL (Time-Based Expiration)

TTL sets a fixed lifetime on each cached entry. When the timer expires, the entry is evicted and the next request fetches fresh data. Simple to implement. The downside: stale data during the TTL window.

How to set TTL: driven by non-functional requirements. "Search results should be no more than 30 seconds stale" gives you a 30-second TTL.

Versioned Keys

Versioned keys include a version number in the cache key: event:123:v42. On update, increment the version atomically in the database. Write new data as event:123:v43. The old entry (v42) becomes unreachable -- nobody asks for it. No race conditions because version changes are atomic.

Two cache lookups per request (version + data), but no invalidation complexity.

Consistency Models

Cache and database can diverge. Accept eventual consistency or pay for synchronous writes.

Strong consistency (write-through): Cache and DB always agree. Slower writes.

Eventual consistency (write-behind): Cache updated first, DB catches up. Fast writes, staleness window.

Stale reads (TTL): DB is truth. Cache may be stale until expiry. Simplest.

Cache Warming

A cold cache after deploy or restart means 100% miss rate. Every request hits the database. Latency spikes. DB may become overloaded.

Pre-populate on deploy: Startup script loads the top N most-accessed keys from the database.

Lazy loading: Cache fills organically as requests come in. Slower ramp-up but no extra logic.

Predictive warming: Analyze access patterns to pre-load keys that will be needed next.

Common Deep Dives (What Interviewers Ask Next)

"Cache stampede when a popular entry expires"

A cache stampede (thundering herd) happens when a popular cache entry expires and thousands of concurrent requests all see a miss simultaneously. Every one queries the database. Your DB, sized for 1,000 QPS during normal operation, gets hit with 100K identical queries. It crashes.

Solutions:

Distributed lock (SETNX): First request acquires lock and rebuilds cache. Others wait or serve stale data.

Probabilistic early refresh: As cache entry ages, each request has a random chance of triggering a background refresh before expiration.

Background refresh: Dedicated process refreshes critical entries before they expire. Users never trigger rebuilds.

"Millions of reads for the same hot key"

A celebrity posts. Millions of users read it. Even with data in memory, serializing and sending it 500K times per second overwhelms a single cache node.

Request coalescing combines duplicate in-flight requests. If 1000 requests arrive for the same key simultaneously, only one fetches from the backend. The rest wait for that result.

Cache key fanout distributes a hot key across multiple entries: feed:celebrity:1 through feed:celebrity:10. Clients randomly pick one. 500K RPS across 10 keys = 50K each.

"What eviction policy should I use?"

Eviction policies determine which entry gets removed when the cache is full.

LRU (Least Recently Used): Evict the item accessed longest ago. Default in Redis. Best general-purpose choice.

LFU (Least Frequently Used): Evict the item with the lowest access count. Better for power-law distributions.

FIFO (First In, First Out): Evict the oldest inserted item. Rarely the right answer.

"Redis or Memcached?"

Redis is the default answer for interviews. It supports rich data structures (strings, lists, sets, sorted sets, hashes), built-in persistence (RDB snapshots + AOF), primary-replica replication, and pub/sub. Single-threaded event loop -- no lock contention.

Memcached is simpler: pure key-value strings, multi-threaded, no persistence. Use only when you need maximum simplicity.

"How does Redis scale?"

Redis Cluster shards data across multiple nodes using 16384 hash slots. Each key maps to a slot via CRC16(key) mod 16384. The client routes directly to the correct shard -- no central coordinator.

Each shard has replicas for failover. If a primary dies, a replica is promoted automatically via gossip protocol. Each shard handles ~100K ops/sec.

"How do you handle distributed cache node failures?"

Consistent hashing distributes keys across cache nodes using a hash ring. When a node fails, only its keys (1/N of total) need to remap to the next node clockwise on the ring. Naive hash-mod remaps all keys.

Virtual nodes (100-200 per physical node) ensure even distribution across the ring.

"Multiple caching layers?"

A multi-layer cache hierarchy puts faster, smaller caches in front of slower, larger ones. L1 (in-process, 0.1ms) catches 80% of requests. L2 (Redis, 1-5ms) catches 15% more. L3 (CDN, 10-40ms) catches 4%. Only 1% reaches the database.

Add layers incrementally. Most systems need only Redis. Add CDN for global users. Add in-process for ultra-hot keys.

When to Use in Interviews

Common scenarios:

Bit.ly / URL Shortener: Cache short-to-long URL mapping in Redis with no expiration (URLs rarely change). CDN for global traffic.

Ticketmaster: Cache event details and venue info. Seat availability cannot be cached -- stale data causes overselling.

News Feed (Facebook/LinkedIn): Pre-compute feeds for active users. Cache recent posts aggressively with short TTL.

YouTube: Cache video metadata (titles, thumbnails). View counts can be eventually consistent. CDN for thumbnails.

When NOT to use caching:

Write-heavy systems (Uber location tracking) -- focus on write scaling instead

Small scale (1000 users) -- a well-indexed database handles the load

Strongly consistent (financial transactions) -- stale data causes real problems

Real-time collaborative (Google Docs) -- caching actively hurts when every keystroke must be visible

Summary

Cache-aside + Redis -- default for high read traffic. Simple, effective.

CDN -- static assets, global users. 90%+ origin load reduction.

Write-through -- when freshness is critical. Accept slower writes.

Write-behind -- when write throughput matters. Accept data loss risk.

Invalidation -- TTL for simplicity, versioned keys for race-free updates, event-driven for critical data.

Eviction -- LRU is the default. LFU for power-law access patterns.

Stampede protection -- distributed lock, probabilistic refresh, or background refresh.

In interviews, demonstrate that you understand both the latency benefits and the consistency trade-offs. Show that you know when to cache aggressively and when to skip caching entirely.

30-second elevator pitch: "For any operation that takes more than a few seconds, I return a job ID immediately and process asynchronously. A message queue decouples submission from processing. Workers pull jobs, execute them, and update status. The client polls for completion or gets a webhook callback. I add exponential backoff for retries and a dead letter queue for poison messages."

What You Will Learn

Core Patterns

- Synchronous vs asynchronous processing

- Submit-and-poll pattern (return job_id, client polls status)

- Message queues (decouple producer from consumer)

- Worker pools (scale processing independently)

>

Reliability

- Retry with exponential backoff and jitter

- Dead letter queues for poison messages

- Idempotency keys for safe retries

- Exactly-once processing semantics

>

Scaling

- Backpressure (queue absorbs burst)

- Priority queues (paid users first)

- Rate-limiting workers (protect downstream)

- Auto-scaling workers on queue depth

>

Deep Dives

- Progress tracking and notification strategies

- Webhook callbacks vs polling

- Job dependencies and DAG execution

- Implementation choices (Redis, RabbitMQ, SQS, Kafka)

The Problem

A user requests a PDF report. Your server queries multiple databases, generates charts, renders 500 pages, and assembles the document. This takes 3 minutes. But HTTP requests typically timeout after 30 seconds. Even if the connection stays open, the server thread is blocked, unable to serve other requests. On mobile, the connection drops when the user switches apps.

The synchronous model breaks for any task that takes more than a few seconds: video transcoding, data export, batch email sending, image processing, or report generation. The solution is the same in every case: accept immediately, process asynchronously.

6 problems that use this pattern: YouTube video processing, Dropbox file sync, Stripe payment webhooks, report generation, data export, batch notifications.

The Solution: Submit and Poll

The fundamental pattern is simple. The client submits a request. The server acknowledges immediately with a job ID and HTTP 202 Accepted. The server enqueues the task. A background worker picks it up and processes it. The client polls for status using the job ID.

The API response is fast (50ms). The processing happens independently. The client can navigate away, close the app, and come back later to check status.

What interviewers want to hear: "I would not process this synchronously. I would return a job ID immediately and process in the background. The client polls for status, or I send a webhook when it is done."

Message Queues

The queue is the critical component that decouples submission from processing. It absorbs burst traffic, guarantees delivery, and enables independent scaling of producers and consumers.

The producer (your API server) pushes messages to the queue. Workers (consumers) pull messages, process them, and acknowledge completion. If a worker crashes before acknowledging, the message returns to the queue for another worker.

Queue Architecture

The architecture separates three concerns: the API server handles request validation and enqueueing, the queue provides durable storage and delivery guarantees, and the worker pool handles processing.

Worker Pools

Workers are the processing engines. Each worker pulls a message from the queue, executes the task, updates the status store, and acknowledges the message.

Scale workers independently from your API servers. If the queue depth grows, add more workers. If it shrinks, scale down. This is the core benefit of the async pattern: API throughput and processing throughput scale independently.

Job States

Every job follows a state machine: queued, processing, completed, failed.

Store the current state and metadata (progress percentage, error message, result URL) in Redis or a database so clients can query it.

Reliability Patterns

Retry with Exponential Backoff

When a task fails (network error, service unavailable, transient bug), retry it. But do not retry immediately in a tight loop, that will overwhelm the failing service. Use exponential backoff: wait 1 second, then 2, then 4, then 8. Add jitter (random delay) to prevent thundering herd when many workers retry simultaneously.

Set a maximum retry count (typically 3-5). After exhausting retries, move the message to a dead letter queue.

Dead Letter Queue

A dead letter queue catches messages that repeatedly fail processing. Instead of blocking the main queue or losing the message, move it aside for inspection. Engineers can examine failed messages, fix the root cause, and replay them.

Common causes of DLQ messages: corrupt input data, schema mismatches, downstream service permanently down, or bugs in worker code.

Idempotency

If a worker processes a job but crashes before acknowledging the queue, the queue will redeliver the message to another worker. Now the job runs twice. For operations like "charge credit card" or "send email," this is a problem.

Make every operation idempotent. Before performing an irreversible action, check if it has already been done using a unique idempotency key (typically job_id + step_name). If the key exists in your database, skip the action and return the previous result.

Exactly-Once Processing

True exactly-once delivery is impossible in distributed systems. What you can achieve is "effectively exactly once" through idempotent operations: the message may be delivered more than once, but the side effect happens exactly once.

Scaling Patterns

Backpressure

When 10,000 users submit reports simultaneously, the queue absorbs the burst. Workers process at their own pace. The queue depth is your pressure gauge: if it grows faster than workers can drain it, you need more workers.

Priority Queues

Not all jobs are equal. Paid users should get their videos transcoded before free users. Priority queues ensure high-priority jobs are processed first.

Implement with separate queues per priority level (workers check high-priority first) or a single queue with priority ordering.

Rate-Limiting Workers

Without rate limits, a burst of 10,000 jobs means 10,000 concurrent calls to your payment API, which causes a cascading failure. Rate-limit workers to protect downstream services.

Strategies include concurrency limits (max N workers), token bucket (acquire token before calling downstream), and adaptive rate adjustment based on downstream latency.

Progress Tracking and Notification

Progress Tracking

For long tasks, users want to know how far along they are. Workers update a progress store (Redis or database) with percentage, current step, and estimated time remaining. The client reads this through your API.

Polling vs Webhook

Two approaches for notifying the client when a task completes.

Polling is simple: the client calls GET /jobs/{id} every few seconds. Works with browsers, mobile, and does not require inbound connections. Downside: wasted requests while the job is still running.

Webhooks are efficient: the server POSTs to a callback URL when the job completes. Instant notification, no wasted requests. But the receiver must have a public endpoint and handle retries and idempotency.

Default: Polling for client-facing applications. Webhooks for server-to-server communication.

Advanced Patterns

Job Dependencies (DAG)

Some tasks depend on others. A video pipeline must extract audio before transcribing it, but can generate thumbnails in parallel. Model dependencies as a directed acyclic graph (DAG).

A job starts when all its dependencies complete. Independent jobs run in parallel. This maximizes throughput while respecting ordering constraints.

Implementation Choices

Redis Queue (Bull, BullMQ)

Simple and fast. Good for small to medium workloads. Queue lives in Redis memory. Supports priority, delay, and retry. Limitation: not as durable as dedicated message brokers.

RabbitMQ

Robust message broker with advanced routing. Supports topic exchanges, dead letter queues, and message acknowledgment out of the box. Good for complex routing patterns.

SQS + Lambda (Serverless)

Fully managed. No infrastructure to operate. Auto-scales from zero. Pay per message. Best for event-driven workloads on AWS. Limitation: 15-minute execution limit for Lambda.

Kafka

High-throughput distributed log. Best for streaming workloads with millions of messages per second. Retains messages for replay. More complex to operate than SQS or RabbitMQ.

Interview default: SQS + Lambda for serverless/AWS environments. Redis + Bull for self-managed applications. Kafka for high-throughput streaming.

Real-World Interview Scenarios

PDF Report Generation

A classic long-running task example combining every pattern: queue, workers, progress tracking, and notification.

Video Transcoding

CPU-intensive with priority queues, auto-scaling workers, and progress tracking through multiple processing stages.

Summary

Core - Submit and poll. Return job_id immediately. Queue decouples submission from processing.

Reliability - Retry with exponential backoff. Dead letter queue for poison messages. Idempotency for safe retries.

Scaling - Backpressure via queue. Priority queues for fairness. Auto-scale workers on queue depth. Rate-limit to protect downstream.

Notification - Polling for browsers/mobile. Webhooks for server-to-server. Progress tracking via Redis.

Implementation - Redis + Bull for simple. RabbitMQ for routing. SQS + Lambda for serverless. Kafka for high throughput.

In interviews, start with the submit-and-poll pattern. Layer on reliability (retry, DLQ, idempotency) and scaling (priority, backpressure, auto-scaling) based on the specific requirements.

30-second elevator pitch: "Large files bypass my application server entirely. Clients get a presigned URL and upload directly to S3 in chunks. An S3 event triggers async processing like thumbnails or transcoding. Downloads go through a CDN. My server only handles metadata and authorization."

What You Will Learn

Upload Patterns

- Presigned URL upload (bypass application server)

- Multipart upload (parallel chunks for large files)

- Resumable upload (recover from network failures)

- Progress tracking (chunk completion percentage)

>

Download Patterns

- CDN edge caching (global distribution)

- Presigned URL download (private file access)

- Adaptive bitrate streaming (video)

>

Processing Patterns

- Thumbnail generation pipeline (eager, lazy, hybrid)

- Video transcoding pipeline (multi-resolution, HLS)

- Content-addressable storage and deduplication

>

Deep Dives

- Abuse prevention and quota systems

- Upload failure recovery and state machines

- Cloud provider comparison (S3, GCS, Azure Blob)

The Problem

Your users need to upload and download files: profile photos, videos, documents, data exports. The naive approach routes everything through your application server. A 500MB video upload consumes server memory, CPU, and bandwidth for minutes. During that time, the server cannot handle other requests. Multiply by thousands of concurrent uploads and your entire system grinds to a halt.

The file passes through your server twice: once from client to server, once from server to storage. That doubles bandwidth consumption and ties up server resources for the entire transfer duration.

5 problems that use this pattern: Dropbox, YouTube, Instagram, Slack file sharing, Google Drive.

The Solution: Direct Upload with Presigned URLs

The core insight: your application server should never touch the file data. Instead, it generates a short-lived presigned URL that grants the client temporary permission to upload directly to object storage.

The flow is straightforward. The client requests an upload URL from your API. Your server validates the request (authentication, file type, quota check), generates a presigned URL from S3, and returns it. The client uploads directly to S3 using that URL. Your server never sees the file bytes.

What interviewers want to hear: "The file never passes through my application server. I use presigned URLs so the client uploads directly to object storage. My server only handles metadata and authorization."

Multipart Upload

For files larger than 5MB, a single PUT request is risky. Network interruptions mean starting over. Multipart upload splits the file into chunks that upload independently, often in parallel.

The process works in three phases. Initiate: the client tells S3 to start a multipart upload and receives an upload ID. Upload parts: the client splits the file into chunks (typically 5-100MB each) and uploads them in parallel, each getting its own presigned URL. Each completed part returns an ETag. Complete: the client sends all ETags to S3, which assembles the final object.

Benefits: Parallel upload (faster), individual chunk retry (resilient), and progress tracking (user experience).

Resumable Upload

Mobile users on flaky networks need to resume where they left off. Resumable upload tracks which chunks are complete and only re-uploads the missing ones.

The server tracks a bitmap of completed chunks per upload ID. When the client reconnects, it queries the server for missing chunks and only uploads those. A 500MB file at 63% completion only needs to upload the remaining 37%.

Upload Failure Recovery

Uploads can fail at any point. The upload state machine tracks progress and enables recovery.

Incomplete uploads are cleaned up after a configurable expiry (typically 24 hours). S3 lifecycle policies can automatically abort stale multipart uploads.

Metadata and Storage Architecture

The metadata flow separates file data from file information. Object storage holds the actual bytes. Your database holds metadata: filename, size, content type, upload status, owner, access permissions, and references to processed variants (thumbnails, transcoded versions).

Downloading: CDN and Presigned URLs

For downloads, the pattern depends on access control. Public content goes through a CDN for global edge caching. Private content uses presigned download URLs with short expiration times.

Interview tip: For high-read content (Instagram photos, YouTube videos), always put a CDN in front. The CDN absorbs the read load that would otherwise hit your storage directly.

Processing Pipelines

After a file is uploaded, you often need to process it: generate thumbnails for images, transcode video to multiple resolutions, scan for viruses, or extract metadata.

Thumbnail Generation

Never process images synchronously in the upload request. Decouple with an event-driven pipeline.

S3 fires a notification on upload completion. A message queue delivers the event to an image worker that generates thumbnails at multiple sizes (150x150, 300x300, 600x600) and converts to WebP. The worker stores output in S3 and updates the metadata database with thumbnail URLs.

Video Transcoding

Video uploads trigger a transcoding pipeline that produces multiple resolution variants and HLS segments for adaptive bitrate streaming.

The pipeline fan-outs: one worker per target resolution. Each worker downloads the raw file, decodes, resizes, encodes, and segments the output. Results go to S3 and CDN.

Content Addressing and Deduplication

For systems like Dropbox where many users might upload the same file, content-addressable storage eliminates duplicates.

The key is the SHA-256 hash of the file content. Same content produces the same hash. Before uploading, check if the hash exists in storage. If it does, skip the upload and just link the user to the existing object. Ten users uploading the same PDF results in one stored copy.

Chunk-level deduplication goes further: edit one paragraph in a 100MB document, and only the changed 4MB chunk needs re-uploading. This is why Dropbox feels instant for small edits to large files.

Abuse Prevention and Quotas

Without limits, a single user could upload terabytes and bankrupt your storage budget. Implement quotas at multiple levels.

Check quotas before generating the presigned URL. Reject oversized files at the API level. Rate-limit upload requests. Scan uploaded content for malware asynchronously.

Real-World Interview Scenarios

Dropbox Sync Flow

The Dropbox pattern combines chunked upload, content addressing, and delta sync across devices.

YouTube Upload Pipeline

YouTube processes 500+ hours of video per minute using resumable upload, fan-out transcoding, and CDN delivery.

Cloud Provider Comparison

The concepts are identical across clouds. Say "object storage" or "S3-compatible storage" in interviews instead of locking to one provider.

Decision Tree

Start with: "The file should never pass through our application server." Then layer on complexity based on the specific requirements: file size, reliability needs, processing pipeline, and access patterns.

Summary

Upload - Presigned URLs for direct-to-storage. Multipart for large files. Resumable for unreliable networks.

Processing - Event-driven pipelines: thumbnails, transcoding, virus scanning, metadata extraction.

Download - CDN for public content. Presigned URLs for private access. Adaptive bitrate for video.

Optimization - Content addressing for deduplication. Chunk-level dedup for delta sync.

Protection - Quotas, rate limits, virus scanning, access control.

In interviews, start with the presigned URL pattern and add complexity based on the problem requirements. Every blob storage problem uses the same core architecture.

30-second elevator pitch: "Write scaling is harder than read scaling because every write must persist to disk. I exhaust vertical scaling and database tuning first. Then I shard by partition key to spread load linearly. For bursty traffic I add queues and load shedding. For extreme volume I batch writes and use hierarchical aggregation. Each tier builds on the previous one, and I always justify the complexity with back-of-the-envelope math."

The Problem

Consider a social media platform during a major sporting event. Millions of users post reactions simultaneously. Each post triggers multiple writes: the post itself, index updates for search, counter increments for trending topics, and fan-out to follower feeds. A single database handling 1,000 writes per second faces millions of incoming writes.

This is not a software bug you can optimize away. It is physics. Disk I/O has fixed throughput. CPU cores execute a finite number of instructions per second. Network bandwidth is bounded by hardware. When you hit these walls, only architectural solutions work.

The asymmetry between reads and writes makes this harder. Reads can be cached, replicated, and served from memory. Writes must persist to durable storage. You cannot cache a write. Every write must eventually reach disk.

6 problems that use this pattern: YouTube Top K, Strava, Rate Limiter, Ad Click Aggregator, Facebook Post Search, Metrics Monitoring.

What You Will Learn

Tier 1: Vertical Scaling and Database Choice (cheapest, start here)

- Hardware upgrades (200 cores, NVMe SSDs, 10Gbps links)

- Write-optimized databases (Cassandra, time-series, LevelDB, ClickHouse)

- Database tuning (disable FK/triggers, batch WAL syncs, reduce indexes)

>

Tier 2: Sharding and Partitioning (when single DB hits ceiling)

- Horizontal sharding (hash-based, range-based, consistent hashing)

- Partition key selection (flat distribution, minimize variance)

- Vertical partitioning (split tables by access pattern)

>

Tier 3: Queues and Load Shedding (for bursty traffic)

- Write queues (Kafka, SQS) for burst absorption

- Load shedding (drop low-value writes to survive)

- Queue-based write leveling

>

Tier 4: Batching and Hierarchical Aggregation (reduce write volume)

- Batching at app, intermediate, and DB layers

- Hierarchical aggregation for fan-in/fan-out

- Counter sharding for high-contention counters

>

Deep Dives (what interviewers ask next)

- Resharding without downtime (dual-write migration)

- Hot key problem (split all keys, split hot keys dynamically)

- Write amplification and compaction

The Solution: Four Strategies

Write scaling follows a progression from simple optimization to distributed architecture. Each tier addresses a different scaling constraint: hardware limits, single-server capacity, traffic bursts, and write volume.

What interviewers want to hear: "I exhaust vertical scaling and database tuning first. Then I shard by partition key to spread load linearly. For bursts I use queues. For extreme volume I batch and aggregate to reduce the number of writes hitting the database." Most candidates jump straight to Kafka without doing the math.

Tier 1: Vertical Scaling and Database Choice

Before adding distributed complexity, exhaust what a single server can do. Do back-of-the-envelope math: what is your write throughput versus hardware limits? Many candidates assume 4-8 cores and a spinning disk. In practice, servers with 200 CPU cores, NVMe SSDs, and 10Gbps network interfaces are standard in cloud environments.

Vertical Scaling

Writes are bottlenecked by three physical resources: disk I/O, CPU, and network bandwidth. Confirm you are hitting those walls before adding infrastructure.

SSDs instead of HDDs - 10-100x faster random I/O. NVMe SSDs provide 500K+ IOPS versus 200 IOPS for spinning disks.

More RAM - Larger buffer pools mean fewer disk flushes. Hot data stays in memory longer.

Faster CPUs, more cores - Handle more concurrent write transactions. Modern servers offer 200+ cores.

10Gbps+ NICs - Network rarely bottlenecks single-server writes, but matters for replication.

Staff-level insight: Make the case that modern hardware goes further than many assume. A well-tuned PostgreSQL on modern hardware handles 50,000+ writes per second. But interviewers frequently move the goalposts until you must scale architecturally. Show you understand both paths.

Write-Optimized Databases

Most applications use general-purpose databases optimized for a balance of reads and writes. Write-heavy systems benefit from databases that sacrifice read performance for write throughput.

Cassandra achieves superior write throughput through its append-only commit log. Instead of updating data in place (which requires expensive disk seeks to find the right page), Cassandra writes sequentially to disk. This yields 10,000+ writes per second on modest hardware versus roughly 1,000 for a traditional RDBMS updating B-tree indexes in place.

The key insight is the difference between B-tree storage and LSM-tree storage. B-trees update data in place, requiring random disk seeks. LSM-trees (used by Cassandra, LevelDB, RocksDB) append all writes sequentially, then merge and sort in the background.

Other write-optimized options:

Time-series databases (InfluxDB, TimescaleDB) - High-volume sequential writes with timestamps. Built-in delta encoding compresses similar values. Ideal for metrics collection where writes are append-only and ordered by time.

Log-structured databases (LevelDB, RocksDB) - Append new data rather than updating in place. Writes are always sequential. Reads may need to check multiple files.

Column stores (ClickHouse) - Batch writes efficiently for analytics workloads. Compress columnar data aggressively. Optimized for bulk inserts, not single-row writes.

Database Tuning

Before swapping databases, tune what you have. These changes can provide 5-10x improvement:

Disable expensive features - Foreign key constraints, complex triggers, and full-text search indexing all add overhead to every write. Disable during high-write periods or enforce constraints at the application layer.

Batch WAL syncs - PostgreSQL can group multiple transactions into a single fsync call. Instead of one disk sync per transaction, batch 100 transactions into one sync. 10x throughput improvement.

Reduce index overhead - Every index on a table adds a write for every INSERT. A table with 5 indexes means each logical write becomes 6 physical writes. Remove indexes that serve only rare queries.

Write Amplification

One logical write often becomes multiple physical disk writes. Understanding write amplification explains why reducing indexes and tuning WAL matters.

A single INSERT into a table with two indexes generates: one WAL append, two to three disk writes per index update, and one replication sync. That is 6-8 physical writes for one logical write. With 5 indexes, it can be 15-20x amplification.

Interview tip: When you propose adding an index to speed up reads, acknowledge the write cost. "This index will speed up our search queries from O(n) to O(log n), but each write now has an additional index update. Given our 100:1 read-to-write ratio, this trade-off is clearly worth it."

Tier 2: Sharding and Partitioning

When one server handles 1,000 writes per second, 10 servers should handle 10,000. Sharding spreads write volume across multiple servers so each handles a manageable portion. This is the most common write scaling technique you will use in interviews.

Horizontal Sharding

The basic idea: hash each record's key to determine which server stores it. All writes for that key go to one server. Add servers to scale linearly.

Redis Cluster illustrates a straightforward implementation. Each entry has a single string key. Keys are hashed (via CRC16) to determine a slot number from 0-16383. Slot numbers map to cluster nodes. Clients hash the key, look up the responsible server, and send the write there.

Consistent hashing provides a more flexible alternative. Instead of modular arithmetic (which requires rehashing everything when you add nodes), consistent hashing maps both keys and nodes onto a ring. Adding a node only remaps roughly 1/N of existing keys.

Selecting a Partition Key

The most important sharding decision: which field do you hash?

Good key - Hashing user_id spreads data evenly across shards. You realize the Nx gain by multiplying servers. Each user's writes concentrate on one shard, which is manageable because individual users produce limited write volume.

Bad key - Partitioning by country sends most writes to heavily populated regions. China and the US overloaded. New Zealand and Iceland nearly idle. The variance between shards kills your scaling gains.

Principle: Select a key that minimizes variance in writes per shard. Flat is good. Hash a primary identifier (user_id, post_id, device_id) whenever possible.

Also consider the read path. If every request must hit every shard (scatter-gather), you have traded a write bottleneck for a read bottleneck. Choose a scheme that spreads writes evenly AND groups commonly accessed data together. Ask yourself: "How many shards does this request hit?" and "How often does this request happen?"

Partitioning Strategies

Three main approaches, each suited to different workloads:

Vertical Partitioning

Vertical partitioning splits columns by access pattern. Instead of one massive table where content writes, metric updates, and analytics scans compete for the same locks, split into specialized tables.

Schema: post_content (write-once, read-many)

post_id integer, primary key - Post identifier

user_id integer - Author

content text - Post body

media_urls text array - Attachments

created_at timestamp - Publication time

Schema: post_metrics (high-frequency writes)

post_id integer, primary key - Post identifier

like_count integer - Like count

comment_count integer - Comment count

view_count integer - View count

last_updated timestamp - Last metric update

Schema: post_analytics (append-only, time-series)

post_id integer - Post identifier

event_type varchar - Event type (view, click, share)

timestamp timestamp - Event time

user_id integer - Actor who triggered event

Each table lives on a different database instance. Post content uses traditional B-tree indexes optimized for reads. Post metrics might use in-memory storage or specialized counters. Post analytics uses time-series or column-oriented storage optimized for high-volume appends.

Tier 3: Queues and Load Shedding

Sharding handles steady-state load, but real-world traffic is bursty. What happens when order volume quadruples on Black Friday, or ride requests triple on New Year's Eve? Autoscaling helps but takes minutes. Database scaling often means downtime. You either buffer writes or drop the least valuable ones.

Write Queues

Add a message queue (Kafka, SQS) between the application and the database. Writes go to the queue instantly. Workers drain the queue at a sustainable rate matched to database capacity.

The application only knows the write was recorded in the queue, not that it reached the database. Clients may need a callback or polling mechanism to confirm persistence.

Burst absorption - The queue buffers spikes. The database processes writes at a steady rate regardless of incoming traffic.

Unbounded growth risk - If the application writes faster than the database drains for an extended period, the queue grows without bound. Latency increases continuously. Users wait longer and longer for writes to take effect.

Important: Use queues for short-lived bursts, not to mask a database that cannot handle steady-state load. If your queue depth grows monotonically, you have a capacity problem, not a burst problem. Understand your system's tolerance for delayed writes before introducing queues.

Load Shedding

When overwhelmed beyond what a queue can absorb, decide which writes to accept and which to reject. Load shedding keeps the system running by dropping the least valuable writes.

Uber and Strava - Users report location at regular intervals (every 3-5 seconds). If the system is overloaded, dropping one update is acceptable because a fresher update arrives moments later. Drop GPS updates that are within seconds of the previous recorded position.

Analytics - Drop impression events before click events. Clicks drive revenue. A 5% loss in impression tracking is acceptable; a 5% loss in click tracking is not.

Metrics Monitoring - Downsample from per-second to per-minute averages during overload. Approximate monitoring data is far better than no monitoring data.

Tier 4: Batching and Hierarchical Aggregation

You can also change the structure of writes to make them more efficient. Individual writes carry overhead: network round trips, transaction setup, index updates, disk syncs. Databases process batches far more efficiently than equivalent individual operations.

Batching

Combine multiple individual writes into a single bulk operation. The savings come from amortizing fixed costs (network round trip, transaction commit, disk sync) across many writes.

Application layer - Clients buffer writes before sending. Works well when the application is not the source of truth (for example, a Kafka consumer processing events and writing to a database). If the app crashes mid-batch, re-read from Kafka. If the application is the source of truth, batching risks data loss on crash.

Intermediate processing - A Like Batcher receives like events, accumulates count changes per post over a time window, and forwards aggregated updates. If a post gets 100 likes in one minute, that reduces 100 individual counter increments to a single atomic update of +100.

Staff-level nuance: If most posts receive 1 like per hour, a 1-minute batch window provides zero benefit because the batch contains exactly 1 write. Ensure batching actually matches your workload distribution. Batching helps when multiple writes target the same key within the batch window.

Database layer - Redis flushes to disk every 100ms by default (with AOF appendfsync everysec). A burst of 1,000 writes in one batch results in one disk write 100ms after the last write arrives. This is powerful but comes with durability risk: data written to memory but not yet flushed is lost on crash.

Hierarchical Aggregation

For extreme fan-in/fan-out scenarios (like live video comments), aggregate at intermediate nodes instead of writing every event to every destination.

Live comments - Millions of viewers can comment and react. Each event must reach all peers for a shared experience. That means millions of writes from millions of writers - an N-squared problem.

Broadcast nodes - Assign viewers to broadcast nodes (via consistent hashing or geographic proximity). Writers send events to a small number of broadcast nodes instead of directly to all viewers. Each broadcast node forwards to its assigned viewers.

Write processors - Intermediate aggregation layer. Each write processor accumulates events (for example, like counts) over a time window and forwards batched updates upstream. The root aggregator merges results from all write processors.

Result: By aggregating with write processors (fan-in) and distributing with broadcast nodes (fan-out), you transform an N-squared problem into an N-times-M problem where M is the number of intermediate nodes. Trade-off: added latency from extra hops.

Counter Sharding

A specific and common application of write scaling: high-contention counters. When millions of users increment the same counter (like counts, view counts, rate limiter counters), the single row becomes a bottleneck.

Split the counter across N sub-keys. Writers increment a random sub-key. Readers sum all sub-keys to get the total. Write throughput scales linearly with N. Read cost increases linearly with N (scatter-gather). Choose N to balance write throughput against read overhead.

Fan-Out on Write

A related pattern that converts read-time computation into write-time pre-computation:

When a user creates a post, immediately write it to every follower's feed (fan-out on write). Reads become a simple lookup of the pre-computed feed. The trade-off: one post from a celebrity with 10 million followers means 10 million writes. Most production systems use a hybrid: fan-out on write for normal users, fan-out on read for celebrities.

Common Deep Dives

"How do you handle resharding when you need to add more shards?"

You started with 8 shards and now need 16. Rehashing everything offline creates hours of downtime for large datasets. Gradual migration using dual-writes avoids this.

Deploy the new shard configuration. All new writes go to both old and new locations (dual-write). A background job copies existing data from old shards to new shards. Once migration completes and data is verified consistent, switch reads to the new shards and decommission the old ones. Zero downtime because the system stays available throughout.

Why dual-write: No data is lost during migration because both locations receive all writes. The new shards accumulate both migrated historical data and real-time writes. When you cut over, they are fully up to date.

"What happens when a single key is too popular for even one shard?"

A viral tweet gets 100,000 likes per second. Even with perfect hash distribution across shards, ALL writes for this one key route to a single shard. That shard overloads while others sit idle.

Split all keys - Store post1:likes as post1:likes-0, post1:likes-1, through post1:likes-k-1. Each sub-key routes to a different shard. Write volume per shard drops by k. Downside: dataset size and read overhead multiply by k. Works when a small k brings the hot shard back within capacity.

Split hot keys dynamically - Split only keys that become hot. For the viral tweet, dynamically create 100 sub-keys. Normal posts keep a single key. This requires a detection mechanism (monitor write rate per key) and coordination so readers know which keys are split.

Reader coordination - Writers and readers must agree on which keys are split. Option 1: readers always check all sub-keys (simple, same read amplification). Option 2: writers announce splits to readers via a metadata service (efficient, more complex). Most production systems use option 1 for simplicity.

"What about compaction overhead in LSM-tree databases?"

Compaction is the background process that merges and sorts SSTable files in LSM-tree databases. It reclaims space from deleted or overwritten entries and improves read performance by reducing the number of files to check.

The trade-off: compaction consumes background I/O that competes with write throughput. During a compaction storm (when many files need merging simultaneously), write latency can spike. Tune compaction concurrency and throttling to match your workload.

When to Use in Interviews

Common scenarios with specific advice:

YouTube Top K - Millions of view/like events per second. Shard counters by video_id. Use hierarchical aggregation: local counters at edge, periodic merge to central store. Top-K computed from pre-aggregated data.

Strava - GPS location updates every 3-5 seconds from millions of active users. Load shed stale updates. Queue for burst absorption during popular events. Time-series database for storage.

Rate Limiter - Counter increment per user per time window. Shard by user_id. Counter sharding for high-traffic users. Redis with atomic INCR for low-latency writes.

Ad Click Aggregator - Click events must be counted accurately (revenue depends on it). Batch clicks per ad_id over short windows. Kafka for durability. Reconciliation jobs for accuracy.

Facebook Post Search - Every new post must be indexed for search. Queue index updates. Batch index writes (Lucene/Elasticsearch segments). Shard index by term or document.

Metrics Monitoring - High-volume time-series writes from thousands of servers. Range partition by timestamp. Batch writes per time window. Downsample old data.

When NOT to use write scaling techniques:

Read-heavy systems - If your read-to-write ratio is 100:1, focus on read scaling. Write optimization provides marginal benefit.

Modest write volume - Do back-of-the-envelope math. 100 writes per second does not need Kafka. A single well-tuned database handles thousands of writes per second. Adding distributed complexity for no reason hurts more than it helps.

When consistency matters most - Each write scaling technique introduces trade-offs. Queues mean eventual consistency. Batching adds latency. Sharding complicates transactions. Show you understand the cost before proposing the solution.

Summary

1. Vertical + DB choice - Hardware upgrades and write-optimized databases first. Cassandra LSM-trees, time-series DBs, tuned WAL. A single well-configured server handles 50K+ writes per second.

2. Sharding - Partition key with minimal variance. Hash user_id or post_id. Consistent hashing for easy resharding. Vertical partitioning for mixed workloads.

3. Queues + load shedding - Kafka/SQS for short-lived bursts. Load shedding when dropping writes is acceptable (GPS updates, analytics impressions). Never use queues to mask steady-state capacity shortfall.

4. Batching + aggregation - Batch at app, intermediate (Like Batcher), or DB layer. Hierarchical aggregation for fan-in/fan-out. Counter sharding for high-contention incrementing.

5. Deep dives - Resharding via dual-write. Hot keys: sub-key split with scatter-gather reads. Write amplification awareness when proposing indexes.

In interviews, demonstrate that you understand the physics behind write bottlenecks, the trade-offs of each technique, and the progression from simple to complex. Always justify complexity with math.

30-second elevator pitch: "Read traffic grows exponentially faster than writes. I handle this in three tiers: first optimize the existing database with indexes and denormalization, then scale horizontally with read replicas, and finally add caching layers. Each tier builds on the previous one, and I always start with the simplest solution."

The Problem

Consider an Instagram feed. When you open the app, dozens of photos load. For each photo, the app fetches image metadata, user information, like counts, and comment previews. That is 100+ read operations just to load your feed. Meanwhile, you might only post one photo per day - a single write.

This imbalance is everywhere:

Twitter/X - For every tweet posted, thousands read it. Viral tweets: millions of reads.

Amazon - For every product uploaded, hundreds browse it. During sales: millions of reads.

YouTube - Billions of video views daily, millions of uploads. A 1000:1 ratio.

The standard read-to-write ratio starts at 10:1 and reaches 100:1 or higher for content-heavy applications.

13 problems that use this pattern: Ticketmaster, Bit.ly, Instagram, Facebook News Feed, YouTube Top K, Yelp, Distributed Cache, Rate Limiter, YouTube, Facebook Post Search, Local Delivery Service, News Aggregator, Metrics Monitoring.

What You Will Learn

Tier 1: Optimize Your Database (cheapest, start here)

- Indexing (B-tree, composite indexes)

- Hardware upgrades (vertical scaling)

- Denormalization (pre-join tables)

- Materialized views (precompute aggregations)

>

Tier 2: Scale Horizontally (when single DB hits ceiling)

- Read replicas (leader-follower replication)

- Database sharding (functional, geographic, hash-based)

>

Tier 3: Add Caching (highest throughput, most complex)

- Application-level caching (Redis, Memcached)

- Cache invalidation (TTL, write-through, write-behind, tagged, versioned keys)

- CDN and edge caching

>

Deep Dives (what interviewers ask next)

- Hot key problem and request coalescing

- Cache stampede and thundering herd

- Cache versioning for race-free invalidation

- Replication lag and read-your-own-writes

The Solution: Three-Tier Progression

Read scaling follows a natural progression from simple optimization to complex distributed systems. Each tier builds on the previous.

What interviewers want to hear: "Before adding infrastructure, I would optimize the existing database. If that is not enough, I would scale horizontally. Caching is my third tier - it gives the biggest throughput gains but adds the most complexity." Most candidates jump straight to Redis. Start with what you have.

Tier 1: Optimize Within Your Database

Before adding more infrastructure, there is typically plenty of headroom in your existing database. Most read scaling problems can be solved with proper tuning and smart data modeling.

Indexing

An index is a sorted lookup table that points to rows in your actual data. Think of it like a book index - instead of scanning every page to find mentions of "database," you check the index at the back which tells you exactly which pages to look at.

Without an index, the database performs a full table scan - it reads every single row to find matches. With an index, it jumps directly to the relevant rows. This turns a linear O(n) operation into a logarithmic O(log n) operation: the difference between scanning 1 million rows versus checking about 20 index entries.

Schema: users table

id integer, primary key - Unique user identifier

email varchar(255), indexed - Most common lookup field

name varchar(100) - Display name

created_at timestamp, indexed - Account creation date

Without index - Full table scan of 10M rows. Cost: ~412,000 units. Time: 100-500ms. The database reads every row checking if email matches.

With index on email - Index scan. Cost: ~8 units. Time: 1-5ms. Jumps directly to the matching row.

What to index: Columns you filter on, join on, or sort by. If users search posts by hashtag, index the hashtag column. If you sort products by price, index price.

Composite indexes cover multiple columns. Column order matters - this is the leftmost prefix rule. An index on (status, created_at) helps WHERE status = 'active', and WHERE status = 'active' AND created_at > X, but NOT WHERE created_at > X alone.

Interview tip: When outlining your database schema, mention which columns you will add indexes to. This shows you are thinking about performance from the start.

Hardware Upgrades (Vertical Scaling)

Sometimes the answer is just better hardware. Boring, but effective.

SSDs instead of HDDs - 10-100x faster random I/O

More RAM - More dataset sits in memory instead of on disk

Faster CPUs, more cores - Handle more concurrent queries

The ceiling: A single machine tops out at roughly 50,000-100,000 reads per second. This is a rough estimate - the actual number depends on your read patterns, data model, and hardware. But in an interview, rough estimates are enough to justify your decision to scale horizontally.

It does not hurt to mention hardware scaling, though this is rarely what the interviewer is looking for, as it sidesteps the architectural question.

Denormalization

Normalization structures data to reduce redundancy by splitting information across multiple tables. This saves storage space but makes queries more complex because you need joins.

Denormalization is the opposite: store redundant data to avoid joins. You trade storage for speed.

Schema: Normalized design (separate tables, requires joins)

users - id (PK), name, email

orders - id (PK), user_id (FK), order_date, total

products - id (PK), name, price, category

Query: Order summary - Join across all tables. Expensive at thousands of QPS. The database matches foreign keys, combines results.

Schema: Denormalized design (single table, pre-joined)

- order_summary - id (PK), user_name, order_date, product_name, price

Query: Order summary - Single table read. No joins. 10x faster. Yes, you store the same user name in multiple places. But for a read-heavy system, this storage cost is worth the speed improvement.

Materialized Views

Materialized views take denormalization further by precomputing expensive aggregations. Instead of calculating average product ratings on every page load, compute them once via a background job and store the results.

The background job runs on a schedule (every 5 minutes), not on every page load. Reads become a simple lookup against the precomputed table. Especially powerful for analytics queries across large datasets.

Tier 2: Scale Your Database Horizontally

When a single database server hits its limits, add more servers. General rule: scale horizontally when you exceed 50,000-100,000 reads per second (assuming proper indexing).

Read Replicas

Read replicas copy data from your primary database to additional servers. All writes go to the primary, but reads can go to any replica. This distributes read load across multiple servers.

Leader-follower replication is the standard approach. One primary (leader) handles writes, multiple secondaries (followers) handle reads. Replication can be synchronous (slower but consistent) or asynchronous (faster but potentially stale).

Beyond throughput, read replicas also provide redundancy. If your primary database fails, you can promote a replica to become the new primary, minimizing downtime.

Replication lag is a common interview question. You need to understand the trade-offs: synchronous replication ensures data consistency but introduces latency. Asynchronous replication is faster but introduces potential stale reads. The "read-your-own-writes" pattern routes a user's reads to the primary for a short window after their write.

Database Sharding

Read replicas distribute load but do not reduce the dataset size each database handles. If your dataset becomes so large that even well-indexed queries are slow, sharding splits data across multiple databases.

For read scaling, sharding helps two ways: smaller datasets mean faster individual queries, and you distribute read load across multiple databases.

Functional sharding splits by business domain:

Geographic sharding is effective for global read scaling:

Hash-based sharding distributes records evenly:

Important: Sharding adds significant operational complexity and is primarily a write scaling technique. For most read scaling problems, adding caching layers is more effective and easier to implement.

Tier 3: Add External Caching Layers

When you have optimized your database but still need more read performance, add caching. Most applications exhibit highly skewed access patterns. Millions read the same viral tweets. Thousands view the same products. You are repeatedly querying for identical data that rarely changes.

Caches exploit this by storing frequently accessed data in memory. While databases read from disk and execute queries, caches serve pre-computed results directly from RAM. Sub-millisecond response times versus tens of milliseconds.

Application-Level Caching (Redis, Memcached)

In-memory caches sit between your application and database. On a cache hit, you get sub-millisecond response times. On a miss, you query the database and populate the cache for future requests.

Popular data naturally stays in cache. Celebrity profiles remain cached continuously due to constant access. Inactive user profiles get cached only when accessed, then expire after the TTL.

Cache Invalidation Strategies

Cache invalidation remains the primary challenge. When data changes, you need to ensure caches do not serve stale data. The diagram below shows all five strategies with their trade-offs:

TTL (Time-Based Expiration) - Set a fixed lifetime for cached entries. When the timer expires, the entry is evicted and the next request fetches fresh data from the database. Simple to implement because you set it once and forget it. The downside: you serve stale data until expiry. If data changes at minute 1 of a 5-minute TTL, users see stale data for 4 minutes.

Write-Through - Update the cache immediately whenever you write to the database. Both stores are always in sync. The cache is always fresh. The cost: every write now has two operations (cache + DB), which adds latency. If either fails, you need error handling to avoid inconsistency.

Write-Behind (Write-Back) - Write to the cache first, then asynchronously batch-write to the database in the background. Writes are fast because the user only waits for the cache write. But there is a window where the cache has new data that the database does not yet have. Risk: if the cache dies before flushing, data is lost.

Tagged Invalidation - Associate cache entries with tags like user:123:posts, user:123:feed, user:123:stats. When user 123 updates their profile, invalidate ALL entries tagged with user:123. Powerful for complex dependency graphs where one change affects many cached items.

Versioned Keys - Include a version number in the cache key: event:123:v42. When the data changes, increment the version to v43 and write new data as event:123:v43. The old entry (v42) is never deleted - it becomes unreachable because nobody asks for v42 anymore. No race conditions because version changes are atomic in the database.

How to set TTL: Ideally driven by non-functional requirements. "Search results should be no more than 30 seconds stale" gives you your exact TTL. User profiles tolerating 5-minute staleness get a 5-minute TTL. Most production systems combine approaches: short TTLs as a safety net, with active invalidation for critical data.

TTL guidelines by data type:

User profiles - 5-15 minutes (change rarely)

Feed/timeline - 30-60 seconds (update frequently)

Config/feature flags - 1-5 minutes (infrequent changes)

Session data - Hours (tied to user session)

CDN and Edge Caching

CDNs extend caching to global edge locations. A user in Tokyo gets cached data from a Tokyo edge server rather than a round trip to your Virginia data center. Response times drop from 200ms to under 10ms.

For read-heavy applications, CDN caching can reduce origin load by 90% or more. Product pages, user profiles, search results - anything multiple users request becomes a candidate. CDNs only make sense for data accessed by multiple users. Do NOT cache user-specific data like personal preferences or private messages - hit rate is zero.

Common Deep Dives (What Interviewers Ask Next)

"Queries slow as dataset grows"

Your app launched with 10,000 users and queries were snappy. Now you have 10 million users and simple lookups take 30 seconds. Without proper indexing, the database performs full table scans on every query. Finding one user by email means reading through all 10 million records.

The problem compounds with joins. Fetching a user's orders without indexes means scanning the entire users table, then the entire orders table. Billions of row comparisons.

Solution: Add indexes on columns you query frequently. An index on email turns that 10M row scan into a few index lookups. For compound queries, column order in the index matters. An index on (status, created_at) helps both status-only queries and combined queries, but NOT created_at-only queries.

"Millions of concurrent reads for the same cached data"

A celebrity posts on your platform. Suddenly millions of users try to read it. Your cache server getting 500,000 requests per second for a single key starts timing out. Even though data is in memory, serializing and sending it 500K times per second overwhelms any single server.

Request coalescing combines duplicate in-flight requests. If 1000 requests arrive for the same key at the same time, only one fetches from the backend. The rest wait for that result. This reduces backend load from potentially infinite to exactly N, where N is the number of application servers.

Cache key fanout distributes a single hot key across multiple entries. Store identical copies under feed:celebrity:1 through feed:celebrity:10. Clients randomly pick one. 500K RPS spread across 10 keys = 50K each. Trade-off: more memory and more complex invalidation, but you stay online.

"Cache stampede when popular entry expires"

Homepage data gets 100K RPS from cache. TTL expires. All 100K requests see a cache miss in the same instant. Every one tries to fetch from the database. Your DB, sized for 1,000 QPS during normal misses, gets hit with 100K identical queries. It crashes - a DDoS from your own application.

Distributed lock (SETNX) - First request acquires lock and rebuilds. Others wait. Downside: if rebuild fails or takes too long, thousands of requests timeout.

Probabilistic early refresh - As cache entry ages, each request has a small random chance of triggering a background refresh. At minute 50 of a 60-minute TTL: 1% chance. At minute 59: 20% chance. Spreads rebuilds across time instead of a single thundering herd.

Background refresh - Dedicated process refreshes critical entries before expiration. Users never trigger rebuilds. Most reliable for your most popular content.

"Cache invalidation when updates must be visible immediately"

A naive approach is delete the cache entry after a write. But this introduces race conditions: a request between delete and rewrite caches stale data again.

Cache versioning solves this. Each record has a version number in the database. On update, increment the version atomically. Cache key includes the version.

How it works: On READ: get version (43) from small version key, then lookup event:123:v43. On WRITE: update DB, increment version to 44, write new data as event:123:v44. Old entry event:123:v43 is never deleted - it becomes unreachable. No race conditions because version changes are atomic in the database. Two cache lookups per request (version + data), but no invalidation complexity.

When to Use in Interviews

Common scenarios with specific advice:

Bit.ly/URL Shortener - Extreme read/write ratio. Cache short-to-long URL mapping in Redis with no expiration (URLs don't change). Use CDN for global traffic. DB only hit for cache misses on unpopular links.

Ticketmaster - Event pages hammered when tickets go on sale. Cache event details, venue info, seating charts. But actual seat availability cannot be cached or you will oversell. Read replicas for browsing, write master for purchases.

News Feed (Facebook/LinkedIn/Twitter) - Pre-compute feeds for active users. Cache recent posts from followed users. Smart pagination - users typically only read the first few items, so cache recent content aggressively.

YouTube - Video metadata creates surprising read load. Cache titles, descriptions, thumbnails (don't change often). View counts can be eventually consistent, updated every few minutes. CDNs for thumbnail images.

When NOT to use:

Write-heavy - Uber location tracking (drivers update every few seconds). Read/write ratio might only be 2:1. Focus on write scaling.

Small scale - "Design for 1000 users." A well-indexed single database handles thousands of QPS. Show judgment by solving the actual problem.

Strongly consistent - Financial transactions, inventory. Stale data causes real problems. You might still cache but with aggressive invalidation and shorter TTLs.

Real-time collaborative - Google Docs needs real-time updates, not read scaling. Caching actively hurts when every keystroke needs to be immediately visible.

Remember: Read scaling is about reducing database load, not just making things faster. If your database handles the load fine but you need lower latency, that is a different problem with different solutions.

Summary

1. Optimize the database - Indexing turns O(n) into O(log n). Denormalization eliminates joins. Hardware buys time.

2. Read replicas - Nx capacity. Understand replication lag and read-your-own-writes consistency.

3. Sharding - Splits data across nodes. Functional, geographic, or hash-based. Primarily a write scaling technique.

4. Application caching (Redis) - Sub-millisecond reads. Cache-aside is the default. Master cache invalidation strategies.

5. CDN - Edge servers worldwide. Reduces origin load 90%+. Only for multi-user content.

In interviews, demonstrate that you understand both the performance benefits and the operational complexity of each approach. Show that you know when to use aggressive caching for static content and when to lean on read replicas for data that needs to stay fresh.