Practice

Cross-Topic Design Drills

These prompts force synthesis across networking, APIs, caching, placement, storage, messaging, and batch processing. Attempt each prompt for 45–60 minutes before reading the strong-answer signals. The numbers are intentionally incomplete: state assumptions and use them to choose architecture.

Practice protocol

  1. Spend 5–7 minutes on requirements and invariants.
  2. Make at least three estimates with units.
  3. Define API/events and the source of truth.
  4. Trace one write and one read.
  5. Deep-dive two listed areas.
  6. Draw two failure injects at random.
  7. Finish with metrics, security, migration, and a decision summary.
Drill 1 URL Shortener, Redirect Edge, and Analytics

Target: mid → senior
Related chapters: 3, 6, 7, 10, 11, 12, 17, 18, 20

Design a public URL-shortening service with custom aliases, very low-latency redirects, expiration, abuse controls, and near-real-time aggregate analytics.

Core requirements

  • Create a short link with optional custom alias and expiry; redirect by code.
  • Prevent two owners from claiming the same alias and make create retries safe.
  • Support link disable/delete, malware/phishing response, and tenant quotas.
  • Provide hourly click counts by link/country/device; raw analytics may lag.
  • Operate in multiple regions; define what happens to alias creation during partition.

Scale and SLO assumptions

  • 1 billion active links; 100 million creates/day; 20 billion redirects/day.
  • Average target URL 300 bytes; redirect payload is tiny; peak is 8× daily average.
  • Top 0.001% of links receive 25% of redirects.
  • Redirect p99 target 50 ms in-region; create p99 300 ms; analytics freshness 5 minutes.

Interviewer deep dives

  • Code generation and alias uniqueness: random/base encoding, collision retry, allocation ranges, or database sequence.
  • Cache/CDN key and invalidation for disable, expiry, and abusive links.
  • Multi-region authority for custom aliases versus local redirect availability.
  • Event pipeline for deduplicated/approximate analytics without slowing redirects.
  • Privacy retention, bot filtering, rate limits, and safe preview/interstitial behavior.

Failure injects

  • A celebrity link reaches 3 million redirects/s and the cache fleet is cold.
  • The abuse team must disable a phishing link globally in under 10 seconds.
  • Two regions are partitioned while both receive the same custom-alias request.
  • Analytics consumers fall four hours behind and raw events approach retention limits.
Strong-answer signals — open after attempting
  • Separates redirect authority from asynchronous analytics and states freshness for each.
  • Uses immutable/versioned target records or cache versioning and an emergency deny path.
  • Quantifies origin load as cache hit ratio changes and handles hot-key request coalescing/edge caching.
  • Defines create idempotency and custom-alias uniqueness under regional failure.
  • Includes bot/abuse controls, privacy, reconciliation, and online schema/key migration.
Drill 2 Multi-device Chat and Presence

Target: senior
Related chapters: 4, 5, 7, 8, 10, 11, 14, 16, 17, 20

Design direct and group chat with WebSockets, offline delivery, multi-device synchronization, read receipts, presence, media attachments, and regional operation.

Core requirements

  • At-least-once durable message delivery; no silent loss after accepted send.
  • Per-conversation ordering where practical; duplicate-safe clients.
  • Reconnect/resume after mobile network changes and gateway restarts.
  • Groups up to 100,000 members; most groups are below 100.
  • Presence can be approximate; message history and membership authorization cannot leak.

Scale and SLO assumptions

  • 50 million daily users; 8 million concurrent connections; 4 billion messages/day.
  • Average message 700 bytes; 2% include a 4 MiB attachment stored separately.
  • Peak send rate 300,000/s; reconnect event may affect 2 million clients.
  • History read p99 200 ms; online delivery target under 500 ms in-region.

Interviewer deep dives

  • Gateway connection state, heartbeats, authentication refresh, and reconnect jitter/resume token.
  • Conversation sequence/ID allocation, partitioning, and write acknowledgment.
  • Fan-out-on-write versus per-user/conversation pull, especially for huge groups.
  • Offline inbox, device cursors, deduplication, and read-receipt aggregation.
  • Attachment direct upload, malware scan, signed download, and deletion.

Failure injects

  • One gateway region restarts and two million clients reconnect in two minutes.
  • A 100,000-member group generates 50,000 messages/minute.
  • A membership removal races with cached authorization and message fan-out.
  • The primary conversation shard fails after commit but before send acknowledgment.
Strong-answer signals — open after attempting
  • Distinguishes transport connection from durable message state and uses resumable cursors.
  • Defines per-conversation ordering scope, idempotent message IDs, and ambiguous send retry.
  • Uses hybrid fan-out for large groups and quantifies write/read amplification.
  • Treats presence as TTL/approximate state and membership as authoritative/fail-closed.
  • Explains gateway draining, backpressure, per-tenant limits, and region failure behavior.
Drill 3 Personalized Social Feed

Target: senior
Related chapters: 3, 9, 10, 11, 12, 14, 16, 17, 20, 21

Design a home feed with follows, posts, ranking, pagination, deletes, privacy changes, celebrity accounts, and fresh-enough delivery.

Core requirements

  • Create posts and retrieve a stable ranked timeline with cursor pagination.
  • Support private accounts, blocks, deletes, and edits without leaking stale content.
  • Normal posts appear within seconds; ranking features may lag minutes.
  • Authors range from one follower to hundreds of millions.
  • Rebuild a user timeline/projection after corruption or algorithm change.

Scale and SLO assumptions

  • 200 million daily users; 500 million posts/day; 30 billion feed reads/day.
  • Median follows 300; p99 20,000; largest authors exceed 100 million followers.
  • Feed page 50 items; metadata 1 KiB/item; images/video are CDN/object references.

Interviewer deep dives

  • Fan-out-on-write, fan-out-on-read, and hybrid celebrity strategy.
  • Ranking pipeline, feature freshness, candidate generation, and pagination stability.
  • Privacy/delete propagation across caches, timelines, search, and CDN.
  • Hot partitions and tenant/user key strategy.
  • Source-of-truth posts/follows versus rebuildable timeline materialization.

Failure injects

  • A user with 200 million followers posts during a global event.
  • A block action must hide prior content within 30 seconds globally.
  • The timeline consumer is six hours behind, but direct profile reads are current.
  • A ranking release produces duplicates and missing items for 2% of users.
Strong-answer signals — open after attempting
  • Builds from access patterns and gives special treatment to skew instead of average fan-out.
  • Uses stable item IDs/cursors and versioned projections to handle ranking changes.
  • Separates immutable post authority from timelines/search/cache and defines reconciliation.
  • Gives privacy changes a faster deny/filter path than ordinary asynchronous propagation.
  • Quantifies fan-out writes, feed storage, queue age, and recovery drain.
Drill 4 Cloud File Storage and Sharing

Target: senior
Related chapters: 7, 10, 12, 15, 17, 19, 20, 21

Design a Dropbox/Drive-like service with folders, direct large uploads, versions, sharing links, sync, preview generation, privacy deletion, and disaster recovery.

Core requirements

  • Files up to 5 TiB with resumable multipart upload and byte-range download.
  • Folder hierarchy, rename/move, versions, soft delete, sharing permissions, and revocation.
  • Clients sync changes using a cursor and recover from missed notifications.
  • Preview/virus-scan/index pipelines may be asynchronous but must be complete/rebuildable.
  • Support regional data residency for enterprise tenants.

Scale and SLO assumptions

  • 100 million users; 10 billion objects; 5 PiB stored; 2 PiB/month egress.
  • Average file 5 MiB but heavy tail; top 1% of files hold most bytes.
  • 1 million uploads and 20 million downloads per hour at peak.

Interviewer deep dives

  • Transactional metadata tree versus immutable object versions and state machine.
  • Direct upload authorization, completion checksum, orphan cleanup, and quota.
  • Change log/cursor and multi-device conflict behavior.
  • Sharing authorization, signed URLs, CDN caching, and revocation latency.
  • Cross-region copy/move, legal hold, deletion evidence, and restore throughput.

Failure injects

  • A lifecycle rule accidentally targets a live prefix.
  • A sharing link is leaked and used for mass egress.
  • Metadata says ready but the referenced object version is missing.
  • A region must be evacuated while 10 TiB/hour of new uploads continue.
Strong-answer signals — open after attempting
  • Uses a database for ownership/metadata and object storage for immutable bytes with explicit states.
  • Makes operations idempotent and reconciles dangling metadata/orphan objects.
  • Treats signed URLs as bearer capabilities and includes revocation/cache-key behavior.
  • Separates versioning from independent backup and quantifies restore/migration bandwidth.
  • Plans deletion across versions, previews, indexes, replicas, caches, and retention policy.
Drill 5 Payment Ledger and Money Movement

Target: senior → staff
Related chapters: 3, 10, 15, 17, 18, 20

Design an internal ledger and payment-orchestration service that handles authorization/capture/refund, ambiguous processor responses, reconciliation, audit, and multi-region reads.

Core requirements

  • Every business operation is idempotent; money is neither created nor destroyed by software retry.
  • Immutable balanced ledger entries and explicit reversals/corrections.
  • External processors can time out after accepting a request.
  • Account balances and transaction history are auditable and privacy/compliance controlled.
  • Failover has explicit RPO/RTO; stale reads cannot authorize spend.

Scale and SLO assumptions

  • 30,000 payment operations/s peak; 20 ledger postings per high-level transaction in worst workflows.
  • Seven years of online/auditable history; 2 KiB average posting plus indexes.
  • Processor p99 3 seconds and occasional asynchronous webhook settlement.

Interviewer deep dives

  • Double-entry schema, posting transaction, currency/precision, and unique operation IDs.
  • State machine for external processor ambiguity and reconciliation files/webhooks.
  • Available balance, holds/reservations, contention, and account partitioning.
  • Outbox events and downstream projections without making them authoritative.
  • Failover acknowledgment, fencing, audit, backup/restore, and regional strategy.

Failure injects

  • The processor times out after a capture may have succeeded.
  • A failover promotes a replica 500 ms behind the former primary.
  • A replayed webhook arrives before an older state-change event.
  • A migration bug created a 0.01% ledger imbalance over two hours.
Strong-answer signals — open after attempting
  • Uses immutable balanced postings and corrections rather than editing historical money.
  • Scopes exactly-once honestly and handles processor ambiguity through stable IDs/status/reconciliation.
  • Makes authorization read the authoritative current balance or reservation, not a stale replica/cache.
  • Defines serializable/locking/conditional behavior and hotspot limits.
  • Includes continuous conservation checks, incident containment, and evidence-preserving repair.
Drill 6 High-demand Ticket Booking

Target: senior
Related chapters: 3, 7, 10, 11, 13, 15, 17, 20

Design seat discovery, temporary holds, payment, purchase, waiting room, and resale/cancellation for events with extreme synchronized demand.

Core requirements

  • Never sell the same seat twice; holds expire and release safely.
  • Users can browse availability, enter a fair waiting room, hold seats, and complete payment.
  • Payment can be ambiguous; retries cannot create duplicate orders.
  • Availability displays may be slightly stale, but final reservation cannot be.
  • Bots and one tenant/event must not overwhelm the whole platform.

Scale and SLO assumptions

  • One event has 80,000 seats and 10 million users arrive within five minutes.
  • Browse peak 1 million requests/s; hold attempts 200,000/s; successful purchases 5,000/s.
  • Hold duration 5 minutes; checkout p99 target 3 seconds excluding user payment challenge.

Interviewer deep dives

  • Seat/inventory transaction, lock/conditional update, lease token, and expiry worker.
  • Waiting-room admission, fairness, signed position/session, and bot resistance.
  • Cache of seat maps versus authoritative availability and invalidation.
  • Payment orchestration, idempotency, order state, and compensation/reconciliation.
  • Event-based sharding, hot row avoidance, and failure during hold/purchase cutover.

Failure injects

  • The hold-expiry worker is delayed by 20 minutes.
  • Database failover occurs after inventory decrement but before API response.
  • A botnet rotates IPs and attempts 5 million holds/minute.
  • The seat-map cache is stale after an event configuration correction.
Strong-answer signals — open after attempting
  • Treats browse as cacheable/stale and final hold/purchase as authoritative atomic operations.
  • Uses bounded admission before the database, not autoscaling as the first line of defense.
  • Stores lease expiry/token and validates ownership on purchase, with idempotent order/payment flow.
  • Explains fairness and abuse dimensions beyond per-IP rate limiting.
  • Quantifies hold cardinality, write contention, queueing, and capacity under failover.
Drill 7 Metrics and Log Analytics Platform

Target: senior → staff
Related chapters: 4, 5, 7, 16, 17, 19, 20, 21

Design multi-tenant telemetry ingest, durable buffering, near-real-time queries/alerts, long-term object storage, batch compaction, and deletion/retention controls.

Core requirements

  • Ingest metrics/logs from agents with batching/compression and tenant authentication.
  • Dashboards query recent data; alerts evaluate within one minute.
  • Raw data retained in object storage; older data compacted/downsampled.
  • No tenant can exhaust shared cardinality, partitions, or query capacity.
  • Backfill/replay and disaster recovery must be possible.

Scale and SLO assumptions

  • 20 million events/s peak; average encoded event 250 bytes after compression.
  • Largest tenant 12% of ingest; metric labels can create billions of series.
  • Seven days hot, 13 months warm/cold; queries range from seconds to months.

Interviewer deep dives

  • Agent-to-ingest protocol, acknowledgments, retry, local spool, and duplicate IDs.
  • Partition key and hot tenant/cardinality controls.
  • Stream aggregation/alert state versus immutable object files and batch correction.
  • Columnar file size/partitioning, compaction, manifests, and schema evolution.
  • Query admission, caching, approximate algorithms, and noisy-neighbor isolation.

Failure injects

  • A tenant deploys an unbounded user-ID label and creates 100× series cardinality.
  • The broker is unavailable for 30 minutes while agents continue producing.
  • Late events arrive three days after an incorrect clock deployment.
  • A privacy request requires deleting one user identifier from raw and derived data.
Strong-answer signals — open after attempting
  • Calculates byte rate, retention, broker partitions, object file count, and query amplification.
  • Uses quotas/admission by tenant/cardinality and separates critical alerts from bulk queries.
  • Combines streaming freshness with immutable batch correction/rebuild and explicit event-time handling.
  • Treats raw retained data as authority and publishes derived datasets through manifests/versioning.
  • Includes local spool/backpressure, duplicate-safe ingest, and privacy-aware schema/logging.
Drill 8 Distributed Job Scheduler

Target: senior
Related chapters: 3, 10, 14, 15, 17, 18, 20

Design a service for scheduled and ad hoc jobs with priorities, dependencies, retries, cancellation, worker leases, exactly-once-looking effects, and regional failover.

Core requirements

  • Create one-time/recurring jobs and DAG dependencies; query status and history.
  • Dispatch millions of short jobs and thousands of long jobs fairly across tenants.
  • Workers may crash, pause, or lose network; jobs can be retried safely.
  • Support cancellation, deadlines, delayed retry, and poison-job quarantine.
  • Avoid double ownership even during scheduler failover.

Scale and SLO assumptions

  • 100 million jobs/day; peak creation 50,000/s and dispatch 200,000/s.
  • Median duration 2 seconds, p99 30 minutes; payload/reference under 64 KiB.
  • Top tenant uses 15% of capacity; 1% of jobs require scarce GPU workers.

Interviewer deep dives

  • Job state machine, idempotency key, lease/fencing token, and heartbeat/visibility timeout.
  • Time-wheel/delay index or queue scheduling for billions of future jobs.
  • Fair scheduling, priorities, resource classes, quotas, and starvation prevention.
  • DAG dependency counters and atomic ready transition.
  • Scheduler leader/partition ownership, failover, stale worker completion, and recovery.

Failure injects

  • A scheduler partition leader is isolated but continues dispatching.
  • One tenant submits 50 million immediate jobs.
  • A long-running job finishes after its lease expired and a retry also finishes.
  • A bad release causes 30% of jobs to retry every second.
Strong-answer signals — open after attempting
  • Uses lease epochs/fencing and idempotent destination effects rather than claiming one execution.
  • Separates durable job state from ephemeral worker assignment and bounds prefetch.
  • Quantifies backlog/age and designs admission/fairness before autoscaling.
  • Classifies retries and gives DLQ/quarantine an owner and redrive path.
  • Handles recurring-job time zones, deduplication, migration, and regional authority explicitly.
Drill 9 Notification Delivery Platform

Target: mid → senior
Related chapters: 3, 7, 10, 15, 20

Design email, push, SMS, and in-app notifications with templates, preferences, scheduling, provider routing, retries, rate limits, and audit.

Core requirements

  • Accept idempotent send requests and asynchronous bulk campaigns.
  • Respect channel preferences, quiet hours, consent, suppression, and per-tenant quotas.
  • Render versioned templates and localize content.
  • Route across providers; ingest delivery receipts; avoid duplicate sends where possible.
  • Expire time-sensitive messages and expose status/analytics.

Scale and SLO assumptions

  • 2 billion notifications/day; peak 200,000/s; campaigns create 20× bursts.
  • Provider rate limits vary by tenant/channel; email payload average 15 KiB.
  • Transactional notification p99 enqueue 200 ms and delivery start under 30 seconds.

Interviewer deep dives

  • Request idempotency, recipient expansion, scheduling, and message state machine.
  • Preference/consent consistency and whether to snapshot or recheck at send time.
  • Provider adapter retries, ambiguous sends, dedupe keys, and reconciliation callbacks.
  • Priority isolation between password reset and marketing campaigns.
  • Template version, PII minimization, unsubscribe, retention, and audit.

Failure injects

  • The primary email provider times out after accepting messages.
  • A marketing campaign fills the queue while password-reset traffic spikes.
  • Preference service is unavailable; some users recently opted out.
  • A faulty template must be stopped globally after 5% of a campaign sent.
Strong-answer signals — open after attempting
  • Scopes idempotency through provider limitations and keeps an auditable state machine.
  • Uses priority/tenant queues and provider-aware rate/admission control.
  • Makes a risk-based fail-open/fail-closed choice for consent and sensitive messages.
  • Carries template/config version and supports pause/cancel/kill switches.
  • Measures queue age, provider response, receipt lag, duplicate rate, and business completion.
Drill 10 Video Upload, Processing, and Global Playback

Target: senior
Related chapters: 7, 12, 13, 19, 20, 21

Design creator upload, resumable transfer, transcoding, thumbnails, moderation, publication, adaptive streaming, CDN delivery, and view analytics.

Core requirements

  • Videos up to 8 hours/500 GiB; upload resumes and verifies integrity.
  • Generate multiple codecs/resolutions and publish only complete manifests.
  • Global playback starts quickly and supports seeking; private/unlisted access works.
  • Processing can take minutes but status is visible and retryable.
  • Delete/unpublish propagates to CDN, search, derivatives, and analytics policy.

Scale and SLO assumptions

  • 5 million uploads/day; 2 PiB/day new originals; 100 million concurrent viewers at peak.
  • Average playback 4 Mb/s; CDN hit ratio target above 98% by bytes.
  • Transcoding creates 2.5× original bytes and substantial CPU/GPU work.

Interviewer deep dives

  • Multipart direct upload, checksums, metadata states, and abandoned session cleanup.
  • Job graph, idempotent segment outputs, retries, priority, and publish manifest.
  • Segment/key/version design, HTTP ranges/streaming manifests, CDN cache/shield.
  • Hot live/premiere content, origin capacity, signed access, and revocation.
  • View-event sampling/deduplication and batch/stream analytics.

Failure injects

  • A global premiere creates 60 million viewers in one minute.
  • One transcoder version produces corrupt audio after partial rollout.
  • A rights takedown must prevent playback globally within 30 seconds.
  • CDN hit ratio falls from 99% to 90% after a cache-key change.
Strong-answer signals — open after attempting
  • Keeps application servers off the byte path and quantifies CDN-origin amplification.
  • Uses immutable segments/renditions plus an atomic versioned manifest for publication/rollback.
  • Makes processing jobs idempotent with checksums and versioned toolchain/config.
  • Includes hot-object shielding, signed authorization, range behavior, and emergency deny.
  • Separates playback SLO from analytics/moderation freshness and defines deletion workflow.
Drill 11 Search Autocomplete and Query Suggestions

Target: senior
Related chapters: 11, 12, 14, 16, 19, 20, 21

Design low-latency prefix suggestions personalized by locale and recent trends, with safe content filtering, offline rebuild, and online updates.

Core requirements

  • Return top suggestions for prefixes in under 50 ms p99 globally.
  • Blend historical popularity, recent trends, locale, and optional user history.
  • Filter unsafe/removed terms quickly; avoid leaking private queries.
  • Support full rebuild and canary of ranking/model/data versions.
  • Handle hot one-character prefixes and bot traffic.

Scale and SLO assumptions

  • 5 million suggestion requests/s peak; 10 characters per query session.
  • 500 million candidate phrases; top locales hold 80% traffic.
  • Trend updates every minute; historical rebuild daily from trillions of events.

Interviewer deep dives

  • Trie/FST or prefix index representation, shard key, memory footprint, and cache.
  • Offline MapReduce aggregation/top-K per prefix and immutable index publication.
  • Streaming trend deltas and merge with baseline without destabilizing ranking.
  • Personalization privacy and client/server cache-key explosion.
  • Safety deletion/deny layer and global rollout/rollback.

Failure injects

  • A breaking-news prefix becomes 100,000× hotter in seconds.
  • A bad model version must roll back globally without restarting every node.
  • A privacy bug put user identifiers into suggestion candidates.
  • Half the edge fleet serves the old index manifest after control-plane failure.
Strong-answer signals — open after attempting
  • Uses compact immutable index artifacts and atomic version pointer/canary rollout.
  • Precomputes bounded top-K per prefix and special-cases short/hot prefixes.
  • Separates recent streaming delta from batch-corrected baseline and defines freshness.
  • Includes a fast safety deny/filter independent of slow rebuild.
  • Quantifies memory, QPS, cache, shard fan-out, and rebuild/transfer time.
Drill 12 Global Rate Limiting and Quotas

Target: senior → staff
Related chapters: 3, 4, 10, 11, 13, 14, 17, 18

Design per-user, per-tenant, per-token, and global rate limits across many regions, including strict and approximate classes, burst handling, abuse, and control-plane updates.

Core requirements

  • Support token-bucket style rate/burst limits on API gateways.
  • Some expensive operations need strict global quotas; ordinary traffic can tolerate bounded overshoot.
  • Policy changes propagate quickly and old gateways fail safely.
  • Low added latency and high availability; one tenant cannot exhaust global state.
  • Expose remaining quota/retry semantics without creating a side channel.

Scale and SLO assumptions

  • 25 million checks/s across 40 regions; 100 million active keys/minute.
  • Gateway budget under 2 ms p99 for common checks.
  • Top tenant has 2 million keys and 15% traffic; policy updates 10,000/s.

Interviewer deep dives

  • Local token bucket, centralized counter, sharded counter, and regional escrow/leases.
  • Key hashing, hot keys, cache/local state, and failure/clock behavior.
  • Strict uniqueness/quota versus bounded overshoot during partition.
  • Policy distribution, versioning, rollback, and emergency deny.
  • Multi-dimensional limits, fairness, cost budgets, and observability.

Failure injects

  • Regions are partitioned for 20 minutes during an attack.
  • One globally popular API key reaches 3 million checks/s.
  • A policy push accidentally sets a major tenant limit to zero.
  • The centralized quota store is slow and gateway concurrency explodes.
Strong-answer signals — open after attempting
  • Classifies limits by strictness and uses local/escrow approximation where bounded overshoot is acceptable.
  • Quantifies key state, check rate, hot-key load, and cross-region latency.
  • Defines clock/lease/expiry behavior and fails open or closed by risk class.
  • Uses versioned policy with canary, cached last-known-good, audit, and kill switch.
  • Adds admission before the rate-limit dependency itself and avoids retry amplification.
Drill 13 Collaborative Document Editing

Target: senior → staff
Related chapters: 8, 10, 14, 16, 17, 18, 20

Design multi-user real-time document editing with offline changes, presence/cursors, history, permissions, snapshots, conflict handling, and multi-region sessions.

Core requirements

  • Edits from connected users appear quickly and converge; clients can reconnect/resume.
  • Offline edits merge or surface conflict without silent loss.
  • Document history, comments, access revocation, and export are supported.
  • Documents range from tiny notes to very large structured content.
  • Presence is ephemeral; document operations/history are durable.

Scale and SLO assumptions

  • 20 million concurrent users; 2 million active documents; 20 edits/s average per active document.
  • Largest live document has 10,000 editors; operations average 200 bytes.
  • Interactive update target under 200 ms in-region; snapshot every 1,000 ops or 5 minutes.

Interviewer deep dives

  • OT versus CRDT/operation log, operation identity, causality, and convergence invariant.
  • Per-document session leader/partition and multi-region routing/failover.
  • WebSocket backpressure, resume cursor, presence TTL, and reconnect storm.
  • Snapshot/compaction, immutable history, storage growth, and export consistency.
  • Permission revocation race and encryption/privacy deletion.

Failure injects

  • The document-session leader is isolated while clients remain connected to two regions.
  • A 10,000-user document receives 100,000 ops/s.
  • A client reconnects after one month with offline operations based on an old schema.
  • Access is revoked while edits and snapshots are in flight.
Strong-answer signals — open after attempting
  • States the convergence/ordering model rather than saying ‘use WebSockets and CRDT.’
  • Separates ephemeral presence from durable operations and uses resume/version tokens.
  • Defines single-document ownership or merge behavior during regional partition.
  • Bounds operation log with snapshots while preserving audit/replay and schema compatibility.
  • Handles hot documents, fan-out, backpressure, authorization refresh, and failback.
Drill 14 Ride Dispatch and Location Matching

Target: senior
Related chapters: 4, 8, 10, 14, 16, 17, 18, 20

Design real-time driver location ingest, nearby-driver search, trip request matching, offer/accept leases, pricing inputs, and regional failure behavior.

Core requirements

  • Drivers publish location every few seconds; riders request nearby available drivers.
  • One trip must not be confirmed with two drivers; offers expire quickly.
  • Location may be approximate/stale; trip state and payment cannot be ambiguous.
  • Cities/airports create extreme geographic hotspots.
  • Operate during mobile disconnects and partial regional service failures.

Scale and SLO assumptions

  • 10 million online drivers; location update every 4 seconds → 2.5 million updates/s.
  • 500,000 trip requests/minute peak globally; target match decision under 2 seconds.
  • One airport cell can contain 50,000 drivers and receive 5,000 requests/s.

Interviewer deep dives

  • Geospatial cell/index, time expiry, update partitioning, and hot-cell splitting.
  • Candidate search versus authoritative driver/trip reservation state.
  • Offer lease/state machine, idempotent accept, and stale client fencing.
  • Regional ownership, border/city routing, failover, and duplicate dispatch prevention.
  • Event stream for ETA/pricing/analytics without blocking dispatch.

Failure injects

  • A stadium exit creates 100× demand in four adjacent cells.
  • A driver accepts after the offer lease expired and another driver was confirmed.
  • The location index is current but the trip-state database is degraded.
  • A region partition splits a metro area across routing boundaries.
Strong-answer signals — open after attempting
  • Treats locations as TTL/approximate and reservations/trip transitions as authoritative conditional state.
  • Uses cell hierarchy/bucketing and special handling for hot geography.
  • Defines lease token/epoch and idempotent accept to fence stale offers.
  • Separates candidate ranking from final commit and degrades safely when authority is unavailable.
  • Quantifies update rate, cell fan-out, state expiry, and event amplification.
Drill 15 Feature Flag and Configuration Distribution

Target: senior → staff
Related chapters: 6, 7, 8, 10, 11, 12, 17, 18, 20

Design a control plane and low-latency data plane for feature flags/configuration with targeting, versions, audit, gradual rollout, rollback, SDK caching, and global propagation.

Core requirements

  • Operators create versioned config/flags with validation, review, scheduling, and audit.
  • SDK evaluation is local/low latency and supports last-known-good during control-plane outage.
  • Changes propagate globally within seconds; emergency disable is faster and reliable.
  • Targeting supports tenant/user attributes without leaking sensitive data.
  • Old SDKs remain compatible and malformed policy cannot crash applications.

Scale and SLO assumptions

  • 10 million SDK instances; 100 billion local evaluations/day; 100,000 config changes/day.
  • 100,000 tenants and 1 million flags/config objects; update payload median 20 KiB.
  • Normal propagation p99 10 seconds; emergency kill p99 2 seconds.

Interviewer deep dives

  • Authoritative versioned model, validation, approval, and optimistic concurrency.
  • Snapshot plus delta stream, CDN/poll/long connection, and resume cursors.
  • Atomic bundle publication, signatures/checksums, schema compatibility, and rollback.
  • SDK cache/last-known-good, fail-open/closed defaults, and stale policy bounds.
  • Tenant isolation, audit, experimentation consistency, and regional control-plane failure.

Failure injects

  • A malformed rule causes 10% of SDKs to throw during evaluation.
  • The control plane is unavailable during a production incident requiring an emergency kill.
  • Half the fleet misses a delta and has a 30-minute-old version.
  • Two administrators concurrently edit and publish the same flag.
Strong-answer signals — open after attempting
  • Separates strongly controlled authoring from highly available local evaluation.
  • Uses immutable versions/bundles with atomic pointer, validation, signature/checksum, and rollback.
  • Provides snapshot+delta reconciliation and measures fleet version distribution/completeness.
  • Defines risk-based last-known-good/default behavior and an independent emergency path.
  • Handles compatibility, audit, tenant policy, and safe organization-wide rollout.
Drill 16 Webhook Delivery Platform and Public API

Target: senior
Related chapters: 6, 7, 9, 10, 13, 15, 20

Design a public API plus reliable tenant webhooks with subscriptions, signatures, retries, ordering, replay, secrets rotation, quotas, and observability.

Core requirements

  • Tenants create webhook endpoints/subscriptions and receive immutable domain events.
  • At-least-once delivery with per-endpoint ordering where feasible; manual replay.
  • HMAC/signature verification, secret rotation, endpoint validation, and SSRF defense.
  • Slow/broken tenants cannot affect others; payload/schema evolves compatibly.
  • Public API supports idempotent mutations, pagination, and rate limits.

Scale and SLO assumptions

  • 10 million events/s internal peak; fan-out average 2.5 subscriptions/event.
  • One tenant owns 1 million endpoints; endpoint latency ranges 50 ms to timeout.
  • Delivery retention/replay 30 days; payload median 3 KiB.

Interviewer deep dives

  • Subscription index/fan-out, tenant partitioning, and hot-event handling.
  • Delivery attempt state, idempotent event ID, per-endpoint queue/order, and retry schedule.
  • Signing canonical payload, timestamp/replay protection, secret rotation, and SSRF/network egress.
  • Backpressure, endpoint circuit breaker, DLQ/disable policy, and customer diagnostics.
  • Schema/version compatibility, replay, deletion/privacy, and end-to-end trace IDs.

Failure injects

  • A tenant endpoint returns 500 for 24 hours and accumulates 500 million attempts.
  • A popular event fans out to 20 million subscriptions at once.
  • A signing-secret rotation leaves half of deliveries using an old secret.
  • A customer claims an event was never delivered, but broker metrics are green.
Strong-answer signals — open after attempting
  • Separates internal event authority from per-tenant delivery state and proves completeness per endpoint.
  • Uses stable event/delivery IDs, bounded retries, endpoint isolation, and safe replay.
  • Treats URLs as untrusted network destinations and includes SSRF/DNS/IP controls.
  • Scopes ordering and accepts that retries can reorder unless serialized per key/endpoint.
  • Gives customers observable attempt history while protecting payload secrets/PII.