Networking Basics
Networking questions in system design are usually path questions: which names resolve to which addresses, which hops and middleboxes a flow crosses, where encryption terminates, where packets can be dropped or fragmented, and how retries amplify a partial failure. Senior answers move comfortably between application symptoms and network causes without treating the network as either magic or inherently unreliable.
How to use this chapter
- Read the mental model and mechanics without taking notes.
- Close the page and explain the topic aloud in five minutes.
- Work the quantitative example on paper.
- Answer the question ladder without revealing the answer signals.
- Complete one design drill and score yourself with the senior checklist.
Learning objectives
- Explain layers, encapsulation, IP addressing, subnets, routing, ports, and connection tuples.
- Describe ARP/NDP, gateways, NAT, firewalls, TLS termination, and common middleboxes.
- Break latency into propagation, transmission, serialization, processing, and queueing components.
- Reason about MTU, fragmentation, packet loss, jitter, bandwidth-delay product, and head-of-line effects.
- Design private/public connectivity, service discovery, and multi-region traffic paths.
- Use network telemetry and systematic debugging at a senior level.
Mental model
A network delivers packets hop by hop according to routing state. Applications usually address endpoints by name, establish transport state, negotiate security, and exchange messages whose bytes are split into packets. Every hop has a link rate, queue, policy, and failure mode. Latency is the sum of unavoidable propagation plus avoidable handshakes, serialization, processing, and queueing.
Core mechanics
Layers and encapsulation
Application protocols such as HTTP are carried by transport protocols such as TCP or QUIC/UDP, which are carried by IP, which is carried by a link technology such as Ethernet or Wi‑Fi. Each layer adds headers and its own addressing or state. The layered model is conceptual; real systems cross layers for performance and policy.
A connection is often identified by source address, source port, destination address, destination port, and protocol. Load balancers, firewalls, NAT devices, and connection tracking use some or all of this tuple. Long-lived connections therefore interact differently with backend changes than independent requests.
IP addressing, subnets, and routing
IPv4 and IPv6 addresses identify interfaces within routing scopes. A subnet prefix defines which addresses are considered directly reachable. Traffic to another subnet is sent to a router/default gateway. Routers use longest-prefix matching and control-plane protocols to populate forwarding tables.
Private address ranges are not globally routed; internet access commonly uses network address translation. In cloud designs, route tables, security groups/firewalls, gateways, and peering/transit constructs determine reachability. Overlapping private ranges complicate mergers, peering, and multi-cloud connectivity, so address planning is an architectural concern.
Local delivery: ARP and NDP
On an IPv4 local network, Address Resolution Protocol maps an IP address to a link-layer address. IPv6 uses Neighbor Discovery Protocol. Stale neighbor state, duplicate addresses, and broadcast/neighbor-table pressure can produce symptoms that look like intermittent application failure.
In most interviews you do not need packet-level protocol detail, but you should know that reaching an IP still requires local link resolution and that a “same subnet” path differs from a routed one.
NAT, firewalls, and middleboxes
NAT rewrites addresses and often ports, maintaining state for flows. Outbound NAT can exhaust ephemeral ports when many internal clients connect through one public address, especially with short-lived connections. Firewalls and security groups allow or deny traffic based on identity, addresses, ports, direction, and state.
Proxies, service meshes, WAFs, DDoS systems, VPNs, and TLS terminators can inspect or transform traffic. Each adds latency, capacity limits, configuration risk, and observability boundaries. Document the original client identity chain and never blindly trust forwarded headers from untrusted hops.
Latency, bandwidth, and queueing
Propagation delay is distance-limited. Transmission/serialization delay is packet size divided by link rate. Processing delay comes from protocol handling, encryption, and application work. Queueing delay grows under contention and dominates tails near saturation.
Bandwidth is a rate, not a guarantee of low latency. A high-bandwidth intercontinental link still has speed-of-light delay. Small flows can be handshake-limited; bulk transfers can be congestion-window or bandwidth-limited. The bandwidth-delay product estimates how much data must be in flight to fill a path.
MTU, fragmentation, and packet loss
The maximum transmission unit limits packet size on a link. IPv4 routers may fragment in some cases; IPv6 routers do not fragment transit packets, so endpoints rely on path MTU discovery. Blocking needed ICMP messages can create “works for small payloads, hangs for large payloads” failures.
Packet loss triggers transport recovery. Loss can result from congestion, faulty links, wireless interference, policers, or overloaded hosts. Even low loss can significantly hurt high-latency TCP throughput. Application retries are not a substitute for understanding whether the transport is already retrying.
TLS and trust boundaries
TLS authenticates endpoints and protects confidentiality/integrity in transit. Decide where TLS terminates: at the CDN, edge load balancer, sidecar, or application. Re-encryption between hops protects internal traffic but requires certificate issuance, rotation, identity, and debugging.
Server Name Indication and Application-Layer Protocol Negotiation help choose certificate/virtual host and protocol during handshakes. Certificate, clock, trust-store, and rotation failures can take down otherwise healthy services, so certificate expiry and handshake errors are production signals.
Network observability
Start at the application but validate each layer. Use request traces, connection errors, DNS timing, handshake timing, retransmission and loss counters, socket states, packet captures, route/neighbor tables, firewall logs, and load-balancer metrics. Compare affected versus healthy paths and narrow by region, ISP, protocol, address family, and payload size.
A packet capture shows what one interface observed, not the complete truth. Timestamps, offload features, sampling, and encrypted payloads affect interpretation. Senior debugging correlates multiple vantage points.
Decision table
| Decision | Prefer the first option when… | Prefer the second option when… | Senior caveat |
|---|---|---|---|
| Public endpoint vs private connectivity | Internet reachability and simple client access are required. | Controlled trust domains, lower exposure, or predictable routing justify private links/VPN/peering. | Private does not automatically mean authenticated or encrypted; public does not automatically mean insecure. |
| IPv4-only vs dual stack | Legacy constraints and limited scope make IPv4 sufficient. | Global scale, address pressure, or client networks require IPv6 support. | Dual stack doubles some test paths and can create asymmetric failures; monitor by address family. |
| Central NAT vs distributed egress | Simplicity and centralized policy outweigh port/capacity risk. | Scale, failure isolation, or source identity requires distributed egress. | Model ephemeral ports, connection reuse, logging, and failover state. |
| TLS at edge vs end-to-end TLS | Trusted internal network and operational simplicity are acceptable. | Zero-trust, compliance, or multi-tenant boundaries require protection to the workload. | Edge termination still needs authenticated origin connections and header trust rules. |
| One large packet vs many small packets | Payload and path MTU permit efficient larger frames. | Interactive latency or loss recovery benefits from smaller units. | Application messages are segmented by transport; packet-rate CPU and fragmentation both matter. |
| Cross-region synchronous call vs local replica/async | Strong immediate coordination is essential and latency budget allows WAN RTT. | Availability and latency favor local service with reconciliation. | WAN paths add variance and correlated dependency risk, not merely a fixed RTT. |
Quantitative reasoning
Failure modes and production signals
| Failure mode | What users see | Likely cause | Mitigation / design response | Useful signals |
|---|---|---|---|---|
| Routing blackhole | Timeouts to a subset of prefixes/regions | Bad route, withdrawn prefix, asymmetric policy | Fail over route, validate control plane, use path diversity | Reachability by prefix, traceroute/path telemetry, BGP/route changes |
| NAT exhaustion | New connections fail while existing ones work | Ephemeral port/state limit | Reuse connections, add egress IPs, distribute NAT, tune lifecycle | NAT ports, connection creates, TIME_WAIT, allocation errors |
| MTU blackhole | Small requests work; large TLS/data transfers stall | Path MTU discovery blocked or tunnel overhead | Fix ICMP/policy, lower interface/MSS appropriately, test path | Retransmissions, packet size correlation, ICMP too-big messages |
| Queue/bufferbloat | High and variable latency under upload/download load | Oversized queues near saturated link | Traffic shaping, fair queueing, capacity, application backpressure | Queue delay, RTT under load, drops, link utilization |
| TLS boundary failure | Handshake errors across many services | Expired cert, trust-store/clock/SNI issue | Automated rotation, overlapping validity, synthetic handshakes | Handshake failures by reason, cert expiry, clock skew |
| Asymmetric dual-stack issue | Only some clients fail or prefer a slower path | IPv6 route/firewall/DNS mismatch | Parity tests, happy-eyeballs support, family-specific rollback | Success/latency by IPv4 vs IPv6, route and firewall logs |
Senior-level lenses
Draw the path and ownership
For each hop, name who operates it and where telemetry exists. A third-party CDN, cloud load balancer, service-mesh proxy, and application team can each see different pieces. Incident response improves when the design includes correlation IDs, client IP provenance, and health checks at multiple layers.
Connection state changes rollout behavior
DNS or load-balancer changes affect new flows, while existing TCP/WebSocket connections can live for hours. Draining, GOAWAY/close signals, idle timeouts, and reconnect pacing are part of deployment and failover design.
Timeouts require a budget
A timeout should derive from the caller’s deadline and leave room for fallback or one bounded retry. Arbitrary 30-second defaults create thread/connection accumulation. Configure connect, handshake, request, idle, and overall deadlines separately where the stack permits.
Network failures are partial
One direction, one address family, one region pair, one payload size, or one ISP can fail. Health checks from a single location can declare a service healthy while users cannot reach it. Use diverse probes and degrade by segment rather than triggering unnecessary global failover.
Security policy is routing policy
Firewalls, identities, certificates, and egress controls determine which paths exist. Least privilege reduces blast radius but can make dependency discovery and emergency changes harder. Design automated policy testing and audited break-glass procedures.
Retries consume network and server capacity
A 2% loss event can create far more than 2% extra load if many layers retry independently. Coordinate retry ownership, cap attempts by deadline, add jitter, and expose retry-attempt headers/metrics so amplification is visible.
Interview question ladder
What happens when a host sends an IP packet to a different subnet?
Show strong-answer signals
It uses the route table/default gateway, resolves the next-hop link address with ARP/NDP, encapsulates the IP packet in a local frame, and routers forward by prefix.
What is the difference between bandwidth and latency?
Show strong-answer signals
Bandwidth is transfer rate; latency is delay. Explain propagation, serialization, processing, queueing, and why high bandwidth does not imply low RTT.
What is NAT and why can it be a bottleneck?
Show strong-answer signals
It rewrites addresses/ports and tracks flows; limits include port mappings, state table, throughput, and a centralized failure domain.
Why might small HTTP requests work while large ones time out?
Show strong-answer signals
MTU/path-MTU blackhole, fragmentation/tunnel overhead, proxy body limit, buffering, or bandwidth timeout. Propose payload-size tests and packet/ICMP inspection.
What is the bandwidth-delay product used for?
Show strong-answer signals
Amount of data in flight needed to fill the path; relates to windows, parallel streams, bulk transfer, and high-RTT links.
Where can TLS terminate in a web architecture?
Show strong-answer signals
CDN, edge LB/reverse proxy, sidecar, or app. Discuss re-encryption, identity, header trust, cert rotation, and observability.
Only clients from one mobile carrier experience 20% timeouts. How do you investigate?
Show strong-answer signals
Segment by IPv4/IPv6, DNS answer, edge PoP, path, MTU, TLS, payload, and time. Use diverse probes, carrier path data, CDN logs, packet captures, and controlled routing changes.
A service making short HTTPS calls through one NAT gateway intermittently cannot connect. Explain and fix it.
Show strong-answer signals
Ephemeral port/NAT state exhaustion, TIME_WAIT, poor pooling, destination concentration. Measure mappings, reuse connections, spread egress IPs/gateways, tune pools and timeouts safely.
Design network paths for a three-region API with private databases and public clients.
Show strong-answer signals
Global DNS/anycast/edge, DDoS/WAF, regional L7 LB, private app subnets, controlled egress, database routes, TLS boundaries, health/failover, and asymmetric/partition behavior.
How do you allocate a 500 ms end-to-end deadline across five hops?
Show strong-answer signals
Reserve client/network variance, assign per-hop budgets based on service objectives, propagate deadline, fail fast on insufficient remaining time, bound retries, and collect remaining-budget traces.
Design a migration from overlapping RFC1918 networks after an acquisition without a flag day.
Show strong-answer signals
Inventory flows, introduce proxies/NAT or translation zones, new non-overlapping ranges, DNS/service discovery indirection, dual connectivity, observability, staged workload moves, and eventual removal.
A global incident affects only long-lived connections after a load-balancer configuration change. Explain mechanisms and rollout safeguards.
Show strong-answer signals
Existing flow pinning, stateful NAT/LB tables, idle/drain timeouts, backend membership, proxy protocol changes, reconnect storm. Canary by connections, connection-age metrics, graceful drain, rollback.
Design drills
Trace a mobile client request to api.example.com through recursive DNS, CDN, WAF, regional load balancer, service mesh, application, and database. Mark every address translation, TLS boundary, timeout, and health signal.
What the interviewer is testing
End-to-end path literacy, trust boundaries, and observability ownership.
A database ships 500 MiB/s between regions over a 60 ms RTT path and cannot exceed 180 MiB/s on one stream. Diagnose using BDP, windowing, CPU, compression, loss, and parallelism; propose a safe benchmark plan.
What the interviewer is testing
Quantitative network reasoning and avoiding a simplistic “buy more bandwidth” answer.
Design connectivity for 2,000 customer VPCs to reach a SaaS API privately while tenants have overlapping address spaces. Include identity, routing scale, DNS, quotas, and incident isolation.
What the interviewer is testing
Private-link/proxy patterns, tenant isolation, address overlap, control-plane scale, and supportability.
Common weak answers and how to improve them
“The network is slow.”
Show the stronger answer
Locate latency by DNS, connect, TLS, request, and downstream spans; segment by path and failure domain.
“Use a bigger bandwidth link.”
Show the stronger answer
Check RTT/BDP, loss, packet rate, CPU, queueing, windows, and whether the workload is operation- or byte-bound.
“Private subnet means secure.”
Show the stronger answer
Add authenticated identity, encryption, least-privilege policy, egress controls, and audit.
“DNS failover moves all traffic immediately.”
Show the stronger answer
Account for recursive/client caching and long-lived existing connections.
“Retries fix packet loss.”
Show the stronger answer
Transport already retransmits; application retries can amplify load and must be deadline- and idempotency-aware.