System Design interview fundamentals

Published in dns, job hunting, tech industry, on Feb 25, 2021

System design interview questions are to see how you would build a technical system, most likely web system. Like, how would you build a url shortener? Or how would you build Amazon? Uber? et al. Coding problems test your problem solving abilities. System design questions test your knowledge about how to build web systems, especially web systems at large scale.

Highly recommend this system design fundamentals course by AlgoExpert. The Eductive course comes highly recommended too :) Algo.monster is a comprehensive resource for coding interview prep and understanding CS concepts 📈

Please note: None of the images are at all related to the concept..

Client-server model

Foundation of how computers talk to each other. What happens when you go to a web page on the internet? A client (web browser) requests info from a web server (a.k.a application server). The client only knows that it can speak to servers, not anything about the servers themselves. Makes a DNS query to find the I.P. address. DNS query is for finding i.p. addresses. Websites have i.p. addresses. i.p. addresses are unique per machine. i.p. addresses are like mailboxes. Cloud providers grant these mail boxes (i.p. addresses) and servers for profit. We can trace the HTTP requests.

A server listens for requests on ports. There are 16,000 ports per machine. HTTP protocol is

  • Port 80 is for the World Wide Web and is represented as HTTP at the beginning of a website address. Through the Domain Name System (DNS) website urls like mysite.com get resolved into i.p. addresses like 192.168. 0.1 that a computer can match to a port.
  • Port 443 is for secure web communication. In practice this happens over HTTPS. Putting a site on HTTPS can incur you additional fees from your cloud service provider. You can do it yourself too using a free service like Let's Encrypt. I've previously written instructions about how to do this on Medium. HTTPS certification is definitely a big plus and offered by cloud hosting providers Render and Netlify as well as other players like Shopify, Wix and Squarespace. Most of these platforms provide HTTPS for free but is something to be conscious of while setting up your site and server as you could incur an additional fee for the registration. Google Domains is mostly my go to for DNS registration.

Web servers offering to accept and establish secure connections listen on port 443 for connections from web browsers desiring strong communication security.

DNS, i.p. addresses and HTTP protocols are open standards the web is built on. This brings us to...

Network Protocols

We're into the nerd realm now let's goooooo. Protocols are the rules for interactions. Systems have protocols, whether they be network protocols or communication protocols in human societies. "waving hello and getting a wave back" could be example of human communication protocol. For the internet we're primarily concerned by HTTP, followed by TCP and IP.

IP - Internet Protocol (i.p. address)

Data is sent in IP packet. Fundamental unit it's all built on. IP packets are made of bytes. There's the IP header and there's the data. Header contains useful information about the packet like the source i.p. address and destination of the i.p. address. You know where the ip address where it came from and where it's going to. This makes a big graph a.k.a a network or a spider web that you can "surf". It's an information superhighway.

TCP and HTTP

There's IP protocol, TCP protocol and HTTP. TCP is built on top of the internet protocol (IP) and HTTP is built on top of both protocols. IP packets have TCP headers that get used in networking and network engineering field. TCP connections happen as a handshake between computers, signaling that they're good to communicate with one another. Once a connection is established machines can send data back and forth to each other. TCP connections allow for sending arbitrary data in the form of IP packets. HTTP is a protocol on top of TCP and IP. Tim Berners-Lee invented the HTTP protocol and hence the internet. HTTP makes it easier for developers to build robust systems by providing structure and status codes for requests and responses. When building web systems we're primarily concerned with HTTP requests and HTTP responses instead of what's going on in the TCP and IP layers.

Side note: Al Gore did not invent the internet but he "was the first political leader to recognize the importance of the Internet and to promote and support its development." (-source from year 2000).

Code to send HTTP requests

Example code to send an HTTP request with Javascript:

const axios = require('axios');

// Make a request for a user with a given ID
axios.get('/user?ID=12345')
  .then(function (response) {
    // handle success
    console.log(response);
  })
  .catch(function (error) {
    // handle error
    console.log(error);
  })

Please note the above example uses everyone's favorite friend promises in Javascript.

PHP:

<?php
require_once "HTTP/Request.php";

$req =& new HTTP_Request("http://www.yahoo.com/");
if (!PEAR::isError($req->sendRequest())) {
    echo $req->getResponseBody();
}

or Python:

# importing the requests library 
import requests 
  
# api-endpoint 
URL = "http://maps.googleapis.com/maps/api/geocode/json"
  
# location given here 
location = "delhi technological university"
  
# defining a params dict for the parameters to be sent to the API 
PARAMS = {'address':location} 
  
# sending get request and saving the response as response object 
r = requests.get(url = URL, params = PARAMS) 
  
# extracting data in json format 
data = r.json() 

URL paths separate out logic on the server. Hitting different endpoints like /payments and /auth can do different things given the URL and the HTTP request type.

HTTP request types

  • GET
  • PUT
  • POST
  • DELETE

HTTP headers are key-value pairs that contain metadata for the request. There are standard HTTP header keys that developers can use to send their requests like User-Agent, Accept, Referer, Cache-Control and Host. You can see information about requests from your web browser, the client in the client-server relationship.

Data Storage

Data gets stored in a database. A database is a server. Even your own local computer could be configured to be used as a database. A database needs to be able to write and read data. Data can be stored on Disk or in Memory.

If data is saved on disk the data remains persistent even if the computer crashes or turns off. If something is in memory and the machine turns off the data will be lost.One advantage of reading and writing data to computer's memory instead of disk is that it's much faster than writing to disk. The issue with writing to memory is that the data is not persistent if the machine turns off.

Writing data to files is an example of writing to disk. Even if you reboot the machine the information in the file will stay the same regardless of if the computer turns off and on again. The file is stored "on disk". Writing to a variable in your programming language of choice will be stored in memory. If the server reboots (turns off and on again) data in memory is lost. Data storage can get more complicated, especially when thinking about distributed data storage and data consistency.

Latency and Throughput

Reading 1 megabyte from memory takes about 250 microseconds. Microsecond is only one millionth of a second. Reading one megabyte from memory is fast.

Reading sequentially from a Solid State Drive (SSD) takes about 1,000 microseconds. Reading from SSD drive is 400% (4x) slower than reading from memory. Packet from CA to Netherlands round trip takes 150,000 microseconds. Electricity has to travel. Not all systems need low latency but many do. Lag in video games means higher latency.

Throughput means how much data can be transfered from point A to point B in a given amount of time. Throughput is how many requests can go through the system. You pay cloud providers to increase throughput. You can add more servers to handle more throughput. Adding more servers isn't likely to reduce latency, unless the servers are physically closer to the end user or something..

Availability

Measure availability given the percent of time service is up in a year. People talk about availibility as "nines". If a system is 99.0% availibility that means that the system is down three and a half days a year. It's not a "look on the bright side" kind of number.. If you get up to five nines of availibility (99.999%) that means the system is only down for five and a half minutes in a year. More nines is better from an availability perspective.

5 nines (99.999%) means it's a High Availability (HA) system. Service Level Agreements (SLAs) gaurantee availability for customers. If the numbers are not met typically companies need to pay back money to customers.

Could come with higher latencies or lower throughput to have high availability. Not all systems need to be high availability.

  • No single points of failure. Redundancy can make systems more available.

Add servers then you need a load balancer. But then the load balancer is a single point of failure so you need multiple load balancers. Passive redundancy is when all machines work, if one dies all the machines keep working. Active redundancy is when there are multiple machines but only one or some of the machines are doing work. When one goes down the other machines somehow know and reconfigure themselves and start working. Active redundancy often requires "leader election" so that the machines know who's going to step up and work.

Caching

Caching is used in a lot of algorithms to improve time complexity. Caching is used to speed up a system, reduce latency. There's caching at the client, server or database level. There are even caches at the hardware level, CPU caching. Caching happens at many different levels of systems.

  • Reduce network requests
  • Avoid computationally long operations

Cache static content on the client. Redis is an in memory key value store that can be used on the server.

Writeback cache is when updates are only made to the cache then periodically the database is updated with the values from the cache.

Eviction policies: Least Recently Used (LRU) policy is to get rid of oldest cache data. Least Frequently Used (LFU) is removing uncommonly accessed data from cache.

Proxies

  • Forward proxy: server that sits between clients and another server. Forward proxy is like a client saying "hey proxy, communicate with this server on my behalf". Forward proxies can hide the identity of clients. The IP address for the client will be of the proxy server. This is how Virtual Private Networks (VPNs) work.

  • Reverse proxy: Reverse proxies interact on behalf of the server. Client issues request to the server. The request actually goes to the reverse proxy. Client thinks they're interacting with the server but they're actually interacting with a reverse proxy. Reverse proxies can be tricky to set up and configure but are useful. For instance, reverse proxies can be set up to filter out requests that you want to ignore or handling request logging. Reverse proxies can also be used for caching HTML pages or as a load balancer. Load balancers decide what server requests go to. This is useful for security purposes and for building applications with multiple servers to handle load.

Nginx is a web server that can be used as a reverse proxy. In an nginx.conf file Nginx can set headers on requests and forward requests to specified destinations. See the Nginx reverse proxy docs.

Load balancers

DNS Round Robin. Single domain name gets multiple IP addresses. IP addresses are returned in a load balanced way. To find the IP address of a website you can use the "dig" command from the CLI like dig google.com. There are software load balancers and hardware load balancers.

Weighted round robin if some servers are more powerful than others. Load balancers can perform health checks on servers. You can hash IP addresses so that requests from same client go to same server. This is useful for caching. There are a lot of Server-Selection Strategies.

Hashing

Hash requests that come into load balancer. MD5 or SHA1 are popular hashing functions. Missing cache hits if requests go to different servers. Consistent hashing and rendezvous hashing. Hashing is especially important if using server memory so requests can go to the same server.

Conclusion

Really this is the tip of the iceberg in terms of topics covered within a modern system design interview. This is very much something I'm still learning about! For comprehensive resources on system design interviews I recommended the knowledge products linked to at the top of this article :)