20 System Design Concepts Explained in 10 Minutes

Summary

This video script offers an insightful guide for developers aiming to scale applications and enhance their system design skills. It covers essential concepts like vertical and horizontal scaling, load balancing, CDNs, caching, IP addressing, TCP/IP protocols, HTTP, REST, GraphQL, gRPC, WebSockets, and databases including SQL and NoSQL. It also delves into replication, sharding, and the CAP theorem, providing a comprehensive overview of building scalable and robust systems.

Takeaways

• 🚀 Vertical scaling involves adding more resources like RAM or upgrading the CPU to scale a server, but it has limitations.
• 🔄 Horizontal scaling adds server replicas to handle more requests, allowing for almost infinite scaling and redundancy but introduces complexity.
• 🔄 Load balancers distribute incoming requests to multiple servers, preventing overload and providing fault tolerance.
• 🌐 Content Delivery Networks (CDNs) serve static files from a network of servers worldwide to reduce latency and improve performance.
• 💾 Caching is a technique to store copies of data for faster retrieval, used at various levels including browser, disk, and CPU cache.
• 🌐 IP addresses uniquely identify network devices, and the Internet Protocol Suite (TCP/IP) defines the rules for data transmission over the internet.
• 🔄 TCP is a reliable protocol that ensures data packets are sent and received in order, and is the foundation for many other protocols like HTTP and WebSockets.
• 🌐 DNS translates domain names to IP addresses, with DNS records like A records mapping domains to server IP addresses for quick access.
• 📚 HTTP is an application-level protocol that simplifies data transmission over TCP, following a client-server model with request and response headers and bodies.
• 🔄 REST is a popular API pattern that standardizes HTTP APIs, making them stateless and following consistent guidelines for responses.
• 📈 GraphQL allows clients to make a single request to fetch exactly the resources they need, reducing over-fetching and the number of requests.
• 🔄 gRPC is an RPC framework that uses Protocol Buffers for efficient binary data serialization, improving performance over REST APIs that use JSON.
• 🔄 WebSockets enable bi-directional communication, allowing for real-time data transfer without the need for polling, as in chat applications.
• 💾 SQL databases organize data into tables and rows, providing efficient storage and retrieval with ACID compliance for reliability.
• 🔄 NoSQL databases drop the consistency requirement of ACID and the relational model, allowing for horizontal scaling and different data models like key-value, document, and graph stores.
• 🔄 Sharding and replication are techniques used in databases to scale horizontally and improve read performance, respectively.
• 🔄 The CAP theorem highlights the trade-offs in distributed systems between consistency, availability, and partition tolerance.
• 🔄 Message queues provide durable storage and can be used for decoupling parts of an application, allowing for asynchronous processing of data.

Q & A

What is the difference between vertical scaling and horizontal scaling?

Vertical scaling involves adding more resources like RAM or upgrading the CPU of a single server, which is easy but limited. Horizontal scaling, on the other hand, involves adding replicas of the server to handle subsets of requests, allowing for almost infinite scaling and redundancy without requiring high-end machines.

Why is a load balancer necessary in a horizontally scaled system?

A load balancer is needed to distribute incoming requests evenly across multiple servers, preventing any single server from being overloaded while others are underutilized. It also provides fault tolerance by ensuring that if one server fails, others can continue to handle requests.

What is a Content Delivery Network (CDN) and how does it work?

A CDN is a network of servers distributed worldwide that serves static files like images, videos, and sometimes HTML, CSS, and JavaScript files. It works by caching copies of files from the origin server onto CDN servers, which can be done on a push or pull basis, reducing latency and improving load times for users.

What is the purpose of caching in computer networks?

Caching is used to create copies of data so that it can be retrieved faster in the future. It reduces the need for repeated network requests, which can be expensive in terms of time and resources, by storing data at various levels, from browser cache to CPU cache.

How does the Domain Name System (DNS) translate domain names to IP addresses?

The DNS is a decentralized service that maps domain names to IP addresses using DNS records, such as the A record for address mapping. When a DNS query is made, the system uses these records to retrieve the corresponding IP address, which is then cached by the operating system to avoid repeated queries.

What is the main advantage of using HTTP over TCP for web browsing?

HTTP is an application-level protocol built on top of TCP that provides a higher level of abstraction, making it easier for developers to work with. It follows the client-server model and includes methods like GET and POST, which simplify the process of sending and receiving data over the internet without worrying about individual data packets.

What are the main differences between REST and GraphQL API patterns?

REST is a standardization for HTTP APIs that makes them stateless and follows consistent guidelines, using different HTTP status codes to indicate success or error. GraphQL, introduced by Facebook, allows clients to make a single request and specify exactly which resources they need, reducing over-fetching and the number of requests made.

How does gRPC differ from REST in terms of performance?

gRPC uses protocol buffers, which serialize data into a binary format that is more storage-efficient and faster to send over a network compared to JSON used in REST APIs. This results in a performance boost, especially for server-to-server communication.

What is the purpose of Websockets and how do they differ from HTTP?

Websockets provide bi-directional communication, allowing for real-time data transfer between a client and server. Unlike HTTP, which requires polling to check for new data, Websockets push updates immediately when new data is available, making them ideal for applications like chat apps.

Why are databases like MySQL and PostgreSQL preferred over storing data in text files?

Databases offer more efficient data storage using data structures like B-trees and allow for fast retrieval of data through SQL queries. They are also ACID-compliant, ensuring durability, isolation, atomicity, and consistency, which are crucial for maintaining data integrity.

What is the CAP theorem and how does it relate to database design?

The CAP theorem states that in the presence of a network partition, a database can only guarantee two out of three properties: consistency, availability, and partition tolerance. It helps database designers make trade-offs in replicated systems, often choosing between consistency and availability based on the system's requirements.

What is message queuing and how does it benefit a system with high data intake?

Message queuing involves using a system like a message queue to persist data before it can be processed, especially when the system receives more data than it can handle. This not only helps in managing data flow but also decouples different parts of an application, improving system architecture and reliability.

Outlines

00:00

🚀 Scaling Applications: Vertical vs. Horizontal Scaling

This paragraph introduces key concepts in system design, particularly focusing on scaling strategies for handling increased user load. It explains vertical scaling, which involves upgrading a single server’s resources, and contrasts it with horizontal scaling, where additional servers are added to distribute the load. The paragraph also highlights the importance of load balancers in horizontal scaling to ensure even distribution of requests and discusses the benefits of using content delivery networks (CDNs) for serving static files. Additionally, it briefly touches on caching mechanisms and the fundamentals of networking, such as IP addresses and TCP/IP protocols.

05:01

🌐 Understanding HTTP and API Patterns

This paragraph delves into the HTTP protocol and various API design patterns. It explains the client-server model and breaks down the components of an HTTP request into the header and body. The paragraph then introduces REST, a popular API pattern that emphasizes statelessness and consistent guidelines, and discusses how different HTTP status codes represent the outcome of requests. Other API patterns such as GraphQL, which allows more efficient data retrieval, and gRPC, which offers performance improvements through protocol buffers, are also covered. Additionally, WebSockets are mentioned as a solution for bi-directional communication, particularly useful in real-time applications like chat.

10:02

💾 Database Management: SQL, NoSQL, and Replication Strategies

This paragraph focuses on different database management approaches, starting with relational databases like SQL, which offer ACID compliance for data integrity. It then contrasts this with NoSQL databases, which are designed for horizontal scalability by dropping certain constraints like consistency. The paragraph also explores database scaling techniques such as sharding, which distributes data across multiple machines, and replication, where data is copied across different servers for redundancy and availability. It concludes by discussing the complexities of maintaining data consistency and availability in distributed systems, referencing the CAP theorem and PACELC theorem, and introduces message queues as a solution for handling large data streams.

Keywords

💡Vertical Scaling

Vertical scaling refers to the process of increasing the capacity of a server by adding more resources such as RAM or upgrading the CPU. It is a straightforward method to scale up a system, but it has limitations as there is a physical maximum to how much a single server can be upgraded. In the video, it is mentioned as an initial approach to scaling an application, but it is later contrasted with horizontal scaling as a more limited option.

💡Horizontal Scaling

Horizontal scaling is the practice of adding more servers or replicas to handle a growing number of requests, allowing for nearly infinite scalability. This method is more complex but offers redundancy and fault tolerance, as it eliminates the single point of failure present in vertical scaling. The video emphasizes the power of horizontal scaling in building scalable applications and introduces the concept of load balancers to manage traffic distribution among servers.

💡Load Balancer

A load balancer is a server that acts as a reverse proxy and directs incoming network traffic to the appropriate server or service based on the load balancing algorithm used. It plays a crucial role in horizontal scaling by distributing client requests evenly across multiple servers, ensuring no single server becomes a bottleneck. The script mentions algorithms like round-robin and hashing as methods for load balancing.

💡Content Delivery Network (CDN)

A Content Delivery Network is a distributed network of servers that deliver static files like images, videos, and web assets to users from the server closest to them. CDNs improve the performance and availability of web content by reducing latency and bandwidth usage. The video script explains how CDNs work by caching and distributing copies of data from the origin server to various geographical locations.

💡Caching

Caching is a technique used to store copies of data temporarily in various levels of a system, from browser cache to CPU cache, to speed up data retrieval. The video script discusses caching in the context of network requests and system memory, highlighting its role in reducing the cost and time of fetching data.

💡IP Address

An IP address is a unique identifier assigned to devices on a network, allowing them to communicate with each other. The video script touches on the importance of IP addresses in computer networking and how they are used in conjunction with the Internet Protocol Suite for data transmission.

💡TCP/IP

TCP/IP, also known as the Internet Protocol Suite, is a set of communication protocols used for transmitting data over the internet. The video script explains that TCP ensures reliable data transmission by reordering and resending missing packets, which is why it is the foundation for many other protocols like HTTP and WebSockets.

💡Domain Name System (DNS)

The Domain Name System is a decentralized service that translates human-readable domain names into IP addresses. The video script describes how DNS works, with the creation of A records to map domain names to server IP addresses, and the caching of these records by operating systems to reduce the need for frequent DNS queries.

💡HTTP

HTTP, or Hypertext Transfer Protocol, is an application-level protocol used for transmitting hypermedia documents, such as HTML. The video script explains that HTTP operates on top of TCP and follows a client-server model, with requests and responses structured with headers and bodies. It also introduces different API patterns built on HTTP, such as REST and GraphQL.

💡REST

REST, or Representational State Transfer, is an architectural style for designing networked applications, where APIs are stateless and follow uniform interface constraints. The video script mentions REST as a popular API pattern that uses HTTP methods and status codes to communicate the outcome of requests, with 200 indicating success and 400 or 500 indicating client or server errors, respectively.

💡GraphQL

GraphQL is a query language for APIs and a runtime for executing those queries against data sources. Introduced by Facebook, it allows clients to request exactly the data they need, reducing over-fetching and under-fetching. The video script contrasts GraphQL with REST, highlighting its ability to fetch multiple resources in a single request.

💡gRPC

gRPC is a high-performance RPC framework developed by Google, which uses protocol buffers for efficient, binary serialization of data. The video script describes gRPC as an improvement over REST APIs for server-to-server communication and mentions gRPC-web for browser support, emphasizing the performance benefits of using protocol buffers over JSON.

💡WebSockets

WebSockets provide a full-duplex communication channel over a single, long-lived connection, allowing servers to push data to clients immediately as it becomes available. The video script explains how WebSockets solve the problem of real-time communication in applications like chat apps, where HTTP polling would be inefficient.

💡SQL Database

SQL databases, or relational database management systems like MySQL and PostgreSQL, store data in a structured format using rows and tables. The video script discusses how SQL databases offer ACID compliance, ensuring data integrity and reliability, and contrasts them with NoSQL databases, which sacrifice some of these properties for scalability.

💡NoSQL Database

NoSQL databases are non-relational databases that do not use traditional tables and are designed for horizontal scaling and flexibility in data models. The video script explains that NoSQL databases drop the consistency requirement of ACID and the concept of relations, allowing for different types of databases like key-value stores, document stores, and graph databases.

💡Sharding

Sharding is a technique used to distribute data across multiple machines in a database to improve scalability. The video script describes sharding in the context of NoSQL databases, where a shard key determines the partitioning of data, allowing for horizontal scaling without the need for strict consistency.

💡Replication

Replication in databases involves making copies of data to improve read scalability and provide redundancy. The video script discusses two types of replication: leader-follower, where all writes go to a leader and are then replicated to followers, and leader-leader, where every replica can handle reads and writes but may lead to inconsistency.

💡CAP Theorem

The CAP theorem states that in the presence of a network partition, a distributed database can only guarantee two out of three desirable properties: Consistency, Availability, and Partition tolerance. The video script explains the trade-offs involved in replicated database design and mentions the more nuanced PACELC theorem as an alternative.

💡Message Queue

A message queue is a tool used to decouple and asynchronously process data in a system. The video script describes how message queues can handle situations where data is being received faster than it can be processed, allowing for persistent storage before processing and enabling different parts of an application to operate independently.

Highlights

Scaling apps from a single server to handling more users involves vertical and horizontal scaling.

Vertical scaling is limited, while horizontal scaling allows for almost infinite scalability with load balancing.

Load balancers distribute traffic evenly among servers to prevent overload and add fault tolerance.

Content Delivery Networks (CDNs) serve static files globally to reduce latency.

Caching techniques like CDNs improve data retrieval speed and reduce network requests.

IP addresses uniquely identify devices on a network, essential for computer communication.

TCP/IP protocols, including TCP and UDP, govern data transmission over the Internet.

Domain Name System (DNS) translates domain names to IP addresses for website access.

HTTP is an application-level protocol built on top of TCP for client-server communication.

REST API pattern standardizes stateless HTTP APIs with consistent response codes.

GraphQL allows for fetching specific resources in a single request, reducing over-fetching.

gRPC is an RPC framework using Protocol Buffers for efficient binary data serialization.

WebSockets enable bi-directional communication for real-time applications like chat apps.

SQL databases provide efficient data storage and retrieval with ACID compliance.

NoSQL databases sacrifice consistency for horizontal scaling and flexibility.

Sharding is a technique for horizontal scaling of databases using a Shard key.

Replication creates read-only copies of databases to scale reads, with leader-follower and leader-leader models.

The CAP theorem highlights the trade-offs between consistency, availability, and partition tolerance in distributed systems.

Message queues provide durable storage and enable system decoupling for data processing.

System design involves complex methods for efficient data storage and movement.

neetcode.io offers courses for beginners and interview preparation in system design.

Transcripts

do you want to level up from Junior Dev

so that you can build scalable apps or

maybe get a 100K pay Bump by passing

your system design interview round well

you're gonna need a box of neat codes 20

essential system Design Concepts which

include networking API patterns

databases and more all of which are

covered more extensively on neetcode.io

to start let's say you have a single

server which is accepting and fulfilling

requests for your application but now

we're getting more users so we need to

scale it the easiest thing to do would

be to add more resources like Ram or

maybe upgrade your CPU this is known as

vertical scaling it's pretty easy but

it's very limited a better approach

would be to add replicas so that each

server can handle a subset of requests

this is known as horizontal scaling it's

more powerful because we can almost

scale infinitely and we don't even need

good machines it also adds redundancy

and fault tolerance because if one of

our servers goes down all of the other

servers can continue to fulfill requests

this eliminates our previous single

point of failure but the downside is

that this approach is much more

complicated first of all how do we

ensure that one server won't get

overloaded while the others sit idle

well for that we'll need a load balancer

which is just a server known as a

reverse proxy it directs incoming

requests to the appropriate server so we

can use an algorithm like round robin

which will balance traffic by cycling

through our pool of servers or we could

go with another approach like hashing

the incoming request ID in our case the

goal is to even the amount of traffic

each server is getting but if our

servers are located all around the world

we could even use a load balancer to

route a request to the nearest location

which brings us to content delivery

networks if you're just serving static

files like images videos and sometimes

even HTML CSS and JavaScript files you

can just use a CDN it's a network of

servers located all around the world

cdns don't really run any application

logic they work by taking files hosted

on your server aka the origin server and

copying them onto the CDN servers this

can be done either on a push or pull

basis but cdns are just one technique

for caching which is all about creating

copies of data so that it can be

refetched faster in the future making

Network requests can be expensive so our

browsers will sometimes cache data onto

our disk but reading disk can be

expensive so our computers will copy it

into memory but reading memory can be

expensive so our operating systems will

copy a subset of it into our L1 L2 or L3

CPU cache but how do computers

communicate with each other well well

every computer is assigned an IP address

which uniquely identifies a device on a

network and to round it all out we have

the poorly named TCP slash IP or the

Internet Protocol Suite since it

actually includes UDP as well what but

focusing on TCP for a second there has

to be some set of rules AKA protocols

that decide how we send data over the

Internet just like how in real life we

have a system that decides how we send

mail to each other usually when we send

data like files they're broken down into

individual packets and they're sent over

the internet and when they arrive at the

destination the packets are numbered so

that they can be reassembled in the same

order if some packets are missing TCP

ensures that they'll be resent this is

what makes it a reliable protocol and

why many other protocols like HTTP and

websockets are built on top of TCP but

when you type neetcode.io into your

search bar how does your computer know

which IP address it belongs to well for

that we have the domain name system

which is a largely decentralized service

that works to translate a domain to its

IP address when you buy a domain from a

DNS registrar you can create a dnsa

record which stands for address and then

you can enter the IP address of your

server so then when you search and your

computer makes a DNS query to get the IP

address it'll use the a record mapping

to get the address and then your

operating system will cache it so that

it doesn't need to make a DNS query

every single time but wait a minute why

do we usually use HTTP to view websites

well TCP is too low level we don't want

to have to worry about individual data

packets so we have an application Level

protocol like HTTP which is what

developers like like you and I actually

use on a day-to-day basis it follows the

client server model where a client will

initiate a request which includes two

parts one the request header think of

that as the shipping label you put on a

package it tells you where the package

is going who it's from and maybe some

other metadata about the package itself

the second part is the request body

which is basically the package contents

to test it out just open your Dev tools

Network Tab and click subscribe go on

always taking a closer look at the

request we can see that the response

also includes a header as well as a body

but even with HTTP there are multiple

API patterns we could follow the most

popular one being rest which is a

standardization around HTTP apis making

them stateless and following consistent

guidelines like a successful request

should include a 200 code in its

response header whereas a bad request

from the client would return a 400 code

and an issue with the server would

result in a 500 level code graphql is

another API pattern introduced by

Facebook in 2015. the idea is that

instead of making another request for

every single resource on your server

like you would do with rest with graphql

you can just make a single request AKA a

query and you can choose exactly which

resources you want to fetch this means

you can fetch multiple resources with a

single request and you also don't end up

over fetching any data that's not

actually needed another API pattern is

grpc though it's really considered a

framework released by Google in 2016 it

was also meant to be an improvement over

rest apis it's an RPC framework mainly

used for server-to-server communication

but there's also grpc web which allows

using grpc from a browser which has also

been growing quickly over the last few

years the performance boost comes from

protocol buffers comparing them to Json

which is what rest apis use protocol

buffers are an improvement in that data

is serialized into a binary format which

is usually more storage efficient and of

course sending less data over a network

will usually be faster the downside is

that Json is a lot more human readable

since it's just plain text another app

layer protocol is websockets to

understand the main problem that it

solves let's take chat apps for example

usually when someone sends you a message

you receive it immediately if we were to

implement this with HTTP we would have

to use polling where we would

periodically make a request to check if

there was a new message available for us

but unlike HTTP 1 websockets support

bi-directional communication so when you

get a new message it's immediately

pushed to your device and whenever you

send a message it's immediately pushed

to the receiver's device and for

actually storing data we have SQL or

relational database Management Systems

like MySQL and postgres but why should

we use a database when we can just store

everything in text files stored on disk

well with databases we can more

efficiently store data using data

structures like B trees and we have fast

retrieval of data using SQL queries

since data is organized into rows and

tables relational database Management

systems are usually acid compliant which

means that they offer durability because

data is stored on disk so even if a

machine restarts the data will still be

there isolation means that different

concurrent transactions won't interfere

with each other atomicity means that

every transaction is All or Nothing

lastly we have consistency which means

that foreign key and other constraints

will always be enforced this one is

really important because it led to the

creation of nosql or non-relational

databases consistency makes databases

harder to scale so no SQL databases drop

this constraint and the whole idea of

relations all together there are many

types of nosql databases but popular

ones include key value stores document

stores and graph databases but going

back to consistency for a second if we

don't have to enforce any foreign key

constraints that means we can break up

our database and scale it horizontally

with different machines this technique

is called sharding but how do we decide

which portion of the data to put on

which machine well for that we usually

have a Shard key so given a table of

people our Shard key could be the ID of

the person but sharding can get

complicated a simpler approach is

replication if we want to scale our

database reads we can make read-only

copies of our database this is called

leader follower replication wherever

every right will get sent to the leader

who sends those to the followers but

every read could go to a leader or a

follower there's also leader leader

replication where every replica can be

used to read or write but this can

result in inconsistent data so it would

be best to use it where you can have a

replica for every region in the world

for example it can be complex to keep

replicas in sync so the cap theorem was

created to weigh trade-offs with

replicated design it states that given a

network partition in a database as

database designers we can only choose to

favor either data consistency or data

availability what makes it confusing is

that consistency here means something

different than with acid it's a somewhat

controversial theorem which is why the

more complete pack else theorem was

created lastly we have message cues

they're kind of like databases because

they have durable storage and they can

be replicated for redundancy see or

sharded for scalability but they have

many use cases if our system was

receiving more data than it can process

it would be good to introduce a message

queue so that our data can be persisted

before we're able to process it in doing

so we also get the added benefit that

different parts of our app can become

decoupled and if you've learned anything

by now it's that software engineering is

all about finding new and complicated

ways to store and move data around if

you want to learn more you can check out

my system design for beginners course on

neetcode.io as well as my system design

interview course thank you for

supporting my work and I'll see you soon