System Design BASICS: Horizontal vs. Vertical Scaling

Summary

This video script introduces the fundamentals of system design, starting with the concept of an algorithm turned into an API accessible over the internet. It covers the transition from a single desktop to cloud computing, explaining the importance of scalability through vertical and horizontal scaling. The script discusses the trade-offs between the two, highlighting the benefits of a hybrid approach that combines speed, consistency, and resilience. It concludes by emphasizing the importance of designing a system that is scalable, resilient, and consistent, acknowledging the inevitable trade-offs in the process.

Takeaways

• 💡 The video is an introduction to system design, aimed at beginners.
• 💻 An algorithm written on a computer can be exposed via an API (Application Programming Interface) for others to use, turning it into a service.
• 🔌 The interaction with the service involves sending a 'request' and receiving a 'response'.
• 🖥️ To support the service, a computer might need to be connected to a database and configured with specific endpoints.
• ⚡ Considerations for system reliability include handling power loss scenarios to ensure continuous service.
• 🌐 Hosting services on the cloud is recommended for scalability and reliability, as it provides access to a set of computers for running algorithms.
• 💨 The cloud is essentially a collection of computers that can be rented for computation power.
• 🔍 Scalability can be achieved through vertical scaling (upgrading to a bigger machine) or horizontal scaling (adding more machines).
• 🔄 Vertical scaling involves a single machine and can be limited by hardware constraints, while horizontal scaling involves multiple machines and can be more resilient.
• 🔗 Network calls between servers in horizontal scaling are slower compared to interprocess communication in vertical scaling.
• 💾 Data consistency can be challenging in horizontally scaled systems due to the complexity of managing transactions across multiple servers.
• 🛠️ A hybrid approach combining the strengths of both vertical and horizontal scaling is often used in real-world systems.

Q & A

What is the purpose of exposing code through an API?

The purpose of exposing code through an API (Application Programming Interface) is to allow users to access and use the code remotely over the internet without needing to physically possess the computer where the code is running. This enables the code to be used by multiple users simultaneously, which can be monetized.

What is the difference between a request and a response in the context of APIs?

In the context of APIs, a request is a message sent to the API to perform a specific action or retrieve data. A response is the result or output that the API sends back after processing the request. Essentially, the request initiates the interaction, and the response concludes it by providing the desired information or result.

Why is it important to consider the need for a database when setting up a computer to run an API?

A database is often necessary when setting up a computer to run an API because it serves as a storage system for the data that the API needs to access or manipulate. This can include user data, application data, or any other relevant information that needs to be retrieved, updated, or managed through the API.

What is the significance of hosting services on the cloud?

Hosting services on the cloud is significant because it provides scalability, reliability, and cost-effectiveness. Cloud services can dynamically adjust to the demand, ensuring that the service remains available even during high traffic. Additionally, cloud providers handle much of the maintenance and security, allowing the focus to be on business requirements rather than infrastructure management.

What is the fundamental difference between a desktop and the cloud in terms of computing?

The fundamental difference between a desktop and the cloud is not in the nature of computing but in the accessibility and management. The cloud is essentially a set of computers provided by a service provider for a fee. These computers can be accessed remotely, similar to a desktop, but they offer the advantage of being scalable and managed by the service provider, reducing the need for local maintenance.

Why is scalability an important consideration in system design?

Scalability is crucial in system design because it determines the system's ability to handle an increasing number of requests as user demand grows. A scalable system can efficiently manage more connections and data, ensuring that the service remains responsive and reliable without degradation in performance.

What are the two main approaches to increasing the scalability of a system?

The two main approaches to increasing the scalability of a system are vertical scaling and horizontal scaling. Vertical scaling involves increasing the capacity of a single machine by making it more powerful, while horizontal scaling involves adding more machines to distribute the workload.

What is the primary advantage of vertical scaling compared to horizontal scaling?

The primary advantage of vertical scaling is that it can provide faster processing due to the increased capacity of a single machine. Additionally, it simplifies data management and ensures data consistency, as all data is stored and processed in one location.

What is the main disadvantage of horizontal scaling in terms of data consistency?

The main disadvantage of horizontal scaling in terms of data consistency is that it can become complex to manage and maintain data integrity across multiple machines. This can lead to issues such as dirty reads or inconsistent data states, requiring more sophisticated mechanisms to ensure transactional integrity.

Why might a system designer choose a hybrid approach combining both vertical and horizontal scaling?

A system designer might choose a hybrid approach to leverage the benefits of both vertical and horizontal scaling. This can provide a balance between the fast processing and data consistency of vertical scaling and the resilience and scalability of horizontal scaling. It allows for efficient resource utilization and better adaptability to varying loads.

What are the key considerations in system design according to the script?

The key considerations in system design, as outlined in the script, are scalability, resilience, and data consistency. Designers must balance these factors to create a system that meets business requirements and can adapt to changing conditions without compromising performance or reliability.

Outlines

00:00

💻 Basics of System Design and API Introduction

This paragraph introduces the fundamental concepts of system design, particularly for those new to the field. It explains the process of turning an algorithm into a service by exposing it through an API over the internet. The paragraph discusses the importance of hosting services on the cloud for reliability and the basic differences between a desktop and cloud computing. It also touches on the concepts of scalability, vertical scaling, and horizontal scaling, explaining how they are used to handle increased demand for a service.

05:01

🔄 Scalability Options: Vertical vs. Horizontal

The second paragraph delves deeper into the scalability options available for system design. It contrasts vertical scaling, which involves increasing the capacity of a single machine, with horizontal scaling, which involves adding more machines to distribute the load. The paragraph highlights the pros and cons of each method, including load balancing, resilience to failure, data consistency, and hardware limitations. It also introduces the concept of a hybrid solution that combines the strengths of both scaling methods, suggesting an initial focus on vertical scaling before moving to horizontal scaling as user trust and demand grow.

Keywords

💡System Design

System design refers to the process of planning and creating a system, particularly in the context of software and computing. It involves understanding the requirements and constraints to build a system that meets those needs. In the video, system design is the overarching theme, focusing on how to expose an algorithm as a service and handle various challenges such as scalability and reliability.

💡Algorithm

An algorithm is a set of rules or steps to be followed in calculations or other problem-solving operations, especially by a computer. In the script, the algorithm is the core functionality that the video discusses, emphasizing how it can be exposed as a service and utilized by paying customers.

💡API (Application Programming Interface)

An API is a set of protocols and tools for building software applications. It allows different software components to communicate with each other. In the video, the API is used to expose the algorithm to users over the internet, enabling them to send requests and receive responses.

💡Request

A request in the context of computing is a message sent to a server or a service asking for a certain action or information. The video explains that when users want to use the exposed algorithm, they send a request, which triggers the algorithm to process and return a response.

💡Response

A response is the answer or result returned by a server or a service in response to a request. The script mentions that the output of the algorithm is returned as a response when a request is made, highlighting the interaction between the user and the system.

💡Database

A database is a structured set of data held in a computer, especially one that is accessible in various ways. In the video, a database is mentioned as a potential requirement for the system, suggesting that it might be used to store data or results generated by the algorithm.

💡Cloud

The cloud refers to remote servers on the internet used for data storage and access, rather than local servers or personal computers. The video discusses hosting services on the cloud as a way to ensure reliability and scalability, contrasting it with traditional desktop setups.

💡Scalability

Scalability is the ability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged to accommodate that growth. The script discusses two types of scalability: vertical and horizontal, explaining how they can be used to handle more requests as the system grows.

💡Vertical Scaling

Vertical scaling, also known as scaling up, involves increasing the capacity of a single server or system by adding more resources such as CPU, RAM, or storage. The video contrasts vertical scaling with horizontal scaling, noting that it involves making a single machine more powerful.

💡Horizontal Scaling

Horizontal scaling, or scaling out, involves adding more machines or servers to a system to handle more requests. The video explains that this approach distributes requests across multiple machines, enhancing the system's ability to handle increased load.

💡Load Balancing

Load balancing is the distribution of workload across multiple computers or a computer cluster to ensure no single system is overwhelmed and to ensure redundancy. The script mentions load balancing in the context of horizontal scaling, where it is necessary to distribute requests among multiple servers.

💡Data Consistency

Data consistency refers to the accuracy and reliability of data across a system. The video discusses the challenges of maintaining data consistency in a distributed system, especially when using horizontal scaling, where transactions might need to be coordinated across multiple servers.

💡Hybrid Solution

A hybrid solution combines elements of different approaches to solve a problem. In the video, a hybrid solution is suggested for system design, where systems start with vertical scaling for fast interprocess communication and data consistency, and then move to horizontal scaling for better scalability and resilience.

Highlights

Introduction to system design basics for beginners.

Concept of exposing code through an API for internet use.

Explanation of request-response model in APIs.

Importance of hosting services on the cloud for reliability.

Clarification of cloud computing as a set of computers provided for a fee.

The role of cloud providers in managing server configurations and reliability.

Discussion on handling increased load with scalability.

Definition and explanation of vertical scaling.

Definition and explanation of horizontal scaling.

Comparison of pros and cons between vertical and horizontal scaling.

The concept of load balancing in horizontal scaling.

Advantages of single machine architecture in terms of data consistency.

Challenges of maintaining data consistency in a distributed system.

Differences in communication speed between interprocess and network calls.

Hardware limitations and the practicality of vertical scaling.

The linear scalability of horizontal scaling with user addition.

Real-world application of a hybrid approach combining vertical and horizontal scaling.

Strategic approach to scaling: starting with vertical and transitioning to horizontal as trust grows.

The essence of system design: balancing scalability, resilience, and consistency.

Transcripts

This video is on the basics of system design.

If you have never designed a system before, this is probably the place to start.

So imagine you have a computer with you in which you have written an algorithm.

So some code is running on this computer and this code is

like a normal function. It takes some input and it gives out an output.

Now people look at this code and they decide that this is really useful to them.

So they're ready to pay you so that they can use that code.

Now you cannot go around giving your computer to everybody.

So what you do is you expose your code using some

protocol, which is going to be running on the internet,

and by exposing your code using something called an

a p i application programmable interface, when your code does run,

it'll give an output and instead of storing that in the file or storing it in

some database or something like that,

you return that and that's called a response. Interestingly,

the thing that is sent to you is called a request

where people request you. So that's what it is. There's a request sent,

and for each request,

there's a corresponding response that your computer will be sending back.

Imagine setting up this computer.

It might require a database to be connected to it.

It's within the desktop itself.

You might require to configure these endpoints that people are connecting to.

And you also need to take into consideration what happens if there's a power

loss. If someone pulls the plug or something like that,

you cannot afford to have your service go down because there's lots of people

paying money for you. You should host your services on the cloud.

So what's the difference between a desktop and a cloud? Nothing really.

The cloud is a set of computers that somebody provides to you for money,

of course. So if you pay a cloud solution, for example,

Amazon Web Services, which is the most popular one, if you pay these guys,

they're going to give you computation power.

Computation power is nothing but a desktop that they have somewhere which can

run your algorithm. How will you actually store your algorithm in that desktop?

Well, you can do something like a remote login into that

desktop. That's what the cloud is. It's a set of desktops,

not necessarily desktops,

but a set of computers that you can use to run your service.

The reason we like to do this is because the configuration,

the settings the reliability can be taken care of to a large extent by the

solution providers.

So now that we have our server hosted on a cloud,

which is basically some computer that we don't know about,

we can focus on the business requirements.

What business requirements could we possibly have? Well,

there's lots of people who are using algorithm now,

and it gets to a point where the code that you have running on the machine is

not able to handle all of these connections. So what do you do?

One of the solutions is to buy a bigger machine, Right?

This is solution number one. The solution number two is to buy more machines.

The ability to handle more requests by buying more machines or buying bigger

machines is called scalability.

And this is a very important term that we need to understand. Well,

like we said, we can handle more requests by throwing more money at the problem.

When you're buying bigger machines,

it means that your computer's going to be larger and therefore it can process

the requests faster. So that is called vertical scaling.

And when you're buying more machines,

it means that the request can fall on any one of these machines and it'll be

processed, but because you have more of them,

the requests can be randomly distributed amongst the machines that you have just

bought. And that is called horizontal scaling.

These are two mechanisms by which you can increase the scalability of your

system. Like we said, scalability is being able to handle more requests.

Like any two approaches, we can compare them with the pros and cons.

The first one that we have talked about is we need some sort of load balancing

here. Well, that's not the case here.

If you have a single machine, there's no load to balance as such.

The second point is that with lots of machines,

if one of the machine fails, you can redirect the request to the other ones.

While over here, there's a single point of failure.

So this is a single point failure and here

it is resilient. The third thing to note is that

all the communication that we have between the servers will be over the network

and network calls us slow.

It's io while over here you have interprocess communication.

So that is quite fast. So here,

there is interprocess communication.

While over here we have network calls

Between two services. So that is remote procedure calls.

So this is slow and this is fast.

The fourth point is data consistency. For example,

let's say you are having a transaction where 3 cents some data to four and then

4 cents it to five and 5 cents it to one.

Here you see that the data is complicated to maintain.

If there is a transaction where the operation has to be atomic,

what could happen is that we have to lock all the servers, right?

All the databases that they're using, which is impractical.

So usually what happens is we have some sort of loose transactional guarantee,

and that's, that's the reason why here, the data consistency is a real issue.

While over here, there's just one system on which all the data rec resides,

and that's why this is consistent.

The final point deals with some hardware limitations that you're gonna have

because we cannot just make the computer bigger and bigger and bigger and solve

the problem. There's going to be some hardware limit that we have here.

Point number five, and over here,

this scales well in the sense that the,

the amount of servers that you throw at the problem is almost linear in terms of

how many users are added.

These are the five key differences that vertical scaling and horizontal scaling

have. So what do you think is used in the real world? Both.

we take some of the good qualities of vertical scaling,

which is really fast into process communication and the data being consistent.

So the cache is going to be consistent. There's no dirty reads, dirty rights,

so to speak.

We take these two good qualities from here and we take these two good

qualities from here, which is it scales well because the,

there's a hardware limit over here and it's also resilient in the sense that if

one of the server crashes, somebody else can come up. Okay?

So the hybrid solution is essentially horizontal scaling only,

where each machine has a big box.

I mean each machine, you try to take as big a box as possible,

as feasible money-wise. And then we,

we pick up a solution this way.

Initially you can vertical scale as much as you like later on when the users

start trusting you, you should probably go for horizontal scaling.

So these are the major considerations we have when designing a system.

Is it scalable? Is it resilient? And is it consistent

with these qualities? There's always gonna be some trade-offs that we have,

and that's what system design is.

We design a system which is going to meet the requirements,

and the requirements are such that it's going to be Computer science way

is possible to actually build a system like this.

If you have any doubts or suggestions, you can leave them in the comments below.

If you want notifications for further videos,

hit the subscribe button and I'll see you next time.