What is a Load Balancer?

Summary

In this IBM Cloud video, Bradley Knapp explores the concept of a network load balancer, crucial for managing traffic in high-demand websites. He illustrates the importance of load balancers in distributing user requests across multiple servers to prevent overload, ensuring scalability and reliability. The video discusses three load balancing methods: Round Robin for simple, sequential traffic distribution, smart load balancing for dynamic server load awareness, and random selection for a balance between simplicity and control. The discussion highlights the role of load balancers in cloud-native architectures and their impact on performance and cost.

Takeaways

• 🌐 A network load balancer is essential for distributing incoming traffic across multiple servers to ensure high availability and reliability of a website or application.
• 🚀 It's crucial for scaling a website to handle millions of users by directing traffic efficiently to multiple application servers, preventing any single server from becoming a bottleneck.
• 💡 Load balancers can be hardware devices or software-defined, and they intercept incoming traffic to distribute it among servers based on various algorithms.
• 🔄 The Round Robin algorithm is a basic load balancing method where traffic is distributed sequentially across servers, but it may not be ideal for long user sessions.
• 🤖 Smart load balancing involves load balancers working in cooperation with application servers, which report their load status to the balancer for more dynamic and efficient traffic distribution.
• 📈 Load balancers can help in auto-scaling by turning off underutilized servers or spinning up new ones based on real-time load data, thus optimizing resource usage and cost.
• 🛡️ Load balancers are a key component of cloud-native architectures, ensuring that traffic is evenly distributed among application servers, which in turn communicate with a common database.
• 🔑 There are more complex load balancing algorithms beyond Round Robin, such as random selection, that can provide better distribution without the need for smart load balancing's complexity.
• 💼 IBM Cloud offers expertise and support in designing and architecting load balancing solutions tailored to specific customer needs and traffic patterns.
• 🔧 For more detailed advice or specific scenarios, reaching out to IBM or exploring additional resources can provide in-depth guidance on implementing effective load balancing strategies.

Q & A

What is a network load balancer?

A network load balancer is a hardware device or software-defined device that intercepts incoming traffic from the Internet and distributes it across multiple servers to ensure no single server becomes overwhelmed, thus improving the distribution of network or application traffic and providing high availability.

Why is it important to scale application servers?

Scaling application servers is important to handle increasing loads, such as serving millions of users simultaneously. It allows the infrastructure to grow horizontally by adding more servers rather than just scaling vertically by making existing servers more powerful, which has physical and cost limitations.

How does a load balancer help in managing traffic for a website with high user traffic?

A load balancer helps by distributing incoming traffic across multiple application servers, ensuring no single server is overwhelmed. It also dynamically scales the application servers up or down based on the current load, which helps in maintaining performance and reducing costs.

What is the Round Robin algorithm used by load balancers?

Round Robin is a simple load balancing algorithm that distributes traffic sequentially across servers. It sends the first request to the first server, the second request to the second server, and so on, cycling back to the first server after the last one.

What are the limitations of using Round Robin for load balancing?

Round Robin may not be suitable for scenarios with long user sessions because it can lead to an imbalance in server loads. It does not consider the current load or capacity of the servers, which can result in some servers being overutilized while others are underutilized.

What is smart load balancing and how does it differ from Round Robin?

Smart load balancing, also known as load-aware load balancing, involves the load balancer working in cooperation with the application servers. It takes into account the current load on each server and makes decisions based on this information, unlike Round Robin, which simply cycles through servers without considering their current load.

Why might a company choose not to use smart load balancing?

A company might choose not to use smart load balancing due to its higher complexity in setup and configuration, as well as the potentially higher cost of the required load balancer software or hardware.

What is the random selection algorithm used in load balancing?

The random selection algorithm in load balancing involves the load balancer using a randomizing function to decide which server should handle each incoming connection, providing a balance that is less predictable than Round Robin but simpler than smart load balancing.

How does a load balancer contribute to cloud-native architectures?

In cloud-native architectures, load balancers are key components that assign traffic to different application servers, ensuring scalability and high availability. They also facilitate communication with a common database tier, preventing issues like split-brain scenarios.

What are some other algorithms that can be used for load balancing besides Round Robin and random selection?

There are various other algorithms for load balancing, including Least Connections, which sends traffic to the least busy server, and IP Hash, which directs traffic based on the source IP address to ensure session persistence.

How can IBM assist in designing a load balancing solution?

IBM can assist by providing expertise and resources to help design and architect a load balancing solution tailored to specific needs. This can include consultation on choosing the right type of load balancer, setting up smart load balancing, and integrating with existing infrastructure.

Outlines

00:00

🌐 Introduction to Network Load Balancers

Bradley Knapp from IBM Cloud introduces the concept of a network load balancer by setting a scenario of running a highly popular website that needs to scale for millions of users. He explains the potential bottleneck of a single application server and how scaling out horizontally with multiple servers can help. The load balancer's role is to distribute incoming traffic efficiently among these servers. It can also dynamically scale servers up or down based on the load, thus optimizing performance and cost. The paragraph concludes with an introduction to different load balancing scenarios and algorithms.

05:00

🔄 Load Balancing Scenarios and Algorithms

This paragraph delves into three different load balancing scenarios. The first is a basic Round Robin method, where traffic is distributed sequentially across servers, which may not be ideal for long user sessions. The second scenario introduces smart load balancing, where the load balancer works in tandem with application servers to distribute traffic based on real-time load data, providing a more balanced approach but at a higher complexity and cost. The third scenario offers a middle ground with various algorithms, such as random selection, that provide more control than Round Robin without the full complexity of smart load balancing. The paragraph concludes with an invitation for further discussion and assistance from IBM Cloud in designing load balancing solutions.

Keywords

💡Network Load Balancer

A network load balancer is a system that distributes network or application traffic across multiple servers. In the context of the video, it is crucial for managing traffic to a website, particularly when the website is immensely popular and needs to scale to serve millions of users simultaneously. The load balancer ensures that no single server becomes overwhelmed by distributing the workload evenly across multiple servers.

💡Scalability

Scalability refers to the ability of a system to handle a growing amount of work by adding resources. In the video, scalability is discussed in the context of a website needing to scale to serve a large number of users. The speaker mentions that as the number of users increases, the application server might need to scale out, which means adding more servers to handle the increased load.

💡Application Server

An application server is a software framework and environment used to host applications and services that users access. In the video, the application server is where the data for the website is hosted and served back to the user's device. The concept is integral to the discussion of load balancing, as it is the servers that the load balancer distributes traffic to.

💡Horizontal Scaling

Horizontal scaling, also known as 'scaling out,' involves adding more servers to a system to increase its capacity. The video script mentions horizontal scaling as a solution when a single application server cannot handle the traffic load, suggesting adding more servers to distribute the load.

💡Auto-Scaling

Auto-scaling is the automatic adjustment of computing resources to meet the demands of an application. In the video, auto-scaling is discussed in relation to the load balancer's ability to add or remove application servers based on the current load, helping to manage costs and maintain performance.

💡Round Robin

Round Robin is a basic load balancing algorithm that distributes traffic sequentially across servers. The video uses Round Robin as an example of a simple load balancing method where each incoming connection is directed to the next server in line, cycling through all available servers.

💡Smart Load Balancing

Smart load balancing refers to a more advanced method of load balancing where the load balancer is aware of the load on each server and makes decisions based on this information. In the video, smart load balancing is contrasted with Round Robin, where the load balancer works in cooperation with application servers to distribute traffic based on current load and server capacity.

💡Load Balancing Algorithms

Load balancing algorithms are the methods used by load balancers to decide how to distribute traffic among servers. The video script discusses different algorithms, including Round Robin and random selection, which are used to determine the distribution of traffic to ensure efficient and fair server usage.

💡User Sessions

User sessions refer to the period during which a user is actively engaged with an application or website. The video mentions that user sessions can vary in length, which can affect how load balancing is managed, as some sessions may last only a few minutes while others may last hours.

💡Cloud-Native Architectures

Cloud-native architectures are designed to take advantage of cloud computing's distributed nature, typically including microservices, containers, and dynamic orchestration. The video highlights the load balancer as a key component in cloud-native architectures, emphasizing its role in efficiently distributing traffic among application servers.

Highlights

Introduction to the concept of a network load balancer by Bradley Knapp with IBM Cloud.

Scenario of running a popular website needing to scale for millions of users.

Explanation of how a single user's traffic is handled by an application server.

Challenge of handling traffic from 10,000 users simultaneously.

Limitations of scaling up application servers and the need for horizontal scaling.

The role of a load balancer in distributing traffic among multiple application servers.

Load balancer's function as a hardware or software-defined device.

How load balancers collect and provide information to dynamically scale application servers.

The importance of load balancers in cloud-native architectures.

Description of the Round Robin algorithm used by load balancers for traffic distribution.

Limitations of Round Robin for long user sessions leading to server imbalance.

Introduction to smart load balancing and its cooperation with application servers.

Complexity and cost considerations of setting up smart load balancing.

Alternative load balancing algorithms beyond Round Robin and smart balancing.

Example of a random select algorithm for load balancing.

Offer to share more specific information on load balancing and assistance from IBM.

Transcripts

Hi there, and thanks so much for

stopping by today.

My name is Bradley Knapp with IBM

Cloud. And the question that I

wanted to go over with you guys

today is what is a network

load balancer?

And so I want you to imagine

a scenario, right?

You have decided that you are going

to run a website and this website

is going to be immensely popular.

You need to build it to scale to

serve millions of users,

all of whom are accessing it at the

same time. So imagine

that single user and what

is his experience going to be?

Right.

So you've get your guy, he's

out here and he's got

a laptop.

Right. And so on his laptop, he

wants to open your awesome

new website. So he's going to send

that traffic out into the

Internet, which we're going to

put a little cloud in there for the

Internet.

And then is it transmits through

the Internet to the actual servers

where the data is hosted, it's

going to get down here, and it's

going to get to an application

server. And that application server

is going to be what actually serves

all of the data back

via the Internet to his laptop

so that he can view what's going on

on your website.

Now, that's fine.

If you've just got one person who

accesses your website at a time,

what if you have ten people at once?

Well, that app server is probably

going to be fine to do that.

What if you have 10000 people

who are accessing it at once?

And so that's when you get into

these interesting scenario

where you can saturate

out this application server.

And so if you think about an

infrastructure level definition of

an app server, you can scale them up

to a certain size, but

you've got limits in how big these

application servers can be.

So what do you do?

All right. Well, you need to scale

them out, right?

Instead of having one app server,

you might have three or you might

have four, or you might have five,

or you might have 10 or you might

have 10000.

You were going to horizontally scale

those application servers

so that you have enough to

satisfy your customer load.

But how do you know how much

scaling you need?

And more importantly, how does this

guy with his laptop get

to the application server that he

needs in order to be successful?

And the answer to that is in between

your

customer and all of your

app servers, you are going

to put a load balancer.

What is a load balancer?

It's a hardware device, or it could

be a software defined device, and

that load balancer is going to

intercept all of the

traffic that's coming in from the

Internet. And it's going to decide

what goes to this server, what goes

to this server, what goes to this

server, and so on, and so

forth.

It's also going to provide and

collect information.

So your application servers down

here, they can be talking to this

load balancer. They can be saying,

hey, I'm only being used

20 percent of the time.

Maybe you don't need me right now.

And that load balancer can then

dynamically scale, or auto

scale, and turn that application

server off if you don't need it -

helps drive down costs.

Likewise, if all of your app servers

are checking in and they're saying,

hey, I'm super busy, I'm running at

85, 90 percent utilization,

it's time to bring in help.

Then that load balancer can hit that

auto scale service again and can

say, hey, look, I need another one.

I need another app server to try and

decrease my load.

And so when folks talk about cloud

native architectures, this is a

key component. The load balancer is

a key component because that load

balancers going to assign out that

traffic to all of the different

application servers, and

then the app servers, they're all

going to go down here and they're

going to talk to a common database

tier because you don't want to get

into a goofy split brain scenario

with your database.

But they're going to get

the data that they need and they're

going to serve that back up to the

end customer directly.

So that's how you split up

the data.

That's how you serve it up.

But so now that we're in load

balancers, we said it's going to

decide which application server

gets the traffic, right.

How does it decide what to

do? And so let's get our three

most common scenarios right.

So we're going to go over here and

we're going to have scenario one,

scenario two,

and then scenario three.

So scenario one:

our load balancer right here,

right, it's got all its app

servers and

we're going to call this one a

relatively dumb load

balancer, right?

We don't need a lot of complexity in

it. It's serving a fairly basic

purpose.

And so it's going to use an

algorithm called Round Robin

in order to assign

the traffic.

And so what does Round-Robin mean?

It means that for the first user

that connects, it's going to send

that person to App Server 1.

For the second person who connects,

it's going to send them to

App Server 2, the third person

who connects is going to go to

App Server 3.

And then when the fourth one comes

in, it's just going to start back

over again.

Round-Robin literally means just

going sequentially through it.

Now, Round-Robin is not a perfect

scenario.

If you have user sessions that last

a long time, some people log

in and run for five

minutes. Some people log in and run

for two hours.

Your app servers can get out of

balance with one another when

you're using that round robin

scenario.

And so then you need to think

about things like smart load

balancing. Smart load balancing is

going to be our second piece here.

Right. And this is an application

server load-aware

load balancer.

So again, just like before, we've

got our load balancer here and

we've got our app servers.

But this

scenario, this smart load balancing

scenario, rather than the

load balancer just acting as

king and directing what all happens,

the load balancer works in

cooperation with the application

servers. So the app servers are

constantly transmitting data

back and forth to that load

balancer, letting them know how

busy it is and

that load balancer's then going to

make the decisions to send the

incoming connections.

So your guy over here, if

he comes in and he starts

a user session, that load balancer

is going to say, look, server three

has the least load on it right now.

I'm going to sign him over here.

And it's constantly monitoring that.

Now, obviously, you're going to look

at that and you're going to say, well,

why wouldn't you always use this

kind of setup?

Well, it is more complex

to get set up initially, right?

It takes more configuration.

Also, the load balancer software

or the hardware device that you use

to set this kind of a scenario up,

it is more expensive.

And so while technologically

it's probably always the best choice

for your specific scenario,

it might not be.

And then the third scenario

that we've got down here.

Right. So we've got our load

balancer and all of our app servers.

This is a scenario where you want

a little bit more control

than you do in just a straight round

robin, but you don't want to go all

the way through the set up and

config and everything like that that

you need with a smart load balancing

scenario.

And so there are actually like

nine different algorithms that you

can pick from that split the

difference between these two.

The one that I'm going to use

right now as an example is a

random select, right.

So the load balancer, rather

than sequentially working through an

application server.

One, two, three, one, two, three,

one, two, three, over and over

again. It's just going to run a

randomizing function and it's

going to decide that may be the

first to go to one.

And the third connection goes to two

and connections four through six are

going to go to three and so on and

so forth.

Again, it's just it's different

kinds of algorithms to decide

where you need to go, where you need

to send that traffic.

And if you want to get into

specifics, there's tons of

information out there.

I'd be happy to share that with you.

Let us know in the comments.

If you want to know specific advice,

please feel free to reach out to

anybody in the channel or anybody at

IBM. We'll be glad to help you

design an architect, a load

balancing solution that is going to

work to help you solve

the needs of your customer.