What is WebSocket? Why is it used & how is it different from HTTP?

Harkirat Singh
22 Nov 202333:22

Summary

TLDRThis engaging tutorial delves into the world of distributed computing and real-time communication systems, with a focus on understanding WebSockets and Redis. Designed for developers seeking to enhance their skills, the video covers the need for persistent connections between clients and servers, enabling real-time updates crucial for applications like chat systems and multiplayer games. Through practical examples and code walkthroughs, viewers will learn how to create WebSocket servers, establish bidirectional communication, and leverage Redis for efficient data distribution across multiple servers. With its clear explanations and hands-on approach, this tutorial empowers developers to build scalable, responsive applications that deliver seamless real-time experiences.

Takeaways

  • 👨‍💻 Distributed computing is used by large companies like Swiggy and Zomato to handle high traffic and provide real-time updates.
  • 🔌 WebSockets enable full-duplex communication between the client and server, allowing real-time data transfer.
  • ⚡ Realtime systems like chat apps, multiplayer games, and trading platforms require WebSockets for seamless updates.
  • 📡 WebSockets establish a persistent connection between the client and server, unlike HTTP's request-response model.
  • 🌐 While WebSocket is a protocol, libraries like Socket.IO provide additional abstractions and features on top of it.
  • 📚 Implementing WebSockets in Node.js involves creating a WebSocket server using the 'ws' library and handling connection and message events.
  • 💻 Clients like web browsers, mobile apps, and WebSocket testing tools can connect to and communicate with the WebSocket server.
  • 🔀 In distributed systems, multiple WebSocket servers may need to communicate with each other using technologies like Redis to enable real-time data transfer.
  • 🧱 Concepts like rooms and message types are crucial for efficient message routing and handling in real-time applications.
  • 🚀 Understanding WebSockets and distributed computing principles is essential for building scalable, real-time systems like chat applications and multiplayer games.

Q & A

  • What is the purpose of using WebSockets?

    -WebSockets are used for building real-time systems where the server needs to push events or updates to the client in real-time. They provide a full-duplex communication channel over a single, long-lived connection, allowing both the client and server to send data back and forth.

  • How is a WebSocket connection different from an HTTP connection?

    -Unlike HTTP, which follows a request-response model and closes the connection after each request, a WebSocket connection is persistent and remains open until explicitly closed. This allows for real-time, bidirectional communication between the client and server without the need for constant polling or refreshing.

  • What is the significance of the concept of 'rooms' in WebSocket applications?

    -The concept of 'rooms' is used in WebSocket applications to manage and organize connections based on specific criteria or groups. For example, in a chat application, users in the same room can receive messages from each other in real-time, while users in different rooms are isolated from those messages.

  • Why is the choice between WebSockets and Socket.IO important?

    -While Socket.IO is a popular library that abstracts some of the complexities of WebSockets, it introduces its own protocol and requires both the client and server to use the Socket.IO library. Raw WebSockets, on the other hand, are a standard protocol with widespread client support across various platforms and languages, making them a more robust and interoperable choice for production environments.

  • What is the role of Redis in a distributed chat system?

    -Redis plays a crucial role in a distributed chat system by enabling communication and data sharing between multiple WebSocket servers. It provides features like pub/sub messaging, queues, and in-memory data storage, allowing WebSocket servers to broadcast messages to other servers and clients efficiently.

  • How does a WebSocket server handle multiple client connections?

    -When a client establishes a WebSocket connection with the server, the server creates a separate WebSocket instance for that client. The server can then send and receive messages to/from each client individually, or broadcast messages to multiple clients as needed.

  • What is the difference between WebSockets and WebRTC?

    -WebSockets and WebRTC are both used for real-time communication, but they differ in their underlying transport protocols. WebSockets use TCP, which guarantees reliable message delivery, while WebRTC uses UDP, which is faster but may result in some message loss. The choice between the two depends on the specific requirements of the application.

  • How can you test a WebSocket server before integrating it with a client application?

    -You can test a WebSocket server using tools like Postman or Postwoman, which provide a graphical interface for establishing WebSocket connections and sending/receiving messages. This allows you to verify the server's functionality before integrating it with a client application.

  • What are some typical use cases for WebSockets beyond chat applications?

    -WebSockets can be used in a variety of real-time applications, such as online gaming, collaborative editing tools, stock trading platforms, real-time analytics dashboards, and IoT (Internet of Things) applications.

  • How does WebSocket communication differ from long polling or Server-Sent Events (SSE)?

    -While long polling and SSE are techniques that can be used to achieve real-time communication, they still rely on HTTP requests and responses, which can be inefficient and resource-intensive. WebSockets, on the other hand, provide a dedicated, persistent connection that eliminates the need for constant polling or request overhead.

Outlines

00:00

👨‍💻 Introduction to Advanced Concepts in Distributed Computing

The speaker introduces the topic of distributed computing, explaining that they will be learning advanced concepts at a slow pace. The focus is on understanding how Backpack uses distributed computing for part of their codebase, as many big companies employ distributed computing in some form. The speaker highlights that they will learn about system design for a distributed chat system, as Backpack uses chat, and many companies require real-time communication. They will explore concepts like websockets, how backend systems communicate, Redis, and pub/sub.

05:01

🌐 Importance of Websockets for Real-time Systems

The speaker emphasizes the need for websockets in real-time systems, where the server needs to push events to the client. HTTP requests alone are insufficient for such systems, as the server cannot initiate communication. Examples like real-time todo applications, Google Docs, and chat applications are given, where events need to be pushed from the server to reflect changes across clients. Techniques like long polling and server-sent events are discussed as alternatives, but websockets are presented as the optimal solution for real-time, full-duplex communication between client and server.

10:02

🎮 Use Cases for Websockets and WebRTC

The speaker delves into use cases for websockets, such as real-time trading platforms, chat applications, and multiplayer games. The differences between websockets and WebRTC (Web Real-Time Communication) are discussed, with WebRTC being more suitable for scenarios where some data loss is acceptable, such as games. The speaker also touches on the concept of rooms, where clients are grouped, and events are broadcasted within those rooms. The importance of scalability and distributing load across multiple websocket servers is highlighted.

15:02

🔧 Setting up the Development Environment

The speaker provides instructions for cloning the project repository and opening it in Visual Studio Code. They explain the project structure, including the backend and frontend code for the websocket server, as well as TypeScript configuration files. The speaker also mentions an assignment from the previous week that they may cover if time permits.

20:04

📡 Creating a Simple Websocket Server in Node.js

The speaker walks through the process of creating a simple websocket server using Node.js and the Express.js framework. They explain the need for a native HTTP server to create a websocket server and demonstrate how to handle client connections and messages. The code snippets illustrate logging connections, receiving messages from clients, and sending responses back to the clients.

25:06

🚀 Running the Websocket Server

The speaker provides instructions for running the websocket server locally. They explain the steps involved, including compiling the TypeScript code using the TypeScript compiler (tsc) and running the compiled JavaScript file (node index.js). The speaker also mentions the ability to expose other HTTP endpoints alongside the websocket endpoint using Express.js.

30:08

🕸️ Testing the Websocket Server with a Client

The speaker introduces a tool called Postwoman (previously known as postwoman) that allows testing websocket connections, similar to how Postman is used for testing HTTP requests. They demonstrate how to establish a websocket connection to the local server, send messages, and receive responses. The speaker also mentions the need for writing additional logic in a real-world scenario, such as handling different message types and implementing specific functionalities.

Mindmap

Keywords

💡Distributed Computing

Distributed computing refers to the concept of splitting computational tasks across multiple interconnected computers or systems. In the context of the video, distributed computing is mentioned as a key aspect of how large companies like Backpack handle their codebase. The speaker explains that instead of running on a single server or database instance, their systems are distributed across multiple nodes that communicate with each other, enabling scalability and resilience. This is a crucial concept for building real-time applications like chat systems or multiplayer games.

💡Real-time Communication

Real-time communication refers to the ability to transmit and receive data or messages with minimal delay, allowing for near-instantaneous interactions. The video emphasizes the importance of real-time communication in various applications such as chat systems, multiplayer games, and collaborative tools like Google Docs. In these scenarios, users expect updates or events to be reflected immediately across all connected clients, which traditional client-server models cannot efficiently handle. Real-time communication is a driving factor behind the need for technologies like WebSockets.

💡WebSockets

WebSockets is a computer communications protocol that provides a full-duplex communication channel over a single TCP connection. Unlike traditional HTTP requests, which follow a request-response cycle, WebSockets establish a persistent connection between the client and server, allowing for bidirectional, real-time data transfer. This makes WebSockets well-suited for applications that require continuous updates or real-time events, such as chat systems or multiplayer games. The video introduces WebSockets as a solution for building real-time systems and discusses its advantages over techniques like long polling or server-sent events.

💡Redis

Redis is an open-source, in-memory data structure store that can be used as a database, cache, and message broker. In the context of the video, Redis is mentioned as a crucial component for enabling distributed computing in real-time applications like chat systems. Redis provides features like pub/sub (publish/subscribe), which allows different servers or nodes to communicate and share data efficiently. The video suggests that Redis will be explored further to understand how it facilitates communication between different components of a distributed chat system.

💡Client-Server Model

The client-server model is a distributed computing architecture where a client (typically a user's device or application) initiates requests to a server, which processes the requests and sends back responses. The video contrasts this traditional model with real-time systems, where the server needs to push updates or events to clients without waiting for client requests. While the client-server model works well for simple applications, it falls short for real-time scenarios like chat or multiplayer games, where continuous, bidirectional communication is required.

💡Full-duplex Communication

Full-duplex communication refers to the ability to transmit and receive data simultaneously over a single communication channel. In the context of the video, full-duplex communication is highlighted as a key feature of WebSockets, where both the client and server can send and receive data concurrently over a single persistent connection. This bidirectional communication is essential for real-time applications, as it allows the server to push updates to clients without waiting for requests, and clients to send data to the server asynchronously.

💡Persistent Connection

A persistent connection, also known as a long-lived connection, is a communication channel that remains open and active for an extended period of time, unlike traditional request-response connections that are short-lived. The video emphasizes the importance of persistent connections in real-time systems, as they allow the server to continuously push updates or events to clients without the need for constant reconnections or polling. WebSockets establish persistent connections between clients and servers, enabling efficient real-time communication.

💡Long Polling

Long polling is a technique used to emulate real-time communication in traditional client-server models. In long polling, the client sends an HTTP request to the server and keeps the connection open, waiting for the server to respond with new data or updates. If the server doesn't have any updates, the request remains open until new data becomes available or a timeout occurs. The video mentions long polling as an alternative approach to real-time communication but suggests that it is an inefficient and suboptimal solution compared to WebSockets.

💡Server-Sent Events (SSE)

Server-Sent Events (SSE) is a standard that allows servers to push data to clients over a persistent HTTP connection. Unlike WebSockets, which provide full-duplex communication, SSE is a one-way communication channel where the server can send events or updates to the client, but the client cannot initiate communication directly. The video briefly mentions SSE as another approach for real-time communication before WebSockets but suggests that WebSockets are a more optimal solution.

💡Rooms

In the context of real-time communication systems, rooms refer to logical groupings or channels where clients can be segmented based on specific criteria, such as a chat room or a game lobby. The video mentions the concept of rooms in relation to the Socket.IO library, which provides an abstraction for creating and managing rooms, allowing messages or events to be broadcast to specific groups of connected clients. This concept is essential for scalability and organization in real-time applications, where different clients may need to receive different subsets of data or updates.

Highlights

Introduction to distributed computing and its significance in large-scale web applications.

Explanation of distributed computing using the example of Swiggy's website architecture.

Introduction to the system design of a distributed chat system as a practical application of distributed computing.

Overview of real-time communication systems and their importance in modern web applications.

Introduction to websockets as a solution for real-time communication challenges.

Detailed explanation of the difference between HTTP and WebSocket protocols.

Introduction to Redis and Pub/Sub model for enhancing backend communication.

WebSocket as a persistent connection protocol allowing real-time bidirectional communication.

Use case of WebSocket in financial trading platforms for real-time data updates.

Explanation of WebSocket connections and messages through practical demonstration.

Discussion on scalability and distributed architecture of WebSocket servers.

Comparison between WebSocket and other real-time communication technologies.

Introduction to building a simple WebSocket server and client for a chat application.

Overview of the npm libraries available for creating WebSocket servers in Node.js.

Demonstration of establishing a WebSocket connection using a client application.

Transcripts

play00:00

[Music]

play00:05

So today we're going to learn some

play00:07

Advanced uh slightly advanced concepts

play00:09

but again we're going to take things

play00:11

slow so don't worry too much about it

play00:12

we're going to learn a little bit about

play00:13

distributed computing the reason for

play00:16

this is K backpack does use this for a

play00:18

specific part of their code base well

play00:19

most big companies use distributed C

play00:22

Computing in like some form or fashion

play00:24

uh if you go to a swiigy website uh

play00:27

there isn't like one server that's

play00:28

running on one backet that's running

play00:30

there isn't a single database instance

play00:32

that's running there are a bunch of them

play00:33

they all talk to each other through some

play00:35

form or fashion that is what the high

play00:36

level goal of this week is going to be

play00:38

system designed of a distributed chat

play00:40

system why chat system because backpack

play00:42

uses chat

play00:43

and as you go deep into an organization

play00:47

most company I would say most companies

play00:48

but a lot of companies would have some

play00:51

part of realtime communication that is

play00:52

needed what is realtime communication we

play00:54

will get to but the systems we've built

play00:56

until now follow the client server model

play00:58

very heavily if you might remember huh

play01:01

in our case we have a HTTP server the

play01:03

client hits the HTTP server gets back a

play01:04

response this is different from how

play01:07

realtime systems work how we will see

play01:09

today but that's what we're going to

play01:11

spend our time learning more practically

play01:12

we'll learn about websockets we'll learn

play01:14

about uh how backend systems can talk to

play01:16

each other we'll learn about redis and

play01:18

something called pop sub if we get the

play01:20

time we'll dive a little deeper into

play01:21

redis and understand the other things

play01:23

that reddis provides you red provides

play01:24

you like a mini database what's called

play01:26

like an inmemory database it gives you

play01:28

uh cues it gives you a pub sub it gives

play01:31

you 10 different things so we'll try to

play01:32

cover as many things as we can but more

play01:34

practically we want to be able to look

play01:36

at the chat system of backpack and

play01:38

understand the chat back end which is

play01:40

significantly different from the back

play01:41

end we've seen until now that's the high

play01:43

level of this week system design of a

play01:45

distributed chat system understanding

play01:46

what a websockets understanding redis

play01:49

and contributing to the chat part of an

play01:50

open source codebase which is backpack

play01:53

this is the repository where we have all

play01:54

the checkpoints for today we'll take

play01:56

things slow and slowly uh you know make

play01:58

our way up just just to do a quick check

play02:01

before we start how many of you already

play02:03

know about websockets distributed

play02:05

computing redis do you already know

play02:08

this yes or no and then we will

play02:16

start okay so majorly know which is

play02:20

great so yeah we'll take things slow and

play02:22

let's try to tackle these slightly

play02:24

Advanced backing Concepts this week cool

play02:28

why websockets very important question

play02:31

why do you learn websockets and why were

play02:33

they introduced in the first place to

play02:36

websockets as the name suggests is like

play02:38

it is a socket which basically means it

play02:40

is a connection between a client and a

play02:43

server the way we have done client

play02:45

server connections until now is I

play02:46

apologize for the white screen my bad um

play02:49

is that you have your

play02:51

browser and you have an HTTP server

play02:53

running somewhere these HTTP servers we

play02:56

have deployed on Local Host as well we

play02:58

have deployed them on AWS in various

play03:01

regions as well we have seen okay you

play03:03

can hit this HTTP server through an HTTP

play03:07

request which could be either a get

play03:08

request or a post request so on and so

play03:10

forth can also send some body in here

play03:13

this is one way to talk to a backend

play03:15

server there's one problem here when the

play03:17

problem is okay the server can never

play03:19

push you events directly you always have

play03:21

to ask the server for something when

play03:24

this this does become a problem when is

play03:26

this not a problem this is not a problem

play03:27

for simple websites we have built until

play03:28

now okay H by we have a Todo application

play03:31

and we just hit the back end get back

play03:33

our

play03:33

toos but what if you want to build a

play03:36

realtime too application which means

play03:39

what if our Todo application has a

play03:42

website it also has a mobile

play03:46

app and the person adds a to-do in the

play03:49

mobile app the website should also see

play03:52

that reflected immediately right this

play03:53

happens on notion this happens on Google

play03:55

Docs if you go to Google Docs if you

play03:56

type in one tab you will see your

play03:57

changes in another tab if someone types

play03:59

from from the other side of the world

play04:01

you see those changes being typed in

play04:03

real

play04:05

time so to be able to communicate or

play04:09

like be able to have events being

play04:11

triggered in real time from the

play04:15

server you need a better way to

play04:17

communicate with the server more

play04:18

specifically you need a persistent

play04:21

connection with the server so that the

play04:22

server can keep sending you events okay

play04:24

oh someone added a new too or someone

play04:26

added something at a specific line in

play04:28

Google Docs if the can push you down

play04:30

these events you don't have to worry

play04:32

about sending HTTP requests to get back

play04:35

the response there is a persistent

play04:37

connection that persistent connection

play04:39

always exists and the server can push

play04:40

you things why is this useful in the

play04:43

chat system if you are in a chat system

play04:47

on on whatever is it called messenger SL

play04:49

faceb you type a message you click on

play04:52

send what happens is either an HTTP

play04:56

request or something else communicates

play04:58

this to an HTTP server okay H by someone

play05:01

sent a message let's say hi there let's

play05:04

say your friend has in another part of

play05:06

the world has the same chat window open

play05:09

he should receive this specific message

play05:11

how will he receive it you might say

play05:12

hirat we can just pull every 1 second

play05:15

send an HTP request give me the top 50

play05:17

messages get back the response so after

play05:19

1 second we will ask for the new list

play05:21

and the new message will come here I

play05:23

would say that is true this is what is

play05:24

called long polling basically means you

play05:28

send an empty HTTP request k bro any

play05:31

changes that you have anytime you get a

play05:32

change you push it back to me so you can

play05:35

use long polling for this you can also

play05:37

use another standard that was introduced

play05:39

called server side events which pretty

play05:41

much as the name suggests let's you do

play05:43

what we want which is let the server

play05:46

side events lets the server push events

play05:49

onto the client though this these might

play05:52

feel like Optimal approaches I I'm sure

play05:54

you understand why they're not okay by

play05:56

if I'm pulling every 1 second I'm

play05:57

getting back 50 messages every 1 second

play05:59

second and what if there was no change

play06:01

made if there was no change made we

play06:02

don't really want um to keep asking the

play06:05

server every 1 second H give me all the

play06:07

same messages again and again there is a

play06:09

problem here for this websocket as a

play06:11

protocol was introduced how is websocket

play06:14

different from sttp if this is your HTTP

play06:18

server you send it a request it sends

play06:21

back a response it is a request response

play06:23

model and if you actually go through

play06:25

computer networks a little bit and

play06:27

understand how this protocol Works under

play06:28

the hood you will see see there is a

play06:30

handshake that happens if you might have

play06:31

read this in computer networks if you go

play06:33

one level Below in the OSI model how

play06:35

does http work under the H it works on

play06:37

the transport layer it's create creates

play06:39

a TCP connection and after the request

play06:42

is complete it closes the TCP connection

play06:45

there is no persistent connection

play06:46

between you and the server compared to

play06:49

if you have a websocket server the whole

play06:51

goal is if this is your browser and this

play06:54

is your websocket server there is a

play06:55

persistent connection between the two

play06:57

which means the browser at any time can

play06:59

send you request the server can also

play07:02

send you something not a request but

play07:04

like some data whatever the server wants

play07:05

to push down that is the difference

play07:07

between a websocket server and an HTTP

play07:09

server they are both different protocols

play07:11

SL standards there's some sort of

play07:13

interview question here somewhere is

play07:15

websocket HTTP and the answer to this is

play07:17

okay it is not HTTP but the first

play07:20

request that you send out when you want

play07:21

to create a websocket connection if this

play07:24

is your browser if you want to create a

play07:26

websocket connection from your browser

play07:28

to any open server

play07:30

the first request that goes out from

play07:31

your browser is actually an HTTP request

play07:33

only and under the hood then it gets

play07:36

what's called

play07:38

upgraded to a websocket connection and

play07:41

then you have a persistent connection to

play07:42

the back end but this is like a lot of

play07:44

theoretical jargon so let's see this in

play07:46

action and understand what are the use

play07:48

cases and how and where websites are

play07:50

creating these websocket connections

play07:52

HTTP is great but does not cover a few

play07:54

use cases the most important use case it

play07:56

doesn't cover is the server sending

play07:58

events or server side events what if you

play08:00

want to build a realtime system what is

play08:03

a realtime system the binance usdt if

play08:06

you go to Zera if you go to binance if I

play08:08

go to

play08:10

this specific page this is like the

play08:12

current price of uh binance the current

play08:15

chart which sort of keeps updating as

play08:17

you can see this number is updating very

play08:19

quickly this list is updating very

play08:21

quickly this number which is the current

play08:23

price is updating very quickly what do

play08:24

you think is happening do you think

play08:25

binance every second is sending out HTTP

play08:27

requests the answer is no it has has

play08:29

created a single websocket connection to

play08:31

a binance server and the binance server

play08:33

is pushing down events yellow this thing

play08:35

has changed yellow the price has changed

play08:37

yellow the chart has changed so all

play08:39

those events binance is pushing down to

play08:41

the client how can we confirm how did we

play08:43

confirm that an HTTP request is made we

play08:45

used to refresh this and we used to seea

play08:47

an HTTP request goes out the response

play08:50

comes back similarly you can also track

play08:53

um websocket events between a client and

play08:56

a server oh sorry the websocket

play08:58

connection so so you will usually see

play09:00

this one okay if you the webside

play09:03

connection is made between the browser

play09:04

and WSS which stands for websocket

play09:07

secure col binance do sorry stream.

play09:11

binance.com Stream So This is the end

play09:14

point where they've hosted their

play09:15

websocket server and when I send the

play09:17

request I don't just get a single

play09:19

response if I click on messages you will

play09:21

see I am getting a lot of messages in

play09:23

real time H by the depth has changed

play09:26

what is the depth this thing right here

play09:28

yeah you know the price has changed the

play09:30

chart has changed all these are coming

play09:33

back as messages from the binance server

play09:36

to me which is how this page is being

play09:38

updated there's a single web socket

play09:40

connection there aren't a thousand HTTP

play09:42

requests that are going out every second

play09:44

to update this page this is very common

play09:47

in realtime systems what is a realtime

play09:49

system where after you've opened a

play09:51

website you're getting a bunch of

play09:52

changes from the back end a chat

play09:54

application is a basic example of it if

play09:56

you have seen multiplayer games whatever

play09:58

multiplayer Flappy word or if you play

play10:00

Minecraft with your friend or even if

play10:01

that famous fortnite game that people

play10:03

play all of them require the server to

play10:06

give you back the location of other

play10:08

opponents who hit a gun you know you get

play10:11

that gun going right next to your ear or

play10:13

whatever so all of these things come

play10:15

back as events from the server it is

play10:17

never you keep asking the server bro

play10:19

what is the current state what is the

play10:20

current state they're all pushed down by

play10:22

the backend server how sometimes through

play10:24

websockets there are other ways to do it

play10:25

web RTC is also like a decent use case

play10:28

here okay you can actually create a web

play10:29

TC connection and web TC also lets you

play10:31

do data there's a significant difference

play10:33

like there slight differences between

play10:35

websockets and uh you know web RTC is

play10:37

like an interview question when should

play10:39

you use web sockets when should you use

play10:41

web RTC under the hood they are both

play10:44

trying to achieve the same thing if from

play10:46

your browser you create a connection to

play10:48

a websocket server it can send you back

play10:51

let's say we considering fortnite here

play10:54

so it will send you back the an array of

play10:57

let's say oh this person moved here this

play10:59

this person moved here this person

play11:00

pressed a gun

play11:01

Flaka you can do the same thing with uh

play11:05

a web RTC server but like the high level

play11:08

difference the biggest difference is

play11:09

webc uses UDP and websocket uses TCP

play11:13

which means here you will definitely get

play11:14

back all the events here some events

play11:16

might get missed so depending on the use

play11:18

case for example in games a lot of times

play11:20

you don't worry you need you don't need

play11:22

every movement update of the end user

play11:24

you need some movement updates even if

play11:26

one movement update gets missed it's

play11:27

fine so whenever you're getting back at

play11:29

like 30

play11:30

FPS and you don't really care if one

play11:33

gets missed it's fine you use web RTC if

play11:36

it is something like chat huh by if a

play11:38

chat event gets missed that really

play11:40

that's really bad I I should see every

play11:42

chat message that everyone else is

play11:43

sending me then you use websockets or

play11:45

tcps this is a basically an interview

play11:47

question that me or may not get asked

play11:49

when should you use websockets versus

play11:52

web RTC this is the answer you need all

play11:54

the data definitely web sockets if not

play11:57

web

play11:57

RTC HTTP is not enough for something

play12:01

like this or even something like this

play12:04

what is this this is chat in WhatsApp

play12:06

chat in telegram or even chat in

play12:08

backpack backpack has a chatting section

play12:10

if some of you might have opened it it

play12:12

may or may not work for you because

play12:14

you're not we're not running all the

play12:15

services locally at the moment but if

play12:16

you go into backpack the fourth tab here

play12:18

is like a

play12:19

chat this chat requires it to be

play12:21

scalable real time and the goal through

play12:25

today and tomorrow is going to be how

play12:26

this chat is built and how you can

play12:27

understand the whole thing and start

play12:28

make changes inside it and the way this

play12:31

was built was like super scalable so

play12:33

we'll try to understand how it is super

play12:35

scalable and what are the systems we

play12:36

need to understand to able to make like

play12:38

a production worthy realtime system not

play12:41

necessarily chat you can extend this to

play12:42

other use cases as well HTTP is not

play12:45

enough when you need to uh need

play12:47

real-time updates from the back end you

play12:49

could either use server side events just

play12:50

that there's a technology that was

play12:52

introduced before websockets which tried

play12:54

to solve this problem but like didn't

play12:55

really catch enough heat then there is

play12:58

long pulling can you still use HTTP but

play13:01

every 1 second you send a request k bro

play13:03

anything you have and if the server

play13:04

doesn't have anything the request sort

play13:05

of stays hung if the server does get

play13:07

anything it returns back some data with

play13:08

it that's called long polling it's like

play13:10

a very ugly way to do this and then

play13:12

finally websockets which is like

play13:13

persistent connection between client and

play13:15

server unlike HTTP who's like a

play13:18

transient connection send request get

play13:20

back response connection closes that's

play13:21

HTTP what is web socket persistent

play13:23

connection always open until you

play13:25

explicitly close it an though all the

play13:27

data keeps rolling in that's what

play13:29

websockets let you do and what is the

play13:30

ideal solution here to build a real time

play13:32

system it is

play13:34

websockets question is what are

play13:36

websockets websocket is like the

play13:38

official um definition websocket is a

play13:41

communication protocol that provides a

play13:43

full duplex communic that provides full

play13:45

duplex communication channels over a

play13:46

single long lived connection hopefully

play13:48

that is

play13:50

self-explanatory what does full duplex

play13:52

mean it means client can send events

play13:54

server can send events they can do their

play13:55

like send whatever they want over a

play13:57

single long lived connection so you

play13:59

don't have to create multiple

play14:00

connections you don't have they're not

play14:02

shortlived like an HTTP request they're

play14:03

completely open until you close the tab

play14:05

or you know you you your Wi-Fi shuts

play14:08

down or you explicitly have to close it

play14:11

you move away from the trading page

play14:12

let's say then you don't need the

play14:13

websocket connection anymore but while

play14:15

you're on that trading page while you're

play14:16

on that chat system you do need a

play14:18

persistent connection so the server can

play14:20

keep telling you how data is sort of

play14:22

being transmitted the good question to

play14:23

ask at this point is I understand this

play14:26

part okay this is the web connection how

play14:29

what happens on the server how does the

play14:30

server know huh there is an event I need

play14:32

to push to herat or you know uh that a

play14:35

game movement has happened how do you is

play14:37

there a single server single websocket

play14:39

server and then okay let's say there is

play14:41

a fortnite like game so is there if this

play14:43

is person one's browser this is person

play14:44

2's browser is there a single websocket

play14:47

server where this guy has connection

play14:48

this guy has a connection this pushes a

play14:50

data here and this pushes it down to the

play14:52

other person and vice versa is this how

play14:54

chat is happening are there multiple

play14:56

backend servers if there are multiple

play14:58

backend servers do they need to

play14:59

communicate with each other what if

play15:02

there is a fortnite game that three

play15:04

people are playing P1 P2 P3 first person

play15:07

sends his data here and has a persistent

play15:09

connection to this one second person has

play15:10

a persistent connection to the second

play15:11

websocket server third person let's say

play15:13

has persistent con to the third

play15:15

websocket server if I send my data here

play15:18

how will it reach P3 does websocket

play15:20

server one need to send it to websocket

play15:22

server 3 which will then forward it to

play15:24

P3 or is there another architecture that

play15:26

is used here to make this a distrib

play15:28

system it is much easier to do if you

play15:30

just assume H there is a single

play15:33

websocket server all people are

play15:35

connected to the same websocket server

play15:37

to data goes here this guy does a for

play15:39

Loop it already has the connections to

play15:41

everyone else sends data to all five

play15:43

this is easy what is hard this single

play15:46

webset server can't really support more

play15:47

than you know X number of users so how

play15:49

do you distribute it how do you create a

play15:51

SW of websocket servers that different

play15:53

users can connect to different websocket

play15:55

servers yet they can receive events

play15:57

that's where redis comes in into the

play15:58

picture and that's the second part of

play16:01

what we'll discuss to most probably this

play16:02

the red disc part we will discuss

play16:03

tomorrow today we'll just try to

play16:06

understand how you can create a

play16:06

websocket server create a simple chat

play16:08

application using it what is the concept

play16:10

of rooms that sort of you need to be

play16:13

able to understand who you have to

play16:14

forward this message to and we'll also

play16:17

if we get time uh try to do an

play16:19

assignment from last week that I gave

play16:21

but I don't think a lot of you have done

play16:22

it cool web sockets and node is

play16:25

websockets a a noj specific concept no

play16:28

it's a protocol HTTP servers can be

play16:30

written in goang can be written in Rust

play16:32

websocket servers can be written in

play16:34

goang goang nodejs Rust whatever

play16:36

language you want we are going to write

play16:37

them in note

play16:39

J there are a bunch of HTTP servers out

play16:42

there Express isn't the only one there

play16:44

is one called Kaa slca there's another

play16:46

one that weate used when we were

play16:48

creating the open API spec file I think

play16:51

that was using Express under the hood

play16:52

but you get the idea there are bunch of

play16:54

HTTP servers in nodejs similarly there

play16:56

are a bunch of npm libraries that let

play16:58

you create websocket servers it really

play17:00

depends on which one you want to use and

play17:02

a popular one some of you might have

play17:04

heard of might be

play17:05

socket.io now socket. iio is not exactly

play17:07

websockets it is some abstraction on top

play17:10

of websockets it is mostly good in most

play17:13

cases it gives you a lot of things out

play17:15

of the box spefically like this concept

play17:18

of room creation y if you know you have

play17:22

multiple people P1 who's in game Lobby

play17:26

L1 P2 who in game Lobby L2 and P3 who's

play17:30

in game Lobby L1 as well if you use

play17:34

socket.io this specific um npm Library

play17:37

it lets you create rooms very easily

play17:39

what is a room this one and this one are

play17:41

in room L1 this person is in room L2

play17:44

that way anytime an event comes to room

play17:47

L1 this guy can in a single sort of line

play17:50

of code forward it to everyone who's in

play17:52

L1 which is like this guy and this guy

play17:54

and then there's another person who's L1

play17:56

send it to this guy so soet I abstracts

play17:59

a lot of this for you it is a good

play18:00

library to usually start with but it

play18:02

isn't used in production for multiple

play18:05

reasons because if you use a socket.io

play18:08

web backend server your client also

play18:11

needs to talk socket.io and socket.io is

play18:13

not the exactly the websocket protocol

play18:16

it's very close you can actually somehow

play18:17

tweak the server so that you can create

play18:20

a a normal websocket connection to a

play18:22

socket.io server but your

play18:25

Android your iOS these

play18:29

or let's say you have to write like a

play18:30

goang application that connects to your

play18:32

websocket server they don't really

play18:34

understand socket.io they only

play18:35

understand websocket they only have

play18:36

clients that are in websocket socket.

play18:38

dio became very popular so someone wrote

play18:40

a C++ socket.io client you know and

play18:42

Android socket. client but usually it's

play18:44

not widely used websocket the protocol

play18:47

is very widely used so if given a choice

play18:49

if an interviewer asks would you use

play18:51

socket.io for this raw websockets the

play18:53

answer is raw websockets why because

play18:55

it's a uh protocol that's you know Pro

play18:58

proven and and out there and everyone

play19:00

has clients for someone has written

play19:01

websocket client implementation on

play19:03

Androids someone has written websocket

play19:05

client implementation in iOS so on and

play19:06

so forth okay we are going to use one of

play19:08

these libraries to create it today uh

play19:10

are present natively on the front end so

play19:12

just like fetch is natively present on

play19:13

the front end your browser understands

play19:17

Fetch and uses it to send out uh HTTP

play19:21

requests your browser also understands

play19:22

websocket and you know it's a browser

play19:24

API that's available to you to create a

play19:27

connection to the websocket server fetch

play19:30

is used to send a request to an HTTP

play19:32

server this is used to create a this is

play19:35

the client side code for it this is in

play19:36

the server side code server side code is

play19:38

different similar to express client side

play19:40

code or this websocket object is similar

play19:42

to this

play19:43

fetch let's look at the code now um the

play19:45

code for this week as I said is at 100x

play19:48

devs SL

play19:51

uh week

play19:53

19 BP repo oh no sorry my bad week 19 WS

play19:57

so please please try to clone this

play19:59

locally I'll give you guys 2 minutes and

play20:01

try to open it in vs code I have it open

play20:04

on webstorm um I'm going to keep it open

play20:06

on webst if you guys don't mind because

play20:07

Visual Studio code has backpack right

play20:09

now and I want to remove backpack from

play20:11

it so it's very simple code so hopefully

play20:14

the editor does not anow you guys so two

play20:16

more minutes please open it in vs code

play20:19

and I'm going to wait for 2

play20:27

minutes

play20:37

open chat if anyone has any questions

play20:38

but would urge you to start to clone the

play20:40

project

play20:43

locally are there no servers that can do

play20:46

peer-to-peer connections like web RTC

play20:48

for messages are there no servers what

play20:51

do you mean by servers do you mean any

play20:52

protocols so generally peerto peer is

play20:54

never done like if if you thinking of

play20:56

doing peerto peer beat using web RTC or

play20:58

any other protocol it will not scale for

play21:00

a thousand Reasons so no one does peer

play21:02

to-peer even web RTC is sort of tweaked

play21:05

to go through a server eventually in

play21:06

scalable systems like Zoom difference

play21:09

between websockets and web hooks doesn't

play21:11

web hooks also keep sending messages so

play21:14

they're like very different things web

play21:16

Hook is

play21:18

a websocket your

play21:21

server your client you sort of own this

play21:24

code you have a connection to your um

play21:27

web socket server web

play21:30

hooks um when you use an external

play21:32

service for example some of you might

play21:33

have heard of Razor pay which lets you

play21:36

do um payments to Razer pay usually HTTP

play21:41

server Razer pay HTTP server will send a

play21:45

request to your HTTP

play21:48

server and this is called a web hook why

play21:51

is it used if a payment of yours fails

play21:54

so razor pay is the person who has

play21:57

connect connection with banks you know

play22:00

SBI HDFC so on and so forth when people

play22:03

come to my website for example to buy a

play22:05

course they actually hit razor pays Endo

play22:07

okay H bro uh whatever I'm paying 1,000

play22:10

rupees which under the hood sort of hits

play22:11

a bank API eventually the bank needs to

play22:14

tell Razer pay ha this transaction is

play22:16

done and then Razer pay needs to tell my

play22:18

backend server H whatever Su paid X

play22:22

rupees so this Razer pay server talking

play22:26

to my HTTP server when you talk across

play22:28

companies that's what's called a web

play22:30

hook okay you go to razor pay dashboard

play22:32

and say bro hit this web backend server

play22:34

request uh endpoint of mine which is

play22:36

called a web hook you are sending a hook

play22:40

to my back end so I can in my database

play22:42

put K some has bought the course and

play22:44

someone gets access to this course so

play22:45

whenever SBI needs to talk to Razer pay

play22:48

razor pay needs to talk to me they hit

play22:49

my web hook that protocol that they

play22:52

using is still HTTP

play22:54

so this is still an HTTP server request

play22:57

only it's happening from one company

play22:57

compy to another that's what is a web

play23:00

hook what is a web socket it is a real

play23:02

time like a different protocol

play23:04

completely different from um HTTP cool

play23:07

that's the high level difference

play23:08

hopefully you guys have cloned it there

play23:10

are five parts let's look at it the

play23:12

first and two sort of go together it's

play23:14

the backend code for a websocket server

play23:17

and the front end code for a websocket

play23:18

server if you go to package.json you

play23:21

will see I have Express you don't really

play23:23

need Express to create websocket servers

play23:25

you can create it without it but I have

play23:26

used Express there's like a way to

play23:28

create uh web soet servers using Express

play23:30

that way you can expose one endpoint for

play23:33

example SL WS endpoint on uh to be a

play23:37

websocket server and then whatever you

play23:39

can have a/ health endpoint slash user

play23:43

slash whatever uh create as another

play23:46

endpoint that only one endpoint is um

play23:49

web so endpoint and others are standard

play23:51

HTP and points you can also not use

play23:53

express if you want to create a pure

play23:55

websocket server and then there are the

play23:57

types of both Express and

play24:00

WS standard TS config um only difference

play24:04

is um the root D is/ SRC and the out D

play24:08

is/ everything else is like standard

play24:10

that you get and if you look at

play24:12

index.ts let's look at the things that

play24:14

I've imported I've done import Express

play24:16

from Express import HTTP from HTTP now

play24:19

this is this is like the native HTTP

play24:22

module that node provides you um I don't

play24:24

think we need this either but the way

play24:26

I've created this code is that

play24:29

um after you do a const app equal to

play24:32

express and have the port we do a server

play24:34

equal to app. create server app I don't

play24:37

think you really need this uh I guess we

play24:40

need

play24:41

it yeah so I think the reason for this

play24:43

is web socket server expects a native

play24:47

HTTP server here

play24:50

that if I go here look at the

play24:54

options uh let's

play24:56

see

play25:00

yeah there you go the server argument

play25:01

that it expects is a native HTTP server

play25:03

so we can't just say yellow Express this

play25:05

doesn't accept Express here which is why

play25:08

I had to create a native HTTP server

play25:10

what my native HTTP server this is just

play25:12

the module that node just provides you I

play25:13

think H inside the box like out of the

play25:15

box as you can see this HTTP module

play25:18

isn't really part of my package.json

play25:21

so this like node just provides you out

play25:23

of the box we use this to create an HTTP

play25:26

server by using simply http create

play25:28

server I think you you don't really need

play25:30

Express I think you can create it

play25:31

without Express as well and you know

play25:32

there's a different way to create routes

play25:34

here like http.get or whatever so you

play25:37

can do that as well I don't think you

play25:38

need as I said you don't really need

play25:40

Express I have created it this way

play25:41

because this is how backpack has done it

play25:42

I'm just basically copy paste it it and

play25:45

try to break it down if you want to

play25:47

build it yourself you actually don't

play25:49

need a lot of these things you can

play25:50

simply this is enough to create I think

play25:53

a websocket

play25:54

server lastly I've said WSS doon

play25:57

connection oh uh I said initialized

play26:00

websocket server this has all the logic

play26:02

similar to all I have to do if I have to

play26:04

create an HTTP server is const app equal

play26:06

to express or basically this line right

play26:08

here this under the hood creates an HTTP

play26:11

server has all the logic for it

play26:12

similarly this Whoever has written this

play26:15

library has written all the logic to

play26:17

upgrade the request create the

play26:18

persistent connection so on and so forth

play26:20

we use this Library out of the box the

play26:22

only sort of syntax you need to know is

play26:24

websocket server so WSS stands for the

play26:26

websocket server Doon connection anytime

play26:29

a person connects to you the control

play26:31

reaches here someone connected that's

play26:34

what I've logged here and then ws. on

play26:37

message anytime someone sends you a

play26:39

message you can whatever put a log here

play26:42

put something in the database do

play26:44

whatever you want with this information

play26:45

for example in the real world someone

play26:46

would send you either a chat message or

play26:48

someone would send you their movement if

play26:50

they're playing uh Minecraft okay this

play26:53

is where I moved you receive that data

play26:55

here from here you have to run your l

play26:57

you can do whatever you want with this

play26:58

data we don't do anything we just send

play27:01

back from the server okay hello you sent

play27:03

this so just to show duplex

play27:05

communication just to show client can

play27:07

send you some data server can respond

play27:09

back with some data this is like five

play27:12

lines of code that you have to write

play27:14

this is enough to write your server side

play27:16

logic is this what you will write in the

play27:18

real world no in the real world if you

play27:21

user send you a message you will have to

play27:23

basically look at who all are you

play27:24

connected to forward the message to a

play27:26

few of those users so you won't ws. send

play27:28

you will actually do a you know for

play27:30

iterate over all the users you have all

play27:32

the web soet connections you have and

play27:33

send it to them we will see that when we

play27:34

reach this part for now fairly simple

play27:37

websocket server on connection which

play27:39

means anytime someone has created a

play27:41

websocket connection control will reach

play27:43

here what do we do here we log we don't

play27:45

have to log we can remove this but we

play27:47

just log to see H actually connections

play27:48

are being made and anytime that user now

play27:51

sends message that user can send you

play27:52

message once twice 10 times 100 times

play27:54

the control will reach here and then I

play27:57

don't don't have to do anything here but

play27:58

what am I doing I'm just responding back

play28:00

bro yellow I am going to respond back

play28:02

with hello you sent this I could also

play28:04

some be something like okay set

play28:07

interval ws. send hello from

play28:12

server every 1 second I could I don't

play28:15

really need a trigger I can simply be

play28:17

like every 1 second now I'm going to

play28:19

send you some data okay so ws. send can

play28:22

be called anywhere now as long as you

play28:24

have access to this WS which is the

play28:27

socket connection for that specific

play28:30

user once the control reaches here this

play28:33

WS variable is the socket connection to

play28:35

that user if another user sends a

play28:37

request a new control will reach here

play28:40

again a new WS variable will be

play28:41

available from which you can send back

play28:42

data I think fairly straightforward

play28:44

stuff here let me know if not we'll do a

play28:46

poll

play28:47

soon but hopefully this backend server

play28:49

logic is straightforward if not we'll do

play28:51

a poll let me know going try to marinate

play28:53

it ignore whatever you're seeing here

play28:55

this is just my

play28:58

ID being my ID so it's actually just

play29:01

async WS comma Rec these are just types

play29:05

that my IDE shows similar to this

play29:07

listener that's a type that my IDE shows

play29:09

don't worry about that if that's what's

play29:10

confusing you and let's try to run this

play29:12

back in server now the way to do that as

play29:15

always is oh by the end in the end we do

play29:16

server. listen Port which is similar to

play29:18

how we used to do it in like an express

play29:20

application we used to do an app. listen

play29:22

rather than doing an app. listen the

play29:23

server that we we have created we do a

play29:25

server. listen Port why is this schol

play29:27

did we use express so that we can do

play29:28

like other things like app.get SL Health

play29:32

expose

play29:37

like a health check in point so on and

play29:39

so

play29:43

forth cool you don't have to add it just

play29:45

showing you eventually if you hit this

play29:47

end point when your server starts you

play29:49

can also get back like health checks and

play29:50

put other logic here the way to run it

play29:53

go into 1-s simple- ws- server and run

play29:57

TSC

play29:58

sp-b this will what will it do convert

play30:01

your typescript code into JavaScript

play30:03

code if this doesn't work for you npx

play30:05

tscp should work once you've done it

play30:08

node dis

play30:09

index.js should start this on Port 3000

play30:13

I was already running something here but

play30:15

if I remove that node dis index.js

play30:18

should start your websocket server we

play30:20

haven't yet written any client side

play30:21

logic we will write that very soon but

play30:24

this is a similar to an HTTP server that

play30:26

has started

play30:28

trade

play30:31

forward shall we proceed let me do a

play30:33

poll real

play30:35

quick I think it's

play30:37

straightforward hopefully because it's

play30:39

like very similar to http servers so

play30:41

like nothing too fancy here just one

play30:43

more extra thing you get you send and

play30:45

receive separately okay very good so

play30:49

now this is in the past when we created

play30:54

HTTP servers we were able to hit them

play30:56

via Postman a mobile app or uh a browser

play31:00

anywhere right similarly websockets also

play31:02

you can hit from anywhere you want if I

play31:04

go to my browser and search for

play31:06

postwoman this was someone who created

play31:09

like aun funny project called postwoman

play31:11

which was a open source version of

play31:13

Postman uh and this has like a realtime

play31:16

part I think Postman also has this like

play31:18

now Postman also lets you create a

play31:19

websocket connection but think of this

play31:21

application which is hops scotch.io now

play31:24

used to be called postwoman in back in

play31:25

the day it let gives you a section

play31:28

called real time which lets you create a

play31:29

websocket connection similar to how we

play31:32

used to go to postman and send HTTP

play31:34

requests this is just another client and

play31:36

if I click on if I put if you put the

play31:38

right address here which is ws/ loal

play31:41

3000 if I open inspect click on the

play31:43

network

play31:45

Tab and click on connect here you will

play31:49

see a websocket connection has been

play31:50

created there are no messages that have

play31:52

been exchanged right now but the

play31:54

connection has been made if I send a

play31:56

message here let's

play31:58

say the way to do that is it sort of

play32:00

lets you put any random body here and

play32:01

send you will see I send a I sent a

play32:04

message I also received the message

play32:07

hello you sent this from the backend

play32:08

server I can also confirm this here so

play32:11

what is this this is just another client

play32:14

uh for hitting a websocket server

play32:17

eventually you want to own this

play32:18

application this will be your own chat

play32:20

application your own game but at least

play32:21

this shows you how can you now it's a

play32:24

generic websocket server that You' have

play32:25

created anyone can talk to it using

play32:27

websocket clients we specifically need

play32:29

to talk to it using a website so either

play32:31

a react application next CH application

play32:34

so we will write a simple client in

play32:36

JavaScript that's able to talk to it but

play32:38

like anyone can talk to it it's an open

play32:40

server that understands the web soet

play32:41

protocol as long as someone has written

play32:43

a websocket client they can talk to it

play32:45

they can send messages they can receive

play32:46

messages what we haven't yet written any

play32:49

logic here in the real world you'll be

play32:51

like message types if message. type

play32:53

equal to

play32:54

chat then only you forward chat messages

play32:57

if it is a message the server does not

play32:59

understand it does nothing we have to do

play33:00

all of that right now but just the

play33:03

communication bit has been hooked up now

play33:05

let's look at the client that I have

play33:07

written this is a simple index

play33:14

[Music]

play33:21

or

Rate This

5.0 / 5 (0 votes)

Related Tags
Distributed ComputingReal-Time CommunicationWebsocketsRedisChat SystemsSystem DesignOpen SourceBack-End DevelopmentSoftware EngineeringTech Education