How Web Sockets work | Deep Dive

Summary

This video dives into advanced WebSocket functionalities, essential for building powerful real-time applications. It revisits the WebSocket handshake, detailing the negotiation process from HTTP to WebSocket, and explains the crucial headers exchanged. The video also explores the structure of WebSocket frames, including the importance of masking and fragmentation for secure and efficient data transmission. Applications in various domains, such as chat systems, web applications, gaming, and stock exchanges, demonstrate the versatility and potential of WebSockets for real-time communication.

Takeaways

• 🌐 Websockets enable real-time two-way communication between browsers and servers, which is more efficient than traditional HTTP requests that require a new connection for each request.
• 🤝 The WebSocket handshake is a crucial step where the client and server negotiate the upgrade from HTTP to a WebSocket connection, involving specific headers and a handshake process.
• 🔑 The 'Sec-WebSocket-Key' header is a randomly generated base64-encoded string used by the server to verify the handshake, ensuring a secure and authenticated connection.
• 🗂️ The 'Sec-WebSocket-Accept' header is generated by the server using a hash function and the client's 'Sec-WebSocket-Key', proving that the server recognizes the handshake initiation.
• 🔄 Masking in WebSockets obscures payload data to prevent it from being interpreted as HTTP content by caching proxies, ensuring smooth data transition through network infrastructure.
• 📦 Frames are the basic units of data exchanged in WebSockets, with each frame having a specific structure including fin bits, opcode, mask bit, payload length, and payload data.
• 🔄 Fragmentation in WebSockets refers to splitting a large message into smaller chunks for transmission, which is useful for handling large messages without overwhelming the connection.
• 📈 The FIN bit in a WebSocket frame indicates whether the frame is the final fragment of a message or if more fragments are to follow, allowing for gradual delivery of large messages.
• 🔒 WSS (WebSocket Secure) is used for secure communication over TLS, ensuring encrypted data transmission between the client and server.
• 🎮 WebSockets are versatile and find applications in various domains such as chat applications, real-time updates in web applications, multiplayer gaming, and real-time stock ticker updates.
• 💡 Understanding the advanced concepts of WebSockets, such as fragmentation and masking, is key to mastering real-time communication technologies and unlocking their full potential.

Q & A

What is the primary purpose of the WebSocket handshake?

The primary purpose of the WebSocket handshake is to establish a real-time communication channel between a client (browser) and a server by upgrading from an HTTP connection to a WebSocket connection.

Why is the WebSocket protocol considered to be more efficient than traditional HTTP for real-time communication?

WebSocket protocol is more efficient for real-time communication because it uses a persistent connection that remains open as long as it is not interrupted by either party, unlike traditional HTTP which creates a new connection for each request and closes it after the request is completed.

What does the 'Sec-WebSocket-Key' header in a WebSocket handshake request contain?

The 'Sec-WebSocket-Key' header contains a randomly generated base64-encoded string that is unique to the handshake. It is used by the server to verify the handshake later.

What HTTP status code indicates a successful WebSocket handshake upgrade?

A successful WebSocket handshake upgrade is indicated by an HTTP status code of 101, which stands for 'Switching Protocols'.

How does the server verify the WebSocket handshake request from the client?

The server verifies the WebSocket handshake request by checking the presence of 'Upgrade: websocket' and 'Connection: Upgrade' headers, and then generates a 'Sec-WebSocket-Accept' header using a cryptographic hash function applied to the 'Sec-WebSocket-Key' from the client, followed by a base64 encoding.

What is the role of masking in WebSocket communication?

Masking in WebSocket communication ensures that the data within the frames is unpredictable and does not resemble valid HTTP requests or responses, which discourages intermediaries from caching or modifying the data.

Why is WebSocket fragmentation important for handling large messages?

WebSocket fragmentation is important for handling large messages because it splits a large message into smaller chunks, preventing buffer overflow and allowing for gradual delivery of the message, which is beneficial for applications where immediate display of partial data is valuable.

What does the 'Fin' bit in a WebSocket frame indicate?

The 'Fin' bit in a WebSocket frame indicates whether the frame is the final fragment of a message or if there are more fragments to come. A value of one means it's the final fragment, while a value of zero means there are more fragments.

What are some applications of WebSocket technology?

WebSocket technology is used in various applications such as chat applications for real-time messaging, web applications for live updates, multiplayer gaming for seamless experiences, real-time drawing apps for collaboration, and stock exchanges for instant updates on stock prices.

How does the WebSocket protocol handle secure communication?

The WebSocket protocol handles secure communication by using 'wss://' as the scheme, which sets a secure flag and initiates a TLS handshake between the server and client for encrypted communication.

What are the reserved bits 'RSV1', 'RSV2', and 'RSV3' in a WebSocket frame used for?

The reserved bits 'RSV1', 'RSV2', and 'RSV3' in a WebSocket frame are typically set to zero and are reserved for future use cases or potential extensions to the WebSocket protocol.

Outlines

00:00

🌐 Introduction to Advanced WebSocket Functionalities

The video begins with a recap of basic WebSocket concepts and introduces the topic of advanced functionalities. The focus is on understanding the WebSocket handshake, which upgrades HTTP to WebSocket for real-time communication. This section details the steps and headers involved in the handshake process, highlighting the differences between HTTP and WebSocket connections.

05:00

🔄 WebSocket Handshake Process

The second part dives into the specifics of the WebSocket handshake process. It explains how the client initiates a handshake with an HTTP GET request, including special headers. The server's response involves a status code of 101, confirming the upgrade. Key headers like 'Upgrade', 'Connection', and 'Sec-WebSocket-Key' are discussed in detail, along with the server's verification process using SHA-1 hashing and Base64 encoding.

10:02

🔒 Ensuring Secure and Authenticated Connections

This section emphasizes the importance of the WebSocket handshake in establishing a secure and authenticated connection. The process includes a random 'Sec-WebSocket-Key' from the client and a corresponding 'Sec-WebSocket-Accept' from the server, ensuring that the handshake is legitimate and secure. If any validation fails, the connection is not established.

📡 WebSocket URI and Data Frames

Here, the video explains the format of WebSocket URIs (WS for insecure, WSS for secure connections) and the structure of data frames. Each frame includes fields like FIN, RSV, Opcode, Mask, Payload length, and Masking key, which are crucial for proper data transmission. The role of masking and fragmentation in ensuring smooth data flow and preventing issues with network infrastructure is also covered.

📦 Handling Large Messages with Fragmentation

This part discusses the concept of fragmentation in WebSockets, which allows large messages to be split into smaller fragments. This prevents buffer overflow and enables gradual data delivery. The video explains the significance of the FIN bit in indicating the final frame of a message and how fragmented messages are processed for efficient data transmission.

🎮 Real-World Applications of WebSockets

The final section explores various real-world applications of WebSockets, such as chat applications, real-time web updates, multiplayer gaming, collaborative tools, and stock ticker updates. It encourages viewers to experiment and understand the underlying concepts for mastery, inviting questions and project ideas to foster community growth.

Keywords

💡WebSockets

WebSockets is a communication protocol that provides full-duplex communication channels over a single TCP connection. It allows for real-time two-way interaction between a client (typically a web browser) and a server. In the video, WebSockets are discussed as a means to enable advanced real-time applications, emphasizing its role in reducing server load compared to traditional HTTP requests which require a new connection for each request.

💡Handshake

The term 'handshake' in the context of the video refers to the initial process where the client and server agree to switch from an HTTP connection to a WebSocket connection. This is a crucial step as it sets up the persistent connection that is characteristic of WebSockets. The script details the headers exchanged and the steps involved in this process, highlighting its importance for establishing real-time communication.

💡Upgrade Request

An 'upgrade request' is an HTTP header sent by the client indicating the desire to switch to a different protocol, in this case, WebSockets. The script explains that the client sends an HTTP GET request with specific headers, including 'Upgrade: websocket', to initiate the handshake process. This request is part of the negotiation between the client and server to establish a WebSocket connection.

💡WebSocket Key

The 'WebSocket Key' is a base64-encoded random string sent by the client in the handshake process. It is used by the server to verify the handshake and ensure that the connection is being initiated by a legitimate WebSocket client. The script mentions that this key is crucial for the server's acceptance of the handshake, contributing to the security of the connection.

💡Sec-WebSocket-Accept

The 'Sec-WebSocket-Accept' header is part of the server's response to the client's handshake request. It contains a cryptographic hash of the 'WebSocket Key' provided by the client, demonstrating that the server has received and accepted the request to upgrade to a WebSocket connection. The script describes how this header confirms the successful establishment of the WebSocket connection.

💡Frames

In WebSockets, 'frames' are the basic units of data exchanged between the client and server. Each frame has a specific structure that includes information about the type of data it carries, such as text, binary, or control frames like ping. The script explains that frames are used to transmit data in a structured way, allowing for efficient communication over the WebSocket connection.

💡Masking

Masking in WebSockets is the process of obscuring the payload data to ensure that it does not resemble HTTP traffic, which can prevent issues with caching proxies that might otherwise interfere with the WebSocket data. The script describes how masking makes the data unpredictable, discouraging incorrect interpretation or manipulation by intermediaries.

💡Fragmentation

Fragmentation in the context of WebSockets refers to the division of a large message into smaller frames for transmission. This is useful for handling large messages that might exceed buffer limitations or cause performance issues. The script explains that fragmentation allows for the gradual delivery of large messages, improving the efficiency and reliability of data transfer over the WebSocket connection.

💡FIN Bit

The 'FIN bit' in a WebSocket frame indicates whether the frame is the final fragment of a message or if more fragments are to follow. A value of one signifies the final fragment, while a value of zero indicates that additional fragments are expected. The script uses the FIN bit as an example to illustrate how messages are broken down and reassembled during fragmentation.

💡Real-Time Applications

The term 'real-time applications' refers to software systems that require immediate responses or updates, such as chat applications, live gaming, or stock ticker updates. The script discusses how WebSockets enable the development of such applications by providing a persistent connection for continuous data exchange without the overhead of repeated HTTP requests and responses.

💡WSS

WSS stands for WebSocket Secure and is used for establishing a secure WebSocket connection over TLS (Transport Layer Security). It is analogous to HTTPS for HTTP connections. The script mentions WSS as an example of how WebSockets can provide secure communication channels, important for applications that require data privacy and integrity.

Highlights

Introduction to advanced WebSocket functionalities for building powerful real-time applications.

WebSockets enable real-time two-way communication between browsers and servers, unlike traditional HTTP.

The WebSocket handshake is a crucial step for upgrading from HTTP to a WebSocket connection.

Explanation of the client and server steps involved in the WebSocket handshake process.

The WebSocket protocol is independent, with its only relationship to HTTP being the handshake.

Details on the HTTP GET request with specific headers for initiating a WebSocket upgrade.

The role of the 'Sec-WebSocket-Key' header in the handshake for server verification.

Server response with a status code of 101 indicates a successful upgrade to WebSocket.

The 'Sec-WebSocket-Accept' header verifies the client's handshake initiation to the server.

WebSocket connections are persistent and do not require constant HTTP requests and responses.

WebSocket frames are the basic units of data exchanged between client and server.

The structure of WebSocket frames includes FIN, RSV, Opcode, Mask, Payload Length, and Masking Key.

Masking ensures a smooth transition of real-time data through various network infrastructures.

Fragmentation in WebSockets allows for the transmission of large messages in smaller chunks.

FIN bit in fragments indicates whether it is the final frame or if more follow.

WebSockets' versatility in applications such as chat apps, web applications, gaming, and stock exchanges.

Emphasis on the importance of understanding WebSocket concepts for true mastery in real-time communication.

Transcripts

hey everyone welcome back previously I

have explained the basics of websockets

and how they enable realtime two-way

communication between browsers and

servers but many of you wanted more so

today we'll dive deeper and explore

Advanced websocket functionalities and

how to build powerful real-time

applications with them we'll first

revisit the websocket handshake which is

a crucial step where the client and

server negotiate the upgrade from HTTP

to a websocket connection we'll then

break down the handshake process in

detail exploring the headers exchanged

and the steps involved on both sides so

let's dive in and get

[Music]

started previously in my web socket

introduction video we learned that HTTP

uses distinct connection for separate

request it increases the load on the

server as the server has to create a new

handshake for every request once the

request is completed connection is

closed on the other hand websocket

connection is persistent as long as it

is not interrupted by either of the

parties the websocket protocol is an

independent TCP based protocol call its

only relationship to http is that its

handshake is interpreted by HTTP servers

as an upgrade request the websocket

handshake process is a crucial step that

establishes a real-time Communication

channel between a client which is

browser and a server it leverages the

familiar HTTP protocol for initial

connection but then negotiates an

upgrade to the websocket protocol here

is a detail breakdown of the client and

server steps involved the client opens a

regular HTTP connection to the servers

specifying a desired websocket endpoint

in the URL usually prefixed with WS or

WSS for a secure connections it sends an

HTTP get request with a specific headers

to indicate the intention to upgrade to

websocket now the client set headers

looks like this wherein the upgrade

websocket headers tells the server that

the client wants to upgrade the

connection to websocket the connection

upgrade header informs the server that

the client desires an upgrade from the

current connection protocol the SEC

websocket key is the header that

contains a randomly generated b64

encoding string unique to this handshake

it will be used by the server to verify

the handshake later and then there are

some additional headers depending on the

server implementation and desired

features the additional headers like SE

websocket version specifying the

websocket protocol version or SE

websocket protocol requesting a specific

sub protocol which might be included and

then the client waits for the service

response to the HTTP upgrade request on

the other end when the server receives

HTTP get request from the client it

checks the request headers to ensure

that they are valid for a websocket

handshake and sends an HTTP response

with a status code of 101 switching

protocols to indicate successful upgrade

to websocket the handshake from server

looks like

this let's go through these steps in

detail the server first verifies the

presence of upgrade websocket and

connection upgrade headers now the

primary goal of the server processing

and generating the SE websocket accept

header is to prove to the client that

the server recognized it as a valid

websocket hand check initiation and is

willing to proceed and the server can

confirm that the hand check request

originated from a genuine web client and

not a malicious actor attempting to

impersonate a websocket connection the

server receives the base 64 encoded

string from the SE websocket key header

the server then generates a new string

by concatenating the received SE

websocket key with a predefined u ID for

example this and then applying a

cryptographic hash function such as sha1

and this new string is then Bas 64

encoded the server sends an HTTP

response with a status code of 101

switching protocols to indicate

successful upgrade to websocket the

response includes the following

headers an upgrade web socket header

echoing back the upgrade request from

the client connection upgrade confirming

the upgrade from HTTP to websocket and

SE websocket accept this is the base 64

for encoded string generated previously

and this verifies that the client

initiated the hand check once the client

receiv receives the service response

with the correct SEC websocket accept

header it validates the response

confirming a successful hand check both

client and server can now start

exchanging data using the websocket

protocol over the established TCP

connection the HTTP connection is

essentially replaced by the websocket

connection now if the SE websocket

accept value does not match the expected

value or if the header field is missing

or if the HTTP status code is not 101

the connection will not be established

and websocket Frames will not be sent if

any code other than 101 is returned from

the server clients have to end the

connection the web socket handic ensures

a secure and authenticated Connection by

including a random key SEC websocket key

generated by the client and verified by

the server through SE websocket accept

header this handshake process paves the

way for realtime two we communication

between the client and server without

the need for a constant HTTP request and

responses websocket has a default Ur

format such as below like I said before

it can be either WS or WSS just like

HTTP protocol the port component is

optional as default Port 0 is used for

ws and 443 is used for

WSS WSS is used as a secure URI as a

secure flag is set and TLS handshake is

done between server and L for secure

Communication in websocket protocol data

is transmitted using sequence of frames

frames in web soet are the basic units

of the data that are exchanged between

the client and server and each frame has

a specific structure fin or F indicates

whether the frame is in the final

fragment or larger message a value of

one means it's the final fragment and

while zero means there are more

fragments to come more on fragments

later RSV 1 rsv2 RSV three these three

beds are the reserve bits they are

typically set to zero and they are

currently reserved for future use case

or potential extensions to the websocket

protocol the op code Fields defines the

type of data the frame carries it could

be a textual data in utf8 encoded format

or a binary data or a ping frame for

keep alive

events The Mask pit indicates if the

payo data is masked masking is always

applied to frames sent from client to

server we'll take a closer look at

masking shortly the payload length

defines the length of the payload data

and the length field itself can vary

from 0 to 125 depending on the size of

the payload the masking key is used to

obscure payload data and the payload

data which is a variable length has the

actual content of the messages such as

text or binary data now coming back to

masking masking main goal in websocket

is to ensure smooth transition of

real-time data through various Network

infrastructure that wasn't necessarily

designed with websockets in mind before

websockets were standardized some

Network infrastructure component

particularly those designed purely for

HTTP traffic try to be smart when it

came to caching these caching proxies

sometimes stored responses from a server

expecting that they can later reuse the

same response for similar HTTP request

however websocket connections don't

follow the traditional HTTP request

response pattern unmasked websocket data

could be wrongly interpreted as HTTP

content and can can be stored

potentially leading to issues later when

a real HTTP request is made masking

obscures the data within websocket

frames in a way that makes it

unpredictable and very unlikely to

resemble valid HTTP request or responses

this discourages proxies and other

intermediaries from attempting to Cache

or modify the data as it clearly doesn't

fit the HTTP model masking basically

makes web soer traffic look distinctly

different from HTTP discouraging

incorrect interpretation or manipulation

fragmentation on the other hands in

websockets refers to the process of

splitting a large message into smaller

chunks for transmission over the network

this technique is particularly useful

for handling a very large messages that

might overwhelm the connection or exceed

buffer limitations on either the client

or server it is to prevent buffer

overflow imagine a user uploading a

large file through a websocket

connection and without fragmentation the

entire file would be sent as a single

message this could potentially overflow

buffers on the client or server leading

to connection errors or performance

issues fragmentation allows for gradual

delivery of large messages the receiver

can start processing the initial

fragments while the remaining fragments

are still being transmitted this can be

beneficial for applications where

immediate display of partial data is

valuable such as receiving updates

during a long running process now when a

message is split up into fragments each

fragment is sent with a fin bit set to

zero for all but the final frame in the

sequence the F bit indicates whether the

current frame is the final frame in the

message or whether more frames will

follow if the fin bit is set to zero it

means that there are more frames coming

and the receiver should continue to wait

for additional frames before processing

the message and when the final fragment

of the message is sent it is sent with

the fin bit set to one this signals to

the receiver that this is the last frame

in the message now websockets are a

versatile communication technology and

find applications in various domains for

example in chat applications for sending

and receiving messages quickly and

effici ly which I have in fact

previously covered in my WhatsApp system

design video it can also enable

real-time Communication in web

application such as changes and updates

on the web pages in gaming industry it

can facilitate seamless multiplayer

gaming experiences or imagine a realtime

drawing app where multiple users can

collaborate on a canvas or maybe in

stock exchanges for real-time updates on

stock ticker prices these advanced

websocket concepts might seem complex

but the possibility is they unlock are

worth the effort remember breaking

things down experimenting and

understanding the why behind each

feature will lead to True Mastery so if

you have any questions or cool project

ideas drop them in the comments and

let's keep the ban Community growing

[Music]