Sockets Tutorial with Python 3 part 1 - sending and receiving data

Summary

This Python tutorial series introduces sockets programming with a focus on creating a server and client using Python's socket library. The video demonstrates setting up a TCP socket, binding it to a local IP and port, and listening for connections. It also covers sending and receiving data between the server and client, and the importance of buffering data streams. The tutorial simplifies the concepts for beginners, highlighting the need for handling larger messages and maintaining open connections in future videos.

Takeaways

• 📝 The tutorial covers creating a basic server and client using Python's socket library.
• 🔌 The socket object is created using `socket.socket()` with the AF_INET family for IPv4 and SOCK_STREAM for TCP.
• 📍 The server binds to a local IP and port, in this case, 'localhost' and port 1234.
• 🔗 The server listens for incoming connections with a queue of five to handle multiple requests.
• 🤝 Upon receiving a connection, the server accepts it and stores the client socket and address information.
• 💬 The server sends a welcome message to the client using UTF-8 encoded bytes.
• 🔄 The client connects to the server using the same socket object but with the connect method instead of bind.
• 📡 The client receives messages from the server in chunks, determined by the buffer size set in the receive method.
• 🔍 The script explains the concept of a socket as an endpoint for sending and receiving data on a network.
• 🔑 The tutorial touches on the importance of proper buffer management for handling data streams in socket communication.
• 🔄 The script demonstrates a basic example of socket communication but mentions the need for more complex logic for long-term connections.

Q & A

What is the purpose of the 'socket' module in Python?

The 'socket' module in Python provides a way to create and manipulate network connections using the BSD socket interface. It is part of Python's standard library and does not require installation.

What does 'AF_INET' and 'SOCK_STREAM' represent in the context of creating a socket?

'AF_INET' represents the address family, which corresponds to IPv4. 'SOCK_STREAM' represents the type of socket, which in this case is a TCP socket for a streaming connection.

Why is the server binding to 'socket.gethostname()' and port 1234?

The server is binding to 'socket.gethostname()' to listen on the local machine's hostname and port 1234 to specify the port number on which it will accept incoming connections.

What is a socket in networking terms?

A socket is an endpoint for sending and receiving data across a network. It sits at an IP address and a port, facilitating communication between networked devices.

What does the 'listen' method do for a server socket?

The 'listen' method on a server socket allows the server to prepare for incoming connections. It sets up a queue for managing multiple incoming connection requests.

How does the server accept a client connection?

The server accepts a client connection using the 'accept' method, which returns a new socket object representing the connection, as well as the address of the client.

What is the purpose of sending a welcome message to the client socket?

Sending a welcome message to the client socket is a way to establish communication and inform the client that the connection has been successfully established.

What is the difference between binding a socket and connecting a socket?

Binding a socket is done on the server side to specify the IP and port on which it will listen for incoming connections. Connecting a socket is done on the client side to specify the IP and port of the server to which it wants to connect.

Why is it necessary to buffer data when using sockets?

Buffering data is necessary because sockets deal with a stream of data. The buffer determines how much data is received at a time, which is important for handling varying data sizes and maintaining the flow of data.

What is the significance of the 'utf-8' encoding when sending and receiving messages?

The 'utf-8' encoding is significant because it allows for the transmission of text data in a standardized format that can be correctly interpreted by both the sender and the receiver.

How can you handle receiving messages larger than the buffer size?

To handle messages larger than the buffer size, you can implement a loop that continues to receive data until the entire message is buffered. This often involves checking the length of the received message and concatenating the chunks until the complete message is assembled.

What is the role of the 'close' method in socket programming?

The 'close' method is used to close the socket connection, freeing up the resources associated with the socket. It is important to properly close sockets when they are no longer needed to avoid resource leaks.

Outlines

00:00

🔌 Introduction to Python Sockets

This paragraph introduces the concept of sockets in Python programming. The tutorial begins with the creation of a server and client using Python's built-in 'socket' library, which doesn't require installation. The server is set up to use IPv4 and TCP protocols, binding to the local machine's hostname and port 1234. The explanation includes a brief introduction to what a socket is, highlighting its role as an endpoint for communication. The server is configured to listen for incoming connections with a queue size of five, and upon receiving a connection, it sends a welcome message to the client using UTF-8 encoding. The paragraph concludes with the server printing a debug message indicating a successful connection from the client's address.

05:01

📡 Setting Up the Client Socket

The second paragraph focuses on setting up the client side of the socket communication. It starts by copying the server's socket code and modifying it for the client's purpose. Instead of binding, the client's socket is used to establish a connection to the server's IP and port. The paragraph discusses the flexibility of sockets to connect to local or remote machines, and the importance of choosing an appropriate buffer size for receiving data. It demonstrates receiving a message from the server, decoding it from UTF-8 bytes, and printing it. The client's execution is shown, including running the server and client scripts, and observing the welcome message from the server. The paragraph ends with a brief mention of the need for a more complex and long-term connection strategy in real-world applications.

10:02

🔄 Buffering and Handling Data Streams

The final paragraph delves into the intricacies of buffering data in socket communication. It explains the concept of data streaming and the necessity of handling messages larger than the initial buffer size. The paragraph suggests using a loop to continuously receive data and demonstrates a method to accumulate incoming data into a full message variable. It also touches on the importance of knowing when the connection is completed and how to properly close the client socket. The discussion includes a practical example of modifying the client script to handle a continuous stream of data, and the importance of sending information about the size of the incoming data in the header. The paragraph concludes with a teaser for the next video, which will cover advanced buffering techniques and maintaining an open stream for sockets.

Keywords

💡Socket

A socket is a fundamental concept in network programming, representing one endpoint of a two-way communication link between two programs running on a network. In the video, sockets are used to establish a connection between a server and a client using Python. The script demonstrates creating a socket object, binding it to an IP and port, and then listening for incoming connections, which is central to the video's theme of networking with Python.

💡AF_INET

AF_INET is a constant used in socket programming to specify the address family, which in this case indicates an IPv4 Internet Protocol address. The script mentions 'socket.AF_INET' as part of the socket creation process, which defines the type of network protocol the socket will use, tying it to the Internet's most common addressing system.

💡SOCK_STREAM

SOCK_STREAM is a socket type used for creating a connection-oriented, stream-based socket. In the context of the video, it is used to establish a TCP (Transmission Control Protocol) connection, which ensures data packets are received in the same order they were sent and that the connection is reliable. This is crucial for the server-client model demonstrated in the script.

💡Bind

Binding in socket programming refers to the process of assigning a socket to a specific IP address and port number. In the script, the server socket is bound to 'socket.gethostname()' and a specific port (1234), which allows the server to listen for incoming connections on the local machine, illustrating a key step in setting up a server.

💡Listen

The listen method in socket programming is used to put a socket into a passive mode where it waits for incoming connections. The script describes using 's.listen()' on the server socket to prepare it to accept connections, which is a necessary step before the server can accept any client requests.

💡Accept

Accept is a method used with listening sockets to accept a connection from a client. In the script, 's.accept()' is called to accept a connection, which returns a new socket object representing the connection with the client and the address of the client, demonstrating the server's interaction with a client.

💡UTF-8

UTF-8 is a character encoding for electronic communication that can represent any character in the Unicode standard, yet is backward compatible with ASCII. In the video, 'utf-8' is used when sending and receiving messages to ensure that the text data is properly encoded and decoded, which is essential for communication between the server and client.

💡Connect

The connect method in socket programming is used to initiate a connection to a specified IP address and port number. In the client script, 's.connect()', is used to establish a connection to the server, which is a fundamental action for the client to start communication with the server.

💡Buffer

A buffer in the context of networking is a temporary storage area, usually in memory, where data is held while being transferred between different parts of a system. The script discusses buffering data received from the server, which is important for handling data streams and ensuring that all data is received correctly.

💡Stream

A stream in networking refers to a continuous flow of data between a sender and a receiver, without the need to send and receive messages in discrete packages. The video script explains that the socket established is a 'streaming socket', which means it handles data as a continuous flow, an important concept for understanding how data is sent and received in the example.

💡TCP

TCP, or Transmission Control Protocol, is one of the main protocols in the Internet protocol suite. It is a connection-oriented protocol that guarantees the delivery of data packets in the same order they were sent. The script mentions TCP in the context of using 'SOCK_STREAM', which corresponds to TCP, emphasizing the reliable nature of the connections being made.

Highlights

Introduction to a Python tutorial series on working with sockets.

Explanation of creating server and client .py files for socket communication.

Importing the socket library, which is part of Python's standard libraries.

Creating a socket object with the family type AF_INET and socket type SOCK_STREAM for TCP.

Binding the socket to a tuple consisting of an IP address and a port number.

Understanding the concept of a socket as an endpoint for data communication.

Server preparation for incoming connections with the 'listen' method and a queue size of five.

Accepting client connections and storing the client socket object.

Sending a welcome message to the client using UTF-8 bytes.

Client-side socket connection to the server using the 'connect' method.

Receiving and decoding messages from the server in the client script.

Demonstration of running the server and client scripts and establishing a connection.

Discussion on the immediate disconnection issue and the need for a more persistent connection.

Explanation of buffering data in sockets and handling messages larger than the buffer size.

Technique for continuously receiving data by using a while loop and checking message length.

Clarification on the need to close the client socket to properly end the connection.

Preview of the next tutorial focusing on advanced buffering and maintaining an open stream.

Acknowledgment of channel members and support from the community.

Transcripts

what is going on everybody and welcome

to a sockets with Python tutorial series

in this video we're gonna be working

with sockets so we've got a lot of stuff

to cover so let's get into it so I've

got a server dot pi and a client dot pi

already created so if you haven't done

that we should probably haven't go ahead

and make two files and we're gonna get

started in the server dot pi so to get

started we're going to import socket now

socket is part of your standard

libraries so you won't need to install

that or anything like that you already

have it now what we want to do is define

our socket object so s equals socket dot

socket and the socket family type is AF

eye net and then the actual type of

socket is going to be socket dot sock

stream now a fi net corresponds to ipv4

and sock stream corresponds to TCP so

this will be a streaming socket and if

anybody knows what a fi net like why is

it called a fi net like why don't we

just say socket dot I don't know ipv4

let me know down below really curious so

anyways it is what it is so that's our

socket object now what we want to do is

bind to that socket so that's dot bind

and we want to bind it to a tuple based

on sometimes this will be different but

anyway the tuple will be in this case

because of the type of socket an IP and

a ports now this is our server so we're

actually just gonna host the server on

the same machine that we have the code

so we're gonna bind to socket get

hostname so whatever the host name is

basically localhost and then the port

we're gonna go with one two three four

you can go with any pretty much

four-digit port the lower digits are

generally gonna be already occupied by

some other program and you just want to

step on their toes and more likely

they're gonna step on your toes so once

we've bound that socket also let's just

talk real briefly what even is a socket

so so a socket is just the endpoint

generally you're gonna have like two

endpoints right to have a communication

so that's just the end

that receives data so with a socket you

send in you receive data so the socket

itself is not the communication it's

just the end point that like receives

that communication on that end point

sits at an IP and a port so we will s

top bind that socket now what we want to

do is s dot listen this is our server

we're gonna be making some connections

to this server so server has to be

prepared for these incoming connections

and this server will prepare and leave a

queue of five so if it's under some sort

of heavy load and like multiple

connections are coming in so fast that

it can't respond quickly enough and they

start stacking up we'll have a queue of

five should be more than enough for our

purposes so then what we're gonna say is

now we're just gonna listen basically

forever for our connections and if we

get one we're gonna say client socket

and then address equals s dot except so

anybody connects hey we're happy to see

you we're gonna connect and then we're

going to store the client socket object

on into this variable client socket and

then address is where are they coming

from so this will be like the IP address

basically so so what we can accept this

and then the client socket this is just

another socket object much like our this

socket object here we can send

information to this socket we can this

socket will receive information as we

will eventually describe in client pie

okay cool so once we've got that

connection let's just print some general

debugging F strings we're gonna say

connection from address has been

established exclamation mark because

this is serious business okay so now

we're gonna do is a client socket dot

send so we're actually going to send

information to the client socket which I

feel like is weird like the client

socket is a socket object but it's like

a foreign socket object so we're going

to send information to it but I almost

feel like I don't know it would make

more sense to me if like we did s dot

send and then we passed in client socket

this feels weird to me but anyways this

is like our local version of the client

socket so to send them for me

into that client we're gonna say client

socket dot send what do we want to send

to this person so we're gonna say bytes

you also could just do like a string and

then dot in code or whatever anyways

we're gonna say here welcome to the

server and then what type of bytes are

these these are UTF bytes utf-8 bytes so

now that's pretty much all we need to do

to send some information so I'll save

that now this is not nearly enough for

regular use but this is a basic examples

basic as I could make it so now we're

going to work on our client dot pi

so again client is much like server it's

gonna use the same exact socket so I'm

actually going to copy paste that

information now this socket rather than

binding instead what this socket really

wants to do is connect so we're gonna

say s dot connect and then we pass a

tuple again of the IP and the port that

we wish to connect to in this case if

its socket get hostname and the IP same

as for one two three four now often is

the case your client will be like remote

to your server it won't actually be on

the same machine so with sockets you can

communicate to and from Python programs

on the same machine on like a locally

networked set of machines or even

remotely networked machines so it just

doesn't matter in most cases you're

probably not actually gonna use socket

doc yet hostname or at least not all of

the cases you will probably won't do

that you'll connect to an actual public

IP or something like that or a local IP

but for now we're gonna do everything on

the same machine so everybody can follow

along cool so we make our connection and

we could be done but what we're gonna

accept that message that was sent to us

so we're gonna say message equals s dot

R ec v4 receive and we're gonna receive

and this is our buffer so remember this

TCP socket here this is a stream of data

so you've got a stream you have to

decide in how big of chump chunk chunks

or how many chumps uh how big a chunks

of data do we want to receive at a time

so in this case 1024

that's more than Pig enough but it

really depends on what you're doing like

if you're sending massive files you

wouldn't want a really tiny buffer size

you might want something a little larger

so and then it also depends on the

application that you're then gonna use

it with and stuff that this case can get

really complicated really fast like if

you're really into like networking and

stuff you could be paid a decent amount

of money like to come up with the proper

amounts for these things but anyway for

now 1024 good enough so now we're gonna

say is just print message decode and

this was utf-8 so we go to code and

again the the the way sockets are

communicating at the moment at least

with the type of socket that we've

established here is a byte stream so

we're receiving these bytes so they're

sent as bytes the received as bytes and

then we just decode the bytes so in this

case we should get the message saved

everything so now all we need to do is

run it so I'm gonna do CMD here and I

honestly this is my local machine this

is probably not 3/7 yes so instead what

I'm gonna say is PI - 3 7 it shouldn't

actually matter you should probably be

able to do it on any version of Pi 3

anyway

run the server so the server is now

running and now I just need to open up

one more for our client PI - 3 point

hello 3.7 client dot PI and I think what

I'll do is I'll like I don't know

something like that we connect we see

welcome to the server from our server

and then here

or can we see it here connection from

our IP address has been established okay

so it looks like that worked for us but

this is also just like immediately

disconnected which is probably not

really what we're after you're probably

looking for a more long-term connection

but we could just like keep running that

again like every time we wanted to

receive a bit of information but that's

kind of silly most people aren't

actually doing that with sockets they're

gonna be doing something far more

complex so I think the last thing I want

to show you guys before we leave and go

to like the next video is you know the

way that truly sockets are gonna work in

the way that you know you're actually

gonna be buffering data is most of the

time you you just like can't expect that

with this that this will be enough so in

this case you could send just about any

message as long as it was less than 1024

bytes of data and you would be totally

fine and you could you could set this as

big as you could plausibly conceive but

that would be silly what really happens

is again it's a stream so I'm gonna set

this to 8 and then hit save for the

client and I honestly don't remember did

we I don't ya know we haven't done

anything to the server so I'll leave the

server running and instead let's rerun

the client again and you'll see what we

get is just welcome right because we

only received that one little itty-bitty

buffer but we're not actually handling

for really anything here so so actually

what we want to do is while true so Wow

true we receive this data so I'll tab

that over hit save come back up here

rerun it and we get this welcome to this

server right ok so yeah that's what

happens when you buffer data but why do

we have to buffer it I mean like because

we could have gotten away with 1024 but

eventually someone's gonna try to send a

message that's bigger than that and

maybe on your chat lesea make a chat app

maybe you're care you set a character

limit and it becomes more challenging to

do more than that but there's almost

always going to be a reason why it's

gonna be better to buffer also I mean in

most cases you're gonna be like

below let's say certainly below like 10

megabyte buffer so if we're talking like

files and stuff you just you have to be

able to understand how that's gonna work

so then you start thinking well that's

not that hard man all we got to do is

something like this like I don't know

full underscore message and then you

just make that an empty string and then

here

you'll just say maybe something like

this like if if or actually what we'll

do is under here you would say you know

if message is less than or equal to 0

right or lend message rather so if

length of the message is less than or

equal to 0 we're gonna break otherwise

what we want to do is full message plus

equals MSG dot decode utf-8 who needs

Centex right like duh so then after this

we could print full message full massage

and then we come over here we'll break

this one maybe wow I just okay

yet jerk pie puh puh - 3.7 client dot

pie we try to connect nothing happens

but okay so what's going on here well

the problem is we still have this

connection from the server and the we'd

we don't actually know when the

connection is fully completed right

we're still actually the stream and the

connection is still exists so then you

come over here maybe and then I don't

know client socket closed I think is the

proper handling for that unfortunately I

don't think I can cancel this from

running either it's really annoying why

can't I break sockets in running PI -

3:7 server dot pi please don't tell me

that is a goose come over here rerun

client dot pi and then we see welcome to

the server and exits and so in this case

we were actually able to buffer all of

the data

and we didn't really need any fancy

logic we receive the message and we

buffered so what's the big idea well the

big idea is it's meant to be a stream

we're supposed to be accepting data and

a big stream and if server gets

something and it decides it wants to

send some information we're just simply

not going to receive it now we could

kind of treat this like a restful api or

something and like build data until the

client reconnects accepts the buffer and

then clears out with the client god or

something like that but that's not the

way that we typically will work with

sockets instead you're going to likely

send a bit of information in a header

and inform the client how much

information is to come so that's we're

going to be focusing focusing on in the

next video if you got questions comments

concerns whatever feel free to leave

those below quick shout out to my most

recent channel there we go couldn't

click on it channel members omni crux

miguel lutz lautaro alexander and mary

python I love your last name thank you

guys very much for your support it

allows me to keep doing what I love to

do so thank you guys so much and thank

you to everybody else eventually I'm

gonna figure out a decent way to rethink

people who have stuck around some people

are still with me for like well over six

months I forget exactly what we're at

right now for the max but that's

freaking awesome so anyways I'll figure

out a way at some points so anyways in

the next tutorial we'll talk about how

do you actually will buffer and keep the

stream open and all that and do some

more cool stuff with sockets otherwise

I'll see you guys in another video