Face recognition in real-time | with Opencv and Python

Summary

In this tutorial, Sergio demonstrates how to build facial recognition projects using Python libraries. He explains the difference between face detection and facial recognition, guides viewers through installing necessary libraries like OpenCV and face_recognition, and shows how to encode and compare images of faces. The tutorial progresses to real-time face recognition using a webcam, identifying individuals in live video feeds. Sergio also shares tips for improving accuracy and speed, and encourages viewers to engage with the content.

Takeaways

• 😀 The video is a tutorial by Sergio on building visual recognition projects, specifically focusing on facial recognition.
• 🔍 Sergio explains the difference between face detection (surrounding a face with a box) and facial recognition (identifying the person by name).
• 📚 Viewers are instructed to download certain files from Sergio's website, including 'main.pi' and 'simplefacerect.pi', to follow along with the tutorial.
• 🛠️ Two libraries are required for the project: 'opencv-python' and 'face_recognition', which are installed via the terminal using pip commands.
• 📷 The process involves loading and displaying images, converting them from BGR to RGB format, and encoding them for comparison using the 'face_recognition' library.
• 🔗 Sergio demonstrates how to compare two images to determine if they are the same person, using the 'compare_faces' function from the 'face_recognition' library.
• 🖼️ The tutorial includes a practical example of encoding and comparing images of well-known figures like Elon Musk and Messi.
• 💻 Sergio guides viewers through writing code for real-time facial recognition using a webcam, emphasizing the importance of the 'simplefacerect.pi' module.
• 📹 The real-time demonstration includes capturing video frames, detecting faces, and drawing rectangles around detected faces with the 'cv2.rectangle' function.
• 📝 Sergio discusses the importance of image titles for associating names with detected faces and displays names using the 'cv2.putText' function.
• 🔄 The video concludes with suggestions for improving the project, such as enhancing accuracy, speed, and detection capabilities, and encourages viewers to subscribe for more content.

Q & A

What is the main topic of the video?

The main topic of the video is teaching viewers how to build facial recognition projects using Python libraries such as OpenCV and face_recognition.

Who is the presenter of the video?

The presenter of the video is Sergio, who helps companies, students, and freelancers to build visual recognition projects.

What are the two types of face detection mentioned in the video?

The two types of face detection mentioned are phase detection, which involves surrounding any face with a box, and facial recognition, which includes identifying the person by name.

What are the two libraries that need to be installed for the project?

The two libraries that need to be installed are 'opencv-python' and 'face_recognition'.

What is the purpose of converting the image format from BGR to RGB?

The image format is converted from BGR to RGB because OpenCV uses BGR format by default, whereas the face_recognition library requires RGB format.

What is the role of the 'face_recognition' library in the project?

The 'face_recognition' library simplifies the steps of facial recognition by encoding images and comparing them to identify known faces in real-time.

How does the video demonstrate the facial recognition process?

The video demonstrates the facial recognition process by encoding images of known individuals, comparing them to other images or live video feeds, and identifying matches by name.

What is the importance of downloading the files from the presenter's website?

Downloading the files from the presenter's website is important because they contain necessary images and Python files like 'main.pi' and 'simplefacerect.pi' required for the project.

How can the facial recognition accuracy be improved?

The facial recognition accuracy can be improved by using a GPU for faster processing, adding more diverse datasets, and ensuring good lighting conditions for face detection.

What is the significance of the 'simplefacerect.pi' file in the project?

The 'simplefacerect.pi' file is a Python module that is not a library but a custom file needed for the project. It is used for detecting and drawing rectangles around faces in real-time video feeds.

How can viewers test the facial recognition project with their own images?

Viewers can test the facial recognition project by placing their own images in the specified folder, ensuring the images are named with the person's name for correct identification during the project execution.

Outlines

00:00

😀 Introduction to Facial Recognition Tutorial

In this introductory paragraph, Sergio welcomes viewers to a new video focused on teaching students, freelancers, and company employees how to build facial recognition projects efficiently. He differentiates between face detection, which merely outlines faces in a box, and facial recognition, which not only does that but also identifies individuals by name. Sergio emphasizes the necessity of downloading certain files from his website and installing two essential libraries, 'opencv-python' and 'face_recognition', to proceed with the project. He mentions that the tutorial will cover both image and video recognition, including real-time applications.

05:01

🔍 Setting Up the Facial Recognition Environment

This paragraph details the initial setup for facial recognition, including the installation of necessary libraries via the command prompt or terminal. Sergio demonstrates how to load an image using the 'cv2.imread' function and how to display it with 'cv2.imshow'. He also explains the importance of converting the image color format from BGR to RGB using 'cv2.cvtColor'. The paragraph concludes with the encoding of the image using the 'face_recognition.face_encodings' function, which is a crucial step for the algorithm to later compare and identify faces.

10:03

📚 Comparing Encoded Images for Facial Recognition

Sergio explains the process of comparing two encoded images to determine if they represent the same person. He uses the 'face_recognition.compare_faces' function to perform this comparison, which returns a boolean value. The paragraph includes a debugging moment where Sergio corrects a typo that led to an inaccurate comparison result. He stresses the high accuracy of the facial recognition algorithm, which is essential for reliable identification, and suggests that errors are likely due to incorrect image encoding.

15:04

📹 Implementing Real-Time Facial Recognition with Webcam

This paragraph introduces the transition from static image comparison to real-time facial recognition using a webcam. Sergio outlines the steps to capture video streams from the webcam, including initializing the camera with 'cv2.VideoCapture' and capturing frames in a loop. He also discusses the use of 'cv2.waitKey' to update the video feed in real-time and the importance of releasing the camera after use to prevent resource leaks.

20:04

👥 Loading and Encoding Known Faces for Identification

Sergio demonstrates how to load and encode known faces for identification in real-time video streams. He introduces a function to import and encode faces from a folder, which simplifies the process of adding multiple known individuals to the system. The paragraph explains how the 'face_recognition' library uses image file names as identifiers for each person's face, allowing the system to recognize and label faces in real-time.

25:06

🎨 Drawing Rectangles and Labeling Faces in Real-Time

The final paragraph focuses on the visualization aspect of the real-time facial recognition system. Sergio explains how to draw rectangles around detected faces using the coordinates provided by the 'face_recognition' library and how to display the names of the individuals above these rectangles. He discusses adjusting the thickness and color of the rectangles for better visibility and correcting any coordinate errors to ensure that the names are accurately positioned relative to the detected faces.

🚀 Conclusion and Future Improvements

In the concluding paragraph, Sergio wraps up the tutorial by highlighting the simplicity of the facial recognition project and suggesting potential improvements, such as enhancing accuracy, speeding up the process with a GPU, and expanding the dataset for more reliable recognition. He invites viewers to share their thoughts, experiences, and questions in the comments and encourages them to subscribe for more content on facial recognition and computer vision.

Keywords

💡Facial Recognition

Facial recognition is a technology that identifies or verifies the identity of a person using their facial features. In the video, it is the main theme, demonstrating how to build a project that not only detects faces within an image or video but also identifies the person by comparing the facial features with a database of known individuals. The script mentions 'facial recognition' as the process of surrounding each face with a box and naming the person, which is a key part of the tutorial.

💡Phase Detection

Phase detection, as mentioned in the script, is the initial step in facial recognition where a system identifies the presence of a face and outlines it with a box. It is a subset of facial recognition and is essential for the subsequent step of identifying the person. The script differentiates phase detection from facial recognition by stating that the former only detects faces, while the latter includes identification.

💡Image Database

An image database in the context of the video refers to a collection of images used to train or compare facial features for recognition. The script emphasizes the need for a database of images from specific people to perform accurate facial recognition, indicating that the system's accuracy is reliant on the quality and variety of images it has access to.

💡OpenCV

OpenCV, mentioned in the script, is a popular open-source computer vision and machine learning software library. It is used in the video for tasks such as loading images, capturing video from a webcam, and displaying frames. The script instructs viewers to install OpenCV using pip, indicating its importance in the facial recognition project.

💡Libraries

In the script, libraries refer to collections of code that can be imported into a program to provide additional functionality. The video mentions installing two libraries: 'opencv-python' and 'face_recognition', which are essential for the facial recognition project. These libraries simplify complex tasks and provide the necessary tools for processing and recognizing faces in images and videos.

💡Encoding

Encoding in the context of facial recognition is the process of converting an image of a face into a set of unique data points or a 'faceprint' that can be used for comparison. The script describes encoding as a necessary step before comparing faces, where the algorithm analyzes the facial features of known individuals to create a template for identification.

💡Real-time Processing

Real-time processing, as discussed in the script, is the ability of a system to process input data immediately as it is received, without any delay. The video demonstrates how to implement facial recognition in real-time using a webcam, which is crucial for applications like security systems or live event monitoring.

💡Face Comparison

Face comparison is the process of analyzing and comparing facial features between two images to determine if they are of the same person. The script describes using the 'face_recognition' library to compare encoded images and return a boolean value indicating whether the faces match or not, which is central to the identification aspect of facial recognition.

💡RGB Format

RGB stands for Red, Green, and Blue, which are the primary colors used in digital imaging. The script mentions converting the image format from BGR, which is the default in OpenCV, to RGB. This is important because different systems may use different color models, and ensuring consistency is crucial for accurate facial recognition.

💡Webcam

A webcam is a digital camera that captures images or videos and streams them over a network, such as the internet. In the script, the webcam is used as the source for real-time video input, allowing the facial recognition system to process live footage and identify or detect faces in real-time.

💡Accuracy

Accuracy in the context of facial recognition refers to the system's ability to correctly identify or verify individuals. The script mentions that the facial recognition algorithm used in the project has a high accuracy rate, which is critical for reliable identification and is a key selling point for such technologies.

Highlights

Introduction to building visual recognition projects for facial recognition.

Explanation of the difference between phase detection and facial recognition.

Importance of downloading specific files from the instructor's website for the project.

Instructions on installing necessary libraries: opencv-python and face_recognition.

Overview of the face_recognition library developed by Adam Geitgey.

Demonstration of loading and displaying an image using OpenCV.

Process of encoding an image for facial recognition using RGB format.

Explanation of the face encoding process using the face_recognition library.

Comparison of facial encodings to determine if two images are of the same person.

Testing the facial recognition algorithm with images of different people.

Achieving a high accuracy rate of 99% on certain datasets with the facial recognition algorithm.

Transition to real-time facial recognition using a webcam stream.

Importance of using the simplefacerect.py module for real-time face detection.

Instructions on setting up a webcam for real-time face detection.

Process of loading known face encodings from a folder for comparison.

Real-time detection of known faces and drawing rectangles around detected faces.

Displaying names above the detected faces in real-time using OpenCV.

Potential improvements to the project, such as accuracy and speed, and the use of GPUs.

Invitation for feedback and questions from the audience for future tutorial content.

Transcripts

[Music]

oh hi welcome to this new video my name

is sergio and i help company students

and freelancers to ease and efficiently

build visual recognition projects

as you can imagine and you saw that from

the preview we're going to see right now

how to make facial recognition

so let me explain a quick difference one

is phase detection where you just

surround any face with a box and then

there is second what we're going to see

today is facial recognition so it means

surrounding each phase with a box but

also telling exactly who that person is

so putting a name on that face of course

we start from some database of images

that we have from specific people we

will see that

on an image and also on a video and in

real time so let's start before we start

you need to know that you have to

download some files from my website i'm

i'm going to put the link down below and

you need to get these files so we have

images of course you can get your own

images i have some simple images main.pi

and then simplefacerect.pi

this is really important don't miss this

step because you're going to get an

error later

second we need to install two libraries

how to install the libraries we go

on the terminal

command prompt

cmd

and we see command prompt whether you

have the mac or linux is the same you

open the terminal and you need to type

pip install

opencv python is the first library

and anyway all the things that i'm

saying right now so all the comments

will be also

that all my blog posts if you just want

to copy and paste them pip install

opencv dash python is the first one you

press enter i'm not going to do that

because i have already the library then

pip install face underscore recognition

and then you press enter and we're going

to use this library which is called

phase recognition which is a great

library that simplifies all the steps uh

this is the github page i will not go

into details about this one i will leave

also the link for this because i want to

make things

very simple right now

and easy to follow so this is a great

library developed by adam get gay and

we'll use this one

let's start right now

so i'm going to start with an empty file

we're now going to import cv2 which is

the opencv library

and we're going to load an image let me

explain what's the idea now we load a

simple image for example i'm going to

load messy

one dot web ep we load this image and

later we are going to compare this image

with other images

so to check if we have messy somewhere

on these pictures

so if we have messy

the algorithm should tell us yes it's

messy also somewhere here so let's do

that we load first of all emg equals

to load the image we use cv2.in read

and here we need to put the path of the

image i have that on the same folder

messy1.webvp

and now let's display the image cv2 dot

in show

emg

and then emg

i forgot something in show

now

uh this will display the image but we

need a weight key vent to keep the image

on hold so cv2.way

key

zero so it is going to wait until we

press a key

i'm going to run this right away to make

sure that everything is correct and that

we don't get error it works this is

messy

now what is the first step for phase

recognition we need to encode

this image so the algorithm will encode

this image so that later it can compare

this the phase of message with other

faces

there is one step before so we need two

lines one is we need to convert the

image format from bgr to rgb opencv by

default uses the bgr format

so a blue and green and red and we need

red green and blue so we need just to

swap that so

rgb emg equals cv2 dot cvt color so

convert color what do we want to convert

we want to convert tmg then cv2.color

underscore bgr2rgb

we're going to convert the bjr format to

rgb

and now we're going to encode the image

so we'll say

emg encoding

equals

to this we need the library phase

recognition we're going to import

face recognition

and then we'll say face recognition dot

phase encodings and we are going to put

here

rgb emg

emg

and then we are going to say

zero so

uh probably because this loads multiple

image you can load multiple images we're

going to use just zero here as an index

we have the first image encoded so let's

uh restart these i'm going to just

restart to make sure that there is no

type on it i don't get any error i

i run this quite often to

to avoid to have a lot of code and then

it's a mess if you get some error

and of course it's a bit slow when you

have the encoding because it needs to

analyze the image the bigger the image

is lower the encoding is and this image

is quite big

it works let's go to the next step we're

going to do exactly the same process for

another image

i'm going to copy and paste and we just

are going to change

we will be changing only the variables

name so we have

let's load the second image we're going

to load another person let's load elon

musk

so i'm going to load

em g2 see if it's not in read i have

elon musk on another folder images

and then elon musk

by the way everything that i am typing

here will be uh on the

on the link in the description there

will be a blog all the code you can

download everything that i have

and also the test images that i am using

so images elo mask gpg so rgb and g2 mg

encoding two phase recognition rgb emg

too okay that was quite simple

and now let's display also emg2

so now i'm going fast just with this

example but later we will use this in

real time it will be the code it will be

much easier i have another file which

have this really simplified so don't

worry if it's hard for you to follow

there is not much really to understand

it's just a few lines

now

now we need to make the comparison so

what is the idea we have emg encoding

one

and we have emg encoding two

we want to know

if emg one and in emg two there is the

same person

so let's do that

result

equals

phase recognition

dot

compare faces

and right here we're going to put first

emg encoding

and then emg

encoding

2. and let's print the result

let's run this one this might take a few

seconds to run so what are we doing

we're comparing this image elon musk

with this image of messi

and the code should tell us true or

false if true if is the same person or

false

if there are different

people and the result is true

which is a bit awkward uh okay i just

realized that there there is some

uh some typo here so i'm loading i'm

loading emg one again so emg two of

course they're not the same person so i

was surprised to be honest so let's run

this again

by the way this algorithm algorithm is

really precise

this worked great it has like 99

accuracy on some data set so if you are

getting an error is most likely

that there is some type of mode then

it's not working well somehow

so we have a result now it's false so

we're comparing this one

with

this one and of course they're not the

same person

and we see result false now let's test

also with

some other image

so we're going to compare

messi with

this one jeff bezos

um

okay we're going to compare messi wii

jeff bezos

jeff bezos.jpg

so of course now we should get again

false

it takes a few seconds it's of course a

bit slow

uh you might even uh there is some it's

more complex in simulation but you might

get this working with a graphic card

media graphic card and it will be much

faster again we have result false i'm

going to make the last testing so we're

going to compare messi with another

picture of messi so now we have messi

jeff bezos are they the same person

exactly they are not

so now let's load messy and mess but of

course a different picture of him it

wouldn't make any sense with the same

picture

messy

yvp

and now we should get true

and we get results true so we are

comparing this one

with

this one

so despite

i

is a different environment

the despite also the light and the color

it works great we get result true

i can say that this one honestly was

quite simple to achieve

now it will come the fun part of the

project where we're going to have a

simple code to run this in real time and

i will be using my webcam so let's do

that i'm going to

i'm going to

store this somewhere so that i will put

later on the blog

so you will get also some files with

this one while i'm going to

start this from scratch but in real time

from a video

so we import cv2 import phase

recognition and now we import

of from

simple

face rect we are going to import simple

face wreck

keep in mind this is not a library this

is a python file that you need to

download from a website and put together

on the same folder with the main file

otherwise you get an error so let's run

this one

how do we work with a webcam first of

all we're going to get the stream from

the webcam and later we will do all the

rest so first of all let's in

let's load the camera cap equal cv 2.

video

capture

and then we put the index of the camera

so load camera

uh zero so this is say load the first

webcam if you have multiple webcams you

need to put one to load the second two

through the third and so on and i have

three cameras so i need to load the

latest one so i'm going to use index two

most likely we need to use index zero if

you

are not using the webcam you have just

one

while

true now we're going to get the stream

in real time from real time we're going

to get frame after frame

red frame equals cap dot read

so red is true or false true with the

frame false we don't have the frame and

frame is the frame

and now let's show the frame cv two dot

im show

frame and then frame

and now again as before a weight key

event but this time slightly different

key equals cv2 dot weight key

one before we have we had weight key 0

which was freezing the frame

until we will press some key now we say

1 so weights 1 millisecond and go to the

next frame so that we have the video in

real time or like the camera real time

but if we want to quit we can do that if

the key is 27 which corresponds to the s

key on the keyboard this breaks the loop

finally we need to release the camera

cap dot release

and see if it so does destroy all

windows to close all the windows

let's run this one so i'm going to run

main dot pi

i'll stop every run

if everything is working correctly we

should see the live stream of the camera

so we have the camera loading

uh now it's not doing anything just

displaying the camera

let's now

use the library face simple phase rect

not the library

the python file is simple the module

simple phase rect

to compare the faces

how do we do this

first of all we need to load the

encoding faces

how do we load that

so also let me explain i'm putting too

many things and not being clear

we load the encoding faces it means

these are the known faces so later we

will look wherever we have for these

people elon musk jeff bezos messi

reynolds and me

so each time we see one of these

we will get their name

uh when we see some other phase we will

get unknown face so that's

that's the goal

uh the idea

so let's encode all these faces together

so for this i created a simple function

which imports all the faces that are on

a folder

and we need to do this encode

uh faces

from a folder

first of all let's initialize the simple

face rec module

f of simple phase drag let's call

sf

uh sfr equals

a simple face rack so i'm going to

initialize this

then let's load

uh i will put this before loading the

camera

like this

now let's load let's encode all the

faces all the known faces so we will

take the path where they are so i will

put them just on images

and we do this s f

r dot

load encoding images

and right here we need to put the path

of the folder so we have them just on

images

so this is the path where all the images

are

to make sure and to show how this works

i'm going to run this

if everything is working correctly you

will see an output that five

images were found found because i have

five images

of course you can put how many images

you want in that specific folder

so you can put your own images

how many you want whatever you want just

put them here and they will be

automatically added to the encoding

now let's go to the next step once we

have five encoding images found

just after we get

the frame

we are going to

detect if any of the names

are there

detect

faces

so we use

sfr

dot

detect known faces

and then let's uh from the frame

from the frame

what are we going to get in exchange

we're going to get the phase rectangle

for each phase so we're going to get uh

face

locations

face locations

and then we're going to get

the names face names

uh by the way if we're wondering how are

the names loaded if we're not putting

the name anywhere

uh

they will be taken from the image title

so for example this is the image of elon

musk and it's called elon musk dot jpg

jeffbezos.jpg so the name that you put

on this image is the name that you will

see displayed later on the screen

phase locations so let's now first of

all display the face locations

for

uh

face

lock

and for

name

in

a zip so we're going to extract these

together phase locations

and face names

let's first of all print phase location

face lock

and let's now let's now run this one so

see what we get

so now the idea is that we're going to

get phase location so of course the

coordinates to

to display correctly the face

uh in real time

on the frame

and right here you see that i have a lot

of outputs

and even if i move my face you can see

that these numbers somehow are changing

all of them okay now all of them

oh here we have the location of the face

so what i am going to do right now is

i'm going to

take

uh these values i'm going to extract

these values and draw a rectangle

surrounding the face so now i don't have

anything we should draw the rectangle

using these coordinates

uh what are these coordinates there are

four values

the first two is the top left point and

then we have the right bottom point so

top left like this

top left

right bottom and we have the rectangle

of the face

so

we have

top

left

top left

bottom

right

equals

a face lock

zero

so i'm going to extract them one by one

this way

phase one

face two

and face three

so i simply assign to the first value is

top is the y the second value is the x

left

then we have bottom and so on

now let's draw a rectangle for this one

cv2.e

we're going to draw the rectangle where

we're going to draw the rectangle on the

frame so frame

uh

then we have x and y so we have left

and top

top left point

then we have 0.2

uh probably it's better just to call

this with the coordinates will be less

confusing so instead of top this will be

y1

instead of left would be x1

instead of bottom would be y2

instead of right will be x2

so we have left which would be x1 top

will be y1 so this is just basic

geometry with the coordinates then we

have x2

y2

and now the color let's make this

somewhat red so zero blue

the color go from zero to 20 55 we give

zero blue zero for green and let's give

200 of red so that it's red but not too

bright

thickness let's make this two pixels

thick

if everything is correctly is correct

we're going to see a rectangle

surrounding the face

uh

we have the rectangle so red rectangular

surrounding the face

let's now take the name

so the rectangle is working also i will

make maybe a little bit thicker let's

say four pixels

and let's now take also so we have the

name associated to each phase location

let's display that

cv2.put

text

let's put the text on the

we're on for on the frame

or the position of the text let's say

we're going to put the text on the top

of the rectangle

so we will do

x

1

y

1

minus minus 10 px so that it's not

overlapping with the line of the

rectangle so that's why i'm going to say

minus 10 pixels

uh i forgot the text so first we need

the text frame then

um

name

here we show the name

so the text font face now the font c2

dot font

underscore hair shall it doesn't really

matter with the font that we choose

and the size of the text let's say one

color of the text let's make this

completely black

zero

zero zero

and thickness of the text too

and now let's run this one

oh and here we have this in real time

you see

there is my name so it's already a good

start only i believe i mixed some

somewhat the coordinates

so my name was supposed to be right here

on the top

left of the rectangle so i i messed with

the coordinates so i'm going to fix that

and also let's i'm going to use

to make the the name

better looking right here so let's

quickly do that

so i mix the coordinates so we have top

right so this is x two

and this is x

one

uh probably left top right and then

bottom and left

i don't know why they choose to use this

coordinates format

for the face uh face direct library this

it's not common but anyway this is how

you do it

i was saying

okay this is uh let's make the test text

look better um

okay i don't want to take much time with

this video so

i want that it's it will be more clear

so i will say also for this one red

around 200

so ideally was thinking about putting a

red box and white color of the text

but i might do add that code later i

want to make this as simple as possible

to get the important information so how

to make the face wrecked

uh how to get the face then all the rest

uh changing the

the colors uh the the shapes uh

it's all about simple operations with

opencv which we won't make any sense to

to do right now

oh it's working right now so you see now

there is my name on it

but i have to prove you that

it's working well so i have the phone

and i'm going to show the pictures

okay even

let me uh increase the lightning so

we will not have any problem okay you

see ryan reynolds

again

sometimes if it doesn't recognize the

person says unknown but it's

it works quite well uh so far

messy

can we get messy

messiah it's hard

because there is the reflection of the

lightning so elon musk

we get that

so the hardest right here is face

detection because of the face is really

small and the lightning but normally

will work much better

and again jeff bezos

so this is all for this tutorial i hope

you enjoyed this tutorial of course

there there are improvements that you

could make for example

you could improve the accuracy you could

improve the speed

installing this with the gpu you could

improve the uh detection in adding

multiple data sets necessarily of adding

just one single person

for encoding you can add multiple person

for example for messy multiple pictures

of messi

and so on

let me know what you think about this

project in the comment let me know if

you try this

and what projects are you working on and

questions because i will take the

questions also to

make new videos and to improve the

projects

regarding face recognition

i suggest that you

subscribe to get notified because i will

be releasing a lot of new content

regarding

phase recognition computer vision

and so on so this is all for now see you

in the next video