Driver Drowsiness Detection System using OpenCV | Graduation final year Project with Source Code

Summary

This video script guides viewers through building a project that detects eye blinks using OpenCV and dlib. It covers installing necessary modules, initializing face and landmark detectors, calculating eye aspect ratios, and playing an alarm when a blink is detected. The tutorial also demonstrates drawing contours and convex hulls, and concludes with playing music using the pygame module.

Takeaways

• 😀 The video script describes the process of building an application that detects eye blinks and plays an alarm when a blink is detected.
• 🛠️ The project requires the installation of several Python modules including 'opencv-python', 'RPi.GPIO', 'imutils', 'dlib', and 'cmake'.
• 🔍 The 'dlib' library is used for face detection and to detect 68 facial landmarks, which are essential for tracking eye movements.
• 📚 The script introduces the concept of the Eye Aspect Ratio (EAR), which is calculated using the coordinates of the facial landmarks around the eyes to detect blinks.
• 📉 The EAR value remains constant and drops significantly when the eyes are closed, making it a reliable metric for detecting eye blinks.
• 👁️ The script details how to calculate the EAR for both the left and right eyes and then averages them to determine if a blink has occurred.
• 🔶 The convex hull, which is the smallest convex polygon that can enclose a set of points, is used to simplify the shape of the eyes for contour analysis.
• 🔲 The script includes a method to draw the convex hull and contours on the video frames to visualize the eye shapes.
• ⏰ An alarm is triggered when the EAR is below a certain threshold for a specified number of frames, indicating a sustained eye closure.
• 📈 The script mentions adjusting the threshold value based on the user's setup and the frame rate of the webcam.
• 🎶 Towards the end, the script adds functionality to play an alarm sound using the 'pygame' module when the eye blink is detected.

Q & A

What is the purpose of the application being discussed in the script?

The purpose of the application is to detect eye blinks using facial landmarks and play an alarm when a blink is detected.

Which modules are required to be installed for this project?

The required modules are opencv-python, imutils, dlib, and pygame.

Why is the frontal face detector used instead of the traditional OpenCV's Haar Cascade detector?

The frontal face detector is used because it is faster and more effective compared to the traditional Haar Cascade detector.

What is the significance of the EAR (Eye Aspect Ratio) in this project?

EAR is used to detect eye blinks. The value of EAR remains constant except when the eyes are closed, making it an effective way to detect eye blinks.

How is the EAR calculated in the script?

EAR is calculated using the distances between specific facial landmarks of the eyes. It is the sum of the vertical distances divided by twice the horizontal distance.

What are the facial landmarks used for in this project?

Facial landmarks are used to detect 68 points on the face, which include eyes, eyebrows, jawline, lips, nose, etc. These landmarks are used as reference points for detecting eye blinks.

What is the role of the convex hull in this project?

The convex hull is used to determine the minimum boundary that can completely enclose the object, in this case, the eyes. It helps in drawing the outline around the eyes.

How does the script handle the detection of eye blinks?

The script calculates the EAR for both the left and right eyes, averages them, and checks if it is less than a certain threshold for a specific number of frames. If it is, an alarm is played.

What is the threshold value used for detecting eye blinks and how can it be adjusted?

The threshold value is initially set to 0.25, but it can be adjusted between 0.25 to 0.5 depending on the user's setup and the frame rate of the video or webcam.

How is the music played in the application?

Music is played using the pygame module. The music file is loaded and played using the mixer.music.play function.

Outlines

00:00

🛠️ Project Setup and Module Installation

This paragraph introduces the project by explaining the need to set up an application that triggers an alarm. The speaker guides viewers on how to start the project by opening the editor and installing necessary modules using pip install commands for 'opencv-python', 'imutils', and 'dlib'. They also mention potential issues with the 'RP system' and provide steps to install 'Cmake' and 'delete'. The speaker emphasizes the importance of downloading specific files and setting up the environment correctly before beginning the coding process.

05:00

🔍 Face Detection and Landmark Detection

The speaker delves into the technical aspects of the project, focusing on face and landmark detection. They explain the use of the frontal face detector from the 'dlib' library, which is more efficient than traditional OpenCV methods. The paragraph also covers the detection of 68 facial landmarks, which are crucial for various applications like Snapchat filters. The speaker introduces a method to calculate the Eye Aspect Ratio (EAR) using these landmarks, which is a key step in detecting eye blinks. The EAR calculation is detailed, explaining the significance of horizontal and vertical distances between specific points on the face.

10:03

👁️‍🗨️ Eye Blink Detection Algorithm

This paragraph discusses the algorithm used for detecting eye blinks. The speaker explains how the EAR is calculated for both the left and right eyes and then averaged to get the final EAR. They mention the importance of the convex hull in determining the minimum boundary of an object, which is used in this context to detect the eye's shape. The speaker also describes how to draw contours and convex hulls on the image using OpenCV functions. The paragraph concludes with the implementation of a condition to trigger an alarm when the EAR falls below a certain threshold, indicating an eye blink.

15:09

🚨 Alert System and Text Display

The speaker continues with the implementation of the alert system, which is triggered when the EAR is below the threshold for a specified number of frames. They introduce a variable 'flag' to count the number of frames where the EAR is below the threshold. The paragraph also covers how to display warning messages on the frame using 'cv2.putText'. The speaker provides details on the parameters for the text display, such as font type, color, and thickness. They also discuss the use of a variable 'frame_check' to ensure the warning message is displayed for a continuous number of frames.

20:12

🎵 Music Integration and Final Touches

In the final paragraph, the speaker adds music to the project using the 'pygame.mixer' module. They guide viewers on how to install the module and load a music file. The speaker demonstrates how to play the music when an eye blink is detected, enhancing the alert system. They also discuss resizing the frame and terminating the loop when a specific key is pressed. The speaker concludes the project by encouraging viewers to like, share, and subscribe, and provides links for further information and source code.

Keywords

💡Alarm

An alarm in the context of this video refers to a warning signal that is triggered when the system detects an eye blink. The video discusses building an application that plays an alarm when it detects a certain condition, specifically an eye blink. This is part of a larger theme of using computer vision to detect and respond to specific actions or events.

💡OpenCV

OpenCV, or Open Source Computer Vision Library, is a library of programming functions mainly aimed at real-time computer vision. In the video, OpenCV is used to handle video capture and image processing tasks, such as reading frames from a video source and displaying them. It is a crucial tool in building the project discussed in the script.

💡dlib

dlib is a toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real-world problems. In the video, dlib is used for face detection and landmark detection, which are essential for determining the position and shape of facial features like eyes, eyebrows, and lips.

💡Landmarks

In the context of facial recognition and image processing, landmarks refer to specific points on a face that are used to detect and track facial features. The video mentions detecting 68 facial landmarks, which can be used to analyze and understand the position and movement of different parts of the face, such as the eyes and mouth.

💡EAR (Eye Aspect Ratio)

The Eye Aspect Ratio (EAR) is a measure used in facial recognition and emotion detection to determine the openness of an eye. In the video, EAR is calculated using the positions of specific facial landmarks around the eyes. A sudden drop in EAR can indicate an eye blink, which is the key event the application is designed to detect.

💡Convex Hull

A convex hull is the smallest convex set that contains a given set of points. In the video, the convex hull is used to define the boundary of the eyes in the image. It helps in visualizing the shape and extent of the eyes, which is crucial for accurately detecting eye blinks.

💡Contours

Contours in image processing are curves that join all the points lying on the boundary of an object. In the video, contours are used to outline the convex hull of the eyes, providing a visual representation of the eye's shape and helping to determine when an eye blink occurs.

💡Threshold

In the context of the video, a threshold is a predefined value used to determine when an action should be taken. Specifically, the EAR must fall below a certain threshold for a certain number of frames to trigger the alarm. This is a critical parameter in the application for detecting eye blinks accurately.

💡Imutils

Imutils is a series of convenience functions to make basic image processing functions easier with OpenCV. In the video, Imutils is used for tasks like resizing images and converting color spaces, which are common operations in computer vision projects.

💡Pi game modules

Pi game modules refer to a Python library used for playing sounds and music. In the video, the library is used to play an alarm sound when an eye blink is detected. This adds an auditory component to the application, alerting the user to the detected event.

💡Video Capture

Video capture refers to the process of recording video from a camera or other video source. In the script, video capture is initiated using OpenCV's `cv2.VideoCapture` function to read frames from a video source, which is fundamental for processing and analyzing the video content.

Highlights

Introduction to building an application that plays an alarm when detecting eye closure.

Installation of required modules using pip, including opencv-python and imutils.

Potential issues with the RP system and automatic installation of necessary libraries.

Importing and utilizing the imutils library for basic image processing functions.

Setting up the video capture using OpenCV and reading the frames.

Displaying the image on a window using OpenCV's imshow method.

Utilizing the waitKey function for managing window display time.

Initialization of the frontal face detector and explanation of its advantages over traditional methods.

Introduction to the use of dlib's HOG and linear SVM phase detector for face detection.

Importance of detecting 68 facial landmarks for various applications like Snapchat filters.

Explanation of calculating the eye aspect ratio (EAR) for eye blink detection.

Defining a function to calculate the EAR using the coordinates of facial landmarks.

Use of convex hull to determine the minimum boundary of an object, such as an eye.

Drawing contours to visualize the boundary of the detected eyes.

Condition for detecting eye closure based on EAR and frame count.

Displaying alert messages on the frame when eye closure is detected.

Resizing the frame for better visualization using imutils' resize function.

Termination condition for the application using a keyboard interrupt.

Integration of the pygame.mixer module for playing an alarm sound.

Final demonstration of the project with eye closure detection and alarm sound.

Conclusion and invitation for feedback, sharing, and subscription to the channel.

Transcripts

so it will play an alarm so let us see the whole working of it

foreign

[Music]

foreign [Music]

so that is the whole functionality of this application now without wasting any

time let us start building this project

now here we are on our editor and before starting the project I just want you

guys to download these two files download

firstly let us install the important modules your required as projects

by writing pip install opencv python

now midi system may already installed as so it says requirement already satisfied

but RP system May throw us a Time Lego automatically installed

now we need to install imutil showcase series of functions

for the basic image processing functions such as translation rotation resizing

skeletization detecting edges and sorting Contours now what those are is

simply install it by writing pip install imutils

foreign

module so follow alongside me go to your browser and simply type

visual Studios

KY official websites download section say you can download the community

version which is available for free once downloading is complete simply install

it but installation make sure to check desktop development with C plus and

because your body packages so it will take a while to install once it is

installed again come back to your command prompted Yahoo installs C make

by writing pip install cmic

and once it is done you can install delete by writing pip install delete

and this is the simplest way to install it

now sorry modules install logic so let us simply import them and subscribe I'm

importing CV2 now my important

and from imutils foreign

now we will import dlib and from sci-fi dot specials I'll import

distance

my bad this is sci-fi now once this is done I'll write cap

equals to CV2 dot video capture 0.

foreign

true

RIT comma frame equals to cap dot read

now yeah I'm writing cap dot read Now read me

built-in function which return two values where first one is a Boolean

variable that Returns the value true or false based on if the frame is available

or not just comments and second is

uh frame and frame

now I am showbu CV2 guide built-in method which is used to display image on

our window here though parameters except where first one is the window name which

means name of the window which is the image to save it display and in our

case image is stored in frame variables now let us write cv2. wait key one

now weight cubic built-in function which allows us to display a window for our

given millisecond or until any case pressed here time go in millisecond as

parameter except and it waits for the given time to destroy the

windows so it will wait until any key is pressed so this is the standard boiler

template of the opencv Run Guru so you can see it starts recording

now let us start our actual coding by initializing our frontal face

detector and for that many amperage variable here I detect and

yeah but I'm writing delete dot get frontal face detector

now your frontal face detector delivery built-in function and you must be

wondering why we are using frontal face detector instead of traditional opencvs

hard Cascade detector and guys the reason for that is that daily frontal

face detector is quite fast and effective as compared to that one it

does not accept any parameter and Champion so it Returns the sum of

pre-trained hog and linear svm phase detector which is included in the dlib

library where hog stands for histogram of oriented gradients

now let us take one more variable in predict

and yeah

name of this file hook up in the description box to

download

I'll paste it here with DOT dat extension

now what it does is it will detect 68 landmarks from our face these landmarks

are some points on our face which includes eyes eyebrows jawline lips uh

nose baby etc etc in general economic Snapchat where we have different filters

available okay so all these filters use landmarks as a reference point

now if you will see this image so you can see the eyes with six points

detector this is

point zero and point three these are known as long points or horizontal

points which are point one and five and point

two and four these are known as short short points or also vertical points and

ham login points to calculate ear that is I aspect ratio now both is simple or

effective method has to calculate the relation between horizontal and vertical

measurements of an eye this means we don't even require anything like image

processing now there is an equation to calculate the eer using these six points

which is e a r equals to sum of vertical distance upon 2 into horizontal distance

of I yeah per sum of vertical distance means the equilibrium distance between

point one and five and between points 2 and 4 and horizontal distance means the

equilibrium distance between point zero and three so let us Define a function

for it and here I'm writing Define

I aspect ratio which exception parameter I

for you make variable around

between point one and five

now let me copy this and I'll paste it here

let this be Point p ages

distance between point two and four

and let this be Point C is

distance between point zero and three

now equilibrium distance to calculate currently I have taken this built-in

function but I gotta block manually calculated you can do it manually by

using the formula equilibrium distance equals to under root of X2 minus x 1 cos

square plus 2 I 2 minus y 1 square now let us calculate e a r which is sum of

vertical distance that is a and b divided by 2 into horizontal distance

that is point C now we will simply return this ear

now equation arrives with I had that why is it necessary to calculate ear what is

so magical about this ear and the answer is value of eer remains constant so B I

opened here but it suddenly drops here and that is because I close on a

value of a plus b decreases now there is a graph for this

activity that the value of er was constant except one point

or you can say I close within hence we can say this is the most simplest and

most effective way to detect an eye blink now because we have two eyes

that is for left and right eye individually

to calculate the final ear and then is final eer come look check if it is less

than a certain threshold value for a certain period of time or we can say for

certain frames then we will play an alarm for a threshold value can be

different for different users merely it worked between 0.25 to 0.3 but in some

case is it can go to 0.4 or 0.5 as well it all depends on the FPS that is frame

per second of your video webcam now average come look bad May calculate but

before that let us come here and yeah permac

variable declared around gray and here I'll write CV2 dot convert color

off frame

CV2 dot color underscore BGR to Gray

now yeah

now let us detect the faces from this grayscale frames and for that I'll write

detect gray comma zero

now Java detect is the detector this is gray is this grayscale frames and zero

is an index now initialize in a variable name subjects

now let us take a for Loop for subject in subjects

now detector let us detect the landmarks on it and for that I'm writing predict

in Gray subjects

now yapper predict is this predictor Grays is grayscale frames and subjects

is the detector output now is in a variable name shape

now let us write shape equals to face underscore utils

dot shape to NP

shape now here we are converting shapes to a

list of X Y coordinates now let us go back up and yeah but let

us take two points I'll start and I'll end

and yamara right face underscore utils dot facial

underscore landmark underscore 68 underscore ID access

puff left eye

now these are the eye landmarks of the left eye

let us copy it and let us paste it here

to detect the eye line Mark of right eye as well

R start comma R end and here right underscore I

now let's go back down and yeah

to extract the left and right eye so let us write left eye

equals to shape I'll start

and L end

let me copy this and let me do the same for right eye

now once something left and right I could detect earlier let us calculate

the individual ear by using this I detect ratio function and for that my

upper is

foreign

now let us copy this and let us do the same for write I as

well

that actually will be the average of this left and right ear so let us

calculate it by writing e a r equals to left ear plus right ear

whole divided by 2. now once this is done we will find

something called convex Hull a convex Hull is the minimum boundary of an

object that can completely enclose or wrap the object for example let us

consider this rectangle this may we have two borders one in black color and one

in red color and the convex cell of this shape will be the black border because

that is the minimum boundary Choice object to completely wrap current

so let us calculate its convex cell by using cb2's built-in convex cell method

and our left eye calculate so we'll write left eye now is

similarly let us copy this and let us do it for the right eye as well

yeah but we'll write write and this com looks occurring in a

variable name right now let us detect Contours and what is this Contour it is

a curve that will join all the points lying on the boundary of that shape is

example may you can see that the red points are the convex cells while the

green line is the Contour I still remember advanced mathematics class we

manage topic importance

because the online tutor that taught me never mentioned any of the real world

application of it how important it is in computer vision

is example maybe you can see that this red points are the convex cell while the

screen line is the Contour now let us draw it using cv2's built-in draw

Contour functions okay

now this function accepts five arguments where first one is the image which is

second is the Contour that is the indicated contour and yeah instead of

that so we'll write left eye hub for Left Eye

third is the Contour index which represents the pixel coordinates that

are listed in the Contour is say we can indicate the index of the exact point

just comments example but this point will be index one

this will be index two this will be index three and so on but because some

SRA control points so we'll pass any negative value so I'm writing minus one

now fourth argument is the color which will indicate the color of the line now

my upper rock around green line so I'm writing 0 to 550

and final one is the thickness of the line so this is how we will draw the

control on the left eye now let's do the same for right eye and for that manage

to copy here and I'm pasting it here and jumper instead of left eye here I'll

write right I have now these four line of code is just

returned to draw this green outline on the image now yamper my condition if e a

r less than thresh now what thresh is

0.25 works perfectly fine with my system but it might not work perfectly with

your setup so block threshold different values may be 0.3 0.4 or 0.5 could be

tricer so here I'm declining threshold equals to 0.25

[Music]

that is for every frame the value of flag will be incremented and now we'll

print it

warning message

but it will create a small problem and that is

I'll take one more variable which is frame underscore check

now if the value of flag is greater than or equals to this value

then only I'm a warning message display with the help of this conditional

statement

irrespect ratio continuously 20 frames

and with the help of this system

now let us put a warning message here CV2 dot put text we want our text on

frame with the message as alert

third parameter is the coordinates of X and Y axis so let it be 10 30

fourth is the font this may have a message display

so I'm taking font Hershey simplex

fifth is the thickness of the font six is the color of the font

and last is the thickness of line now again value threshold values

now let us run it once

so it gives an attribute error uh okay

this is landmarks guys similarly let us do it here

let us run it again

okay and uh my bad uh here this is orange

let us run it once again

okay generates one more error ah

again my bad that was a typo let us run it once again

and this time it runs successfully so this is the flag value

now let me close my eyes

[Music] and here it is displaying an alert

message

let us do one more thing

let us display one more message this time on Y axis as 325

and yeah let us resize our frame

using imutil's free size function

let us set its width to 450 now is infinite gluco terminate let us

take an if condition that is if cb2.8 q1 and 0xff

equals to equals to ordq

break now we'll destroy all windows

and we will release the cap

and with that I may almost put a project curly I accept one thing and that is

playing music now let's do it as well and for that my pie game modules a mixer

functions

and before importing make sure to install it

and for that come to your command prompt and right with install Pi games

now buy game we made a system we already installed so it says requirement already

satisfied but RK system is automatically installed

now Pi game modules

you can even refer that video after the competition of this one usually

Implement description box now let us initialize our mixer module

here

and let us also load our music file using mixer dot music

dot load and here we will write the path name as

the extension of file and because of the music file already have my project

folder May store here so we don't need to mention the path and look simply

namely and the extension as well now once this is done

and the movement warning message

using mixers already built-in music function that is mixer dot music dot

play let us save it let us run it once again

let me close my eyes

[Music] and here you can see it started playing

the morning music so that's it from this project guys I

hope you like this one and if you do kindly give it a thumbs up share it with

your friends and forgot not to subscribe to our Channel if you have any doubt

related to any of our project you can contact me on my Facebook and Instagram

handles Link in the description box also check description box for the source

code of this project and link to our other YouTube videos as well

this is actually signing off