Driver Drowsiness Detection System using OpenCV | Graduation final year Project with Source Code
Summary
TLDRThis video script guides viewers through building a project that detects eye blinks using OpenCV and dlib. It covers installing necessary modules, initializing face and landmark detectors, calculating eye aspect ratios, and playing an alarm when a blink is detected. The tutorial also demonstrates drawing contours and convex hulls, and concludes with playing music using the pygame module.
Takeaways
- π The video script describes the process of building an application that detects eye blinks and plays an alarm when a blink is detected.
- π οΈ The project requires the installation of several Python modules including 'opencv-python', 'RPi.GPIO', 'imutils', 'dlib', and 'cmake'.
- π The 'dlib' library is used for face detection and to detect 68 facial landmarks, which are essential for tracking eye movements.
- π The script introduces the concept of the Eye Aspect Ratio (EAR), which is calculated using the coordinates of the facial landmarks around the eyes to detect blinks.
- π The EAR value remains constant and drops significantly when the eyes are closed, making it a reliable metric for detecting eye blinks.
- ποΈ The script details how to calculate the EAR for both the left and right eyes and then averages them to determine if a blink has occurred.
- πΆ The convex hull, which is the smallest convex polygon that can enclose a set of points, is used to simplify the shape of the eyes for contour analysis.
- π² The script includes a method to draw the convex hull and contours on the video frames to visualize the eye shapes.
- β° An alarm is triggered when the EAR is below a certain threshold for a specified number of frames, indicating a sustained eye closure.
- π The script mentions adjusting the threshold value based on the user's setup and the frame rate of the webcam.
- πΆ Towards the end, the script adds functionality to play an alarm sound using the 'pygame' module when the eye blink is detected.
Q & A
What is the purpose of the application being discussed in the script?
-The purpose of the application is to detect eye blinks using facial landmarks and play an alarm when a blink is detected.
Which modules are required to be installed for this project?
-The required modules are opencv-python, imutils, dlib, and pygame.
Why is the frontal face detector used instead of the traditional OpenCV's Haar Cascade detector?
-The frontal face detector is used because it is faster and more effective compared to the traditional Haar Cascade detector.
What is the significance of the EAR (Eye Aspect Ratio) in this project?
-EAR is used to detect eye blinks. The value of EAR remains constant except when the eyes are closed, making it an effective way to detect eye blinks.
How is the EAR calculated in the script?
-EAR is calculated using the distances between specific facial landmarks of the eyes. It is the sum of the vertical distances divided by twice the horizontal distance.
What are the facial landmarks used for in this project?
-Facial landmarks are used to detect 68 points on the face, which include eyes, eyebrows, jawline, lips, nose, etc. These landmarks are used as reference points for detecting eye blinks.
What is the role of the convex hull in this project?
-The convex hull is used to determine the minimum boundary that can completely enclose the object, in this case, the eyes. It helps in drawing the outline around the eyes.
How does the script handle the detection of eye blinks?
-The script calculates the EAR for both the left and right eyes, averages them, and checks if it is less than a certain threshold for a specific number of frames. If it is, an alarm is played.
What is the threshold value used for detecting eye blinks and how can it be adjusted?
-The threshold value is initially set to 0.25, but it can be adjusted between 0.25 to 0.5 depending on the user's setup and the frame rate of the video or webcam.
How is the music played in the application?
-Music is played using the pygame module. The music file is loaded and played using the mixer.music.play function.
Outlines
π οΈ Project Setup and Module Installation
This paragraph introduces the project by explaining the need to set up an application that triggers an alarm. The speaker guides viewers on how to start the project by opening the editor and installing necessary modules using pip install commands for 'opencv-python', 'imutils', and 'dlib'. They also mention potential issues with the 'RP system' and provide steps to install 'Cmake' and 'delete'. The speaker emphasizes the importance of downloading specific files and setting up the environment correctly before beginning the coding process.
π Face Detection and Landmark Detection
The speaker delves into the technical aspects of the project, focusing on face and landmark detection. They explain the use of the frontal face detector from the 'dlib' library, which is more efficient than traditional OpenCV methods. The paragraph also covers the detection of 68 facial landmarks, which are crucial for various applications like Snapchat filters. The speaker introduces a method to calculate the Eye Aspect Ratio (EAR) using these landmarks, which is a key step in detecting eye blinks. The EAR calculation is detailed, explaining the significance of horizontal and vertical distances between specific points on the face.
ποΈβπ¨οΈ Eye Blink Detection Algorithm
This paragraph discusses the algorithm used for detecting eye blinks. The speaker explains how the EAR is calculated for both the left and right eyes and then averaged to get the final EAR. They mention the importance of the convex hull in determining the minimum boundary of an object, which is used in this context to detect the eye's shape. The speaker also describes how to draw contours and convex hulls on the image using OpenCV functions. The paragraph concludes with the implementation of a condition to trigger an alarm when the EAR falls below a certain threshold, indicating an eye blink.
π¨ Alert System and Text Display
The speaker continues with the implementation of the alert system, which is triggered when the EAR is below the threshold for a specified number of frames. They introduce a variable 'flag' to count the number of frames where the EAR is below the threshold. The paragraph also covers how to display warning messages on the frame using 'cv2.putText'. The speaker provides details on the parameters for the text display, such as font type, color, and thickness. They also discuss the use of a variable 'frame_check' to ensure the warning message is displayed for a continuous number of frames.
π΅ Music Integration and Final Touches
In the final paragraph, the speaker adds music to the project using the 'pygame.mixer' module. They guide viewers on how to install the module and load a music file. The speaker demonstrates how to play the music when an eye blink is detected, enhancing the alert system. They also discuss resizing the frame and terminating the loop when a specific key is pressed. The speaker concludes the project by encouraging viewers to like, share, and subscribe, and provides links for further information and source code.
Mindmap
Keywords
π‘Alarm
π‘OpenCV
π‘dlib
π‘Landmarks
π‘EAR (Eye Aspect Ratio)
π‘Convex Hull
π‘Contours
π‘Threshold
π‘Imutils
π‘Pi game modules
π‘Video Capture
Highlights
Introduction to building an application that plays an alarm when detecting eye closure.
Installation of required modules using pip, including opencv-python and imutils.
Potential issues with the RP system and automatic installation of necessary libraries.
Importing and utilizing the imutils library for basic image processing functions.
Setting up the video capture using OpenCV and reading the frames.
Displaying the image on a window using OpenCV's imshow method.
Utilizing the waitKey function for managing window display time.
Initialization of the frontal face detector and explanation of its advantages over traditional methods.
Introduction to the use of dlib's HOG and linear SVM phase detector for face detection.
Importance of detecting 68 facial landmarks for various applications like Snapchat filters.
Explanation of calculating the eye aspect ratio (EAR) for eye blink detection.
Defining a function to calculate the EAR using the coordinates of facial landmarks.
Use of convex hull to determine the minimum boundary of an object, such as an eye.
Drawing contours to visualize the boundary of the detected eyes.
Condition for detecting eye closure based on EAR and frame count.
Displaying alert messages on the frame when eye closure is detected.
Resizing the frame for better visualization using imutils' resize function.
Termination condition for the application using a keyboard interrupt.
Integration of the pygame.mixer module for playing an alarm sound.
Final demonstration of the project with eye closure detection and alarm sound.
Conclusion and invitation for feedback, sharing, and subscription to the channel.
Transcripts
so it will play an alarm so let us see the whole working of it
foreign
[Music]
foreign [Music]
so that is the whole functionality of this application now without wasting any
time let us start building this project
now here we are on our editor and before starting the project I just want you
guys to download these two files download
firstly let us install the important modules your required as projects
by writing pip install opencv python
now midi system may already installed as so it says requirement already satisfied
but RP system May throw us a Time Lego automatically installed
now we need to install imutil showcase series of functions
for the basic image processing functions such as translation rotation resizing
skeletization detecting edges and sorting Contours now what those are is
simply install it by writing pip install imutils
foreign
module so follow alongside me go to your browser and simply type
visual Studios
KY official websites download section say you can download the community
version which is available for free once downloading is complete simply install
it but installation make sure to check desktop development with C plus and
because your body packages so it will take a while to install once it is
installed again come back to your command prompted Yahoo installs C make
by writing pip install cmic
and once it is done you can install delete by writing pip install delete
and this is the simplest way to install it
now sorry modules install logic so let us simply import them and subscribe I'm
importing CV2 now my important
and from imutils foreign
now we will import dlib and from sci-fi dot specials I'll import
distance
my bad this is sci-fi now once this is done I'll write cap
equals to CV2 dot video capture 0.
foreign
true
RIT comma frame equals to cap dot read
now yeah I'm writing cap dot read Now read me
built-in function which return two values where first one is a Boolean
variable that Returns the value true or false based on if the frame is available
or not just comments and second is
uh frame and frame
now I am showbu CV2 guide built-in method which is used to display image on
our window here though parameters except where first one is the window name which
means name of the window which is the image to save it display and in our
case image is stored in frame variables now let us write cv2. wait key one
now weight cubic built-in function which allows us to display a window for our
given millisecond or until any case pressed here time go in millisecond as
parameter except and it waits for the given time to destroy the
windows so it will wait until any key is pressed so this is the standard boiler
template of the opencv Run Guru so you can see it starts recording
now let us start our actual coding by initializing our frontal face
detector and for that many amperage variable here I detect and
yeah but I'm writing delete dot get frontal face detector
now your frontal face detector delivery built-in function and you must be
wondering why we are using frontal face detector instead of traditional opencvs
hard Cascade detector and guys the reason for that is that daily frontal
face detector is quite fast and effective as compared to that one it
does not accept any parameter and Champion so it Returns the sum of
pre-trained hog and linear svm phase detector which is included in the dlib
library where hog stands for histogram of oriented gradients
now let us take one more variable in predict
and yeah
name of this file hook up in the description box to
download
I'll paste it here with DOT dat extension
now what it does is it will detect 68 landmarks from our face these landmarks
are some points on our face which includes eyes eyebrows jawline lips uh
nose baby etc etc in general economic Snapchat where we have different filters
available okay so all these filters use landmarks as a reference point
now if you will see this image so you can see the eyes with six points
detector this is
point zero and point three these are known as long points or horizontal
points which are point one and five and point
two and four these are known as short short points or also vertical points and
ham login points to calculate ear that is I aspect ratio now both is simple or
effective method has to calculate the relation between horizontal and vertical
measurements of an eye this means we don't even require anything like image
processing now there is an equation to calculate the eer using these six points
which is e a r equals to sum of vertical distance upon 2 into horizontal distance
of I yeah per sum of vertical distance means the equilibrium distance between
point one and five and between points 2 and 4 and horizontal distance means the
equilibrium distance between point zero and three so let us Define a function
for it and here I'm writing Define
I aspect ratio which exception parameter I
for you make variable around
between point one and five
now let me copy this and I'll paste it here
let this be Point p ages
distance between point two and four
and let this be Point C is
distance between point zero and three
now equilibrium distance to calculate currently I have taken this built-in
function but I gotta block manually calculated you can do it manually by
using the formula equilibrium distance equals to under root of X2 minus x 1 cos
square plus 2 I 2 minus y 1 square now let us calculate e a r which is sum of
vertical distance that is a and b divided by 2 into horizontal distance
that is point C now we will simply return this ear
now equation arrives with I had that why is it necessary to calculate ear what is
so magical about this ear and the answer is value of eer remains constant so B I
opened here but it suddenly drops here and that is because I close on a
value of a plus b decreases now there is a graph for this
activity that the value of er was constant except one point
or you can say I close within hence we can say this is the most simplest and
most effective way to detect an eye blink now because we have two eyes
that is for left and right eye individually
to calculate the final ear and then is final eer come look check if it is less
than a certain threshold value for a certain period of time or we can say for
certain frames then we will play an alarm for a threshold value can be
different for different users merely it worked between 0.25 to 0.3 but in some
case is it can go to 0.4 or 0.5 as well it all depends on the FPS that is frame
per second of your video webcam now average come look bad May calculate but
before that let us come here and yeah permac
variable declared around gray and here I'll write CV2 dot convert color
off frame
CV2 dot color underscore BGR to Gray
now yeah
now let us detect the faces from this grayscale frames and for that I'll write
detect gray comma zero
now Java detect is the detector this is gray is this grayscale frames and zero
is an index now initialize in a variable name subjects
now let us take a for Loop for subject in subjects
now detector let us detect the landmarks on it and for that I'm writing predict
in Gray subjects
now yapper predict is this predictor Grays is grayscale frames and subjects
is the detector output now is in a variable name shape
now let us write shape equals to face underscore utils
dot shape to NP
shape now here we are converting shapes to a
list of X Y coordinates now let us go back up and yeah but let
us take two points I'll start and I'll end
and yamara right face underscore utils dot facial
underscore landmark underscore 68 underscore ID access
puff left eye
now these are the eye landmarks of the left eye
let us copy it and let us paste it here
to detect the eye line Mark of right eye as well
R start comma R end and here right underscore I
now let's go back down and yeah
to extract the left and right eye so let us write left eye
equals to shape I'll start
and L end
let me copy this and let me do the same for right eye
now once something left and right I could detect earlier let us calculate
the individual ear by using this I detect ratio function and for that my
upper is
foreign
now let us copy this and let us do the same for write I as
well
that actually will be the average of this left and right ear so let us
calculate it by writing e a r equals to left ear plus right ear
whole divided by 2. now once this is done we will find
something called convex Hull a convex Hull is the minimum boundary of an
object that can completely enclose or wrap the object for example let us
consider this rectangle this may we have two borders one in black color and one
in red color and the convex cell of this shape will be the black border because
that is the minimum boundary Choice object to completely wrap current
so let us calculate its convex cell by using cb2's built-in convex cell method
and our left eye calculate so we'll write left eye now is
similarly let us copy this and let us do it for the right eye as well
yeah but we'll write write and this com looks occurring in a
variable name right now let us detect Contours and what is this Contour it is
a curve that will join all the points lying on the boundary of that shape is
example may you can see that the red points are the convex cells while the
green line is the Contour I still remember advanced mathematics class we
manage topic importance
because the online tutor that taught me never mentioned any of the real world
application of it how important it is in computer vision
is example maybe you can see that this red points are the convex cell while the
screen line is the Contour now let us draw it using cv2's built-in draw
Contour functions okay
now this function accepts five arguments where first one is the image which is
second is the Contour that is the indicated contour and yeah instead of
that so we'll write left eye hub for Left Eye
third is the Contour index which represents the pixel coordinates that
are listed in the Contour is say we can indicate the index of the exact point
just comments example but this point will be index one
this will be index two this will be index three and so on but because some
SRA control points so we'll pass any negative value so I'm writing minus one
now fourth argument is the color which will indicate the color of the line now
my upper rock around green line so I'm writing 0 to 550
and final one is the thickness of the line so this is how we will draw the
control on the left eye now let's do the same for right eye and for that manage
to copy here and I'm pasting it here and jumper instead of left eye here I'll
write right I have now these four line of code is just
returned to draw this green outline on the image now yamper my condition if e a
r less than thresh now what thresh is
0.25 works perfectly fine with my system but it might not work perfectly with
your setup so block threshold different values may be 0.3 0.4 or 0.5 could be
tricer so here I'm declining threshold equals to 0.25
[Music]
that is for every frame the value of flag will be incremented and now we'll
print it
warning message
but it will create a small problem and that is
I'll take one more variable which is frame underscore check
now if the value of flag is greater than or equals to this value
then only I'm a warning message display with the help of this conditional
statement
irrespect ratio continuously 20 frames
and with the help of this system
now let us put a warning message here CV2 dot put text we want our text on
frame with the message as alert
third parameter is the coordinates of X and Y axis so let it be 10 30
fourth is the font this may have a message display
so I'm taking font Hershey simplex
fifth is the thickness of the font six is the color of the font
and last is the thickness of line now again value threshold values
now let us run it once
so it gives an attribute error uh okay
this is landmarks guys similarly let us do it here
let us run it again
okay and uh my bad uh here this is orange
let us run it once again
okay generates one more error ah
again my bad that was a typo let us run it once again
and this time it runs successfully so this is the flag value
now let me close my eyes
[Music] and here it is displaying an alert
message
let us do one more thing
let us display one more message this time on Y axis as 325
and yeah let us resize our frame
using imutil's free size function
let us set its width to 450 now is infinite gluco terminate let us
take an if condition that is if cb2.8 q1 and 0xff
equals to equals to ordq
break now we'll destroy all windows
and we will release the cap
and with that I may almost put a project curly I accept one thing and that is
playing music now let's do it as well and for that my pie game modules a mixer
functions
and before importing make sure to install it
and for that come to your command prompt and right with install Pi games
now buy game we made a system we already installed so it says requirement already
satisfied but RK system is automatically installed
now Pi game modules
you can even refer that video after the competition of this one usually
Implement description box now let us initialize our mixer module
here
and let us also load our music file using mixer dot music
dot load and here we will write the path name as
the extension of file and because of the music file already have my project
folder May store here so we don't need to mention the path and look simply
namely and the extension as well now once this is done
and the movement warning message
using mixers already built-in music function that is mixer dot music dot
play let us save it let us run it once again
let me close my eyes
[Music] and here you can see it started playing
the morning music so that's it from this project guys I
hope you like this one and if you do kindly give it a thumbs up share it with
your friends and forgot not to subscribe to our Channel if you have any doubt
related to any of our project you can contact me on my Facebook and Instagram
handles Link in the description box also check description box for the source
code of this project and link to our other YouTube videos as well
this is actually signing off
Browse More Related Video
Node.js | AI Text Summarization in 5 minutes!
Face recognition in real-time | with Opencv and Python
Automatic number plate recognition (ANPR) with Yolov9 and EasyOCR
Nestjs backend rest api - Ecommerce project. Introduction & setup.
AR Indoor Navigation from scratch in under 30 minutes with Unity 6
Curso Sistema de Ventas en ASP.NET Core 6 y SQL Server - Parte 06
5.0 / 5 (0 votes)