Driver Drowsiness Detection System using OpenCV | Graduation final year Project with Source Code

Project Guru
7 Jan 202325:08

Summary

TLDRThis video script guides viewers through building a project that detects eye blinks using OpenCV and dlib. It covers installing necessary modules, initializing face and landmark detectors, calculating eye aspect ratios, and playing an alarm when a blink is detected. The tutorial also demonstrates drawing contours and convex hulls, and concludes with playing music using the pygame module.

Takeaways

  • πŸ˜€ The video script describes the process of building an application that detects eye blinks and plays an alarm when a blink is detected.
  • πŸ› οΈ The project requires the installation of several Python modules including 'opencv-python', 'RPi.GPIO', 'imutils', 'dlib', and 'cmake'.
  • πŸ” The 'dlib' library is used for face detection and to detect 68 facial landmarks, which are essential for tracking eye movements.
  • πŸ“š The script introduces the concept of the Eye Aspect Ratio (EAR), which is calculated using the coordinates of the facial landmarks around the eyes to detect blinks.
  • πŸ“‰ The EAR value remains constant and drops significantly when the eyes are closed, making it a reliable metric for detecting eye blinks.
  • πŸ‘οΈ The script details how to calculate the EAR for both the left and right eyes and then averages them to determine if a blink has occurred.
  • πŸ”Ά The convex hull, which is the smallest convex polygon that can enclose a set of points, is used to simplify the shape of the eyes for contour analysis.
  • πŸ”² The script includes a method to draw the convex hull and contours on the video frames to visualize the eye shapes.
  • ⏰ An alarm is triggered when the EAR is below a certain threshold for a specified number of frames, indicating a sustained eye closure.
  • πŸ“ˆ The script mentions adjusting the threshold value based on the user's setup and the frame rate of the webcam.
  • 🎢 Towards the end, the script adds functionality to play an alarm sound using the 'pygame' module when the eye blink is detected.

Q & A

  • What is the purpose of the application being discussed in the script?

    -The purpose of the application is to detect eye blinks using facial landmarks and play an alarm when a blink is detected.

  • Which modules are required to be installed for this project?

    -The required modules are opencv-python, imutils, dlib, and pygame.

  • Why is the frontal face detector used instead of the traditional OpenCV's Haar Cascade detector?

    -The frontal face detector is used because it is faster and more effective compared to the traditional Haar Cascade detector.

  • What is the significance of the EAR (Eye Aspect Ratio) in this project?

    -EAR is used to detect eye blinks. The value of EAR remains constant except when the eyes are closed, making it an effective way to detect eye blinks.

  • How is the EAR calculated in the script?

    -EAR is calculated using the distances between specific facial landmarks of the eyes. It is the sum of the vertical distances divided by twice the horizontal distance.

  • What are the facial landmarks used for in this project?

    -Facial landmarks are used to detect 68 points on the face, which include eyes, eyebrows, jawline, lips, nose, etc. These landmarks are used as reference points for detecting eye blinks.

  • What is the role of the convex hull in this project?

    -The convex hull is used to determine the minimum boundary that can completely enclose the object, in this case, the eyes. It helps in drawing the outline around the eyes.

  • How does the script handle the detection of eye blinks?

    -The script calculates the EAR for both the left and right eyes, averages them, and checks if it is less than a certain threshold for a specific number of frames. If it is, an alarm is played.

  • What is the threshold value used for detecting eye blinks and how can it be adjusted?

    -The threshold value is initially set to 0.25, but it can be adjusted between 0.25 to 0.5 depending on the user's setup and the frame rate of the video or webcam.

  • How is the music played in the application?

    -Music is played using the pygame module. The music file is loaded and played using the mixer.music.play function.

Outlines

00:00

πŸ› οΈ Project Setup and Module Installation

This paragraph introduces the project by explaining the need to set up an application that triggers an alarm. The speaker guides viewers on how to start the project by opening the editor and installing necessary modules using pip install commands for 'opencv-python', 'imutils', and 'dlib'. They also mention potential issues with the 'RP system' and provide steps to install 'Cmake' and 'delete'. The speaker emphasizes the importance of downloading specific files and setting up the environment correctly before beginning the coding process.

05:00

πŸ” Face Detection and Landmark Detection

The speaker delves into the technical aspects of the project, focusing on face and landmark detection. They explain the use of the frontal face detector from the 'dlib' library, which is more efficient than traditional OpenCV methods. The paragraph also covers the detection of 68 facial landmarks, which are crucial for various applications like Snapchat filters. The speaker introduces a method to calculate the Eye Aspect Ratio (EAR) using these landmarks, which is a key step in detecting eye blinks. The EAR calculation is detailed, explaining the significance of horizontal and vertical distances between specific points on the face.

10:03

πŸ‘οΈβ€πŸ—¨οΈ Eye Blink Detection Algorithm

This paragraph discusses the algorithm used for detecting eye blinks. The speaker explains how the EAR is calculated for both the left and right eyes and then averaged to get the final EAR. They mention the importance of the convex hull in determining the minimum boundary of an object, which is used in this context to detect the eye's shape. The speaker also describes how to draw contours and convex hulls on the image using OpenCV functions. The paragraph concludes with the implementation of a condition to trigger an alarm when the EAR falls below a certain threshold, indicating an eye blink.

15:09

🚨 Alert System and Text Display

The speaker continues with the implementation of the alert system, which is triggered when the EAR is below the threshold for a specified number of frames. They introduce a variable 'flag' to count the number of frames where the EAR is below the threshold. The paragraph also covers how to display warning messages on the frame using 'cv2.putText'. The speaker provides details on the parameters for the text display, such as font type, color, and thickness. They also discuss the use of a variable 'frame_check' to ensure the warning message is displayed for a continuous number of frames.

20:12

🎡 Music Integration and Final Touches

In the final paragraph, the speaker adds music to the project using the 'pygame.mixer' module. They guide viewers on how to install the module and load a music file. The speaker demonstrates how to play the music when an eye blink is detected, enhancing the alert system. They also discuss resizing the frame and terminating the loop when a specific key is pressed. The speaker concludes the project by encouraging viewers to like, share, and subscribe, and provides links for further information and source code.

Mindmap

Keywords

πŸ’‘Alarm

An alarm in the context of this video refers to a warning signal that is triggered when the system detects an eye blink. The video discusses building an application that plays an alarm when it detects a certain condition, specifically an eye blink. This is part of a larger theme of using computer vision to detect and respond to specific actions or events.

πŸ’‘OpenCV

OpenCV, or Open Source Computer Vision Library, is a library of programming functions mainly aimed at real-time computer vision. In the video, OpenCV is used to handle video capture and image processing tasks, such as reading frames from a video source and displaying them. It is a crucial tool in building the project discussed in the script.

πŸ’‘dlib

dlib is a toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real-world problems. In the video, dlib is used for face detection and landmark detection, which are essential for determining the position and shape of facial features like eyes, eyebrows, and lips.

πŸ’‘Landmarks

In the context of facial recognition and image processing, landmarks refer to specific points on a face that are used to detect and track facial features. The video mentions detecting 68 facial landmarks, which can be used to analyze and understand the position and movement of different parts of the face, such as the eyes and mouth.

πŸ’‘EAR (Eye Aspect Ratio)

The Eye Aspect Ratio (EAR) is a measure used in facial recognition and emotion detection to determine the openness of an eye. In the video, EAR is calculated using the positions of specific facial landmarks around the eyes. A sudden drop in EAR can indicate an eye blink, which is the key event the application is designed to detect.

πŸ’‘Convex Hull

A convex hull is the smallest convex set that contains a given set of points. In the video, the convex hull is used to define the boundary of the eyes in the image. It helps in visualizing the shape and extent of the eyes, which is crucial for accurately detecting eye blinks.

πŸ’‘Contours

Contours in image processing are curves that join all the points lying on the boundary of an object. In the video, contours are used to outline the convex hull of the eyes, providing a visual representation of the eye's shape and helping to determine when an eye blink occurs.

πŸ’‘Threshold

In the context of the video, a threshold is a predefined value used to determine when an action should be taken. Specifically, the EAR must fall below a certain threshold for a certain number of frames to trigger the alarm. This is a critical parameter in the application for detecting eye blinks accurately.

πŸ’‘Imutils

Imutils is a series of convenience functions to make basic image processing functions easier with OpenCV. In the video, Imutils is used for tasks like resizing images and converting color spaces, which are common operations in computer vision projects.

πŸ’‘Pi game modules

Pi game modules refer to a Python library used for playing sounds and music. In the video, the library is used to play an alarm sound when an eye blink is detected. This adds an auditory component to the application, alerting the user to the detected event.

πŸ’‘Video Capture

Video capture refers to the process of recording video from a camera or other video source. In the script, video capture is initiated using OpenCV's `cv2.VideoCapture` function to read frames from a video source, which is fundamental for processing and analyzing the video content.

Highlights

Introduction to building an application that plays an alarm when detecting eye closure.

Installation of required modules using pip, including opencv-python and imutils.

Potential issues with the RP system and automatic installation of necessary libraries.

Importing and utilizing the imutils library for basic image processing functions.

Setting up the video capture using OpenCV and reading the frames.

Displaying the image on a window using OpenCV's imshow method.

Utilizing the waitKey function for managing window display time.

Initialization of the frontal face detector and explanation of its advantages over traditional methods.

Introduction to the use of dlib's HOG and linear SVM phase detector for face detection.

Importance of detecting 68 facial landmarks for various applications like Snapchat filters.

Explanation of calculating the eye aspect ratio (EAR) for eye blink detection.

Defining a function to calculate the EAR using the coordinates of facial landmarks.

Use of convex hull to determine the minimum boundary of an object, such as an eye.

Drawing contours to visualize the boundary of the detected eyes.

Condition for detecting eye closure based on EAR and frame count.

Displaying alert messages on the frame when eye closure is detected.

Resizing the frame for better visualization using imutils' resize function.

Termination condition for the application using a keyboard interrupt.

Integration of the pygame.mixer module for playing an alarm sound.

Final demonstration of the project with eye closure detection and alarm sound.

Conclusion and invitation for feedback, sharing, and subscription to the channel.

Transcripts

play00:19

so it will play an alarm so let us see the whole working of it

play00:28

foreign

play00:34

[Music]

play00:38

foreign [Music]

play00:45

so that is the whole functionality of this application now without wasting any

play00:49

time let us start building this project

play00:58

now here we are on our editor and before starting the project I just want you

play01:02

guys to download these two files download

play01:10

firstly let us install the important modules your required as projects

play01:18

by writing pip install opencv python

play01:24

now midi system may already installed as so it says requirement already satisfied

play01:27

but RP system May throw us a Time Lego automatically installed

play01:32

now we need to install imutil showcase series of functions

play01:35

for the basic image processing functions such as translation rotation resizing

play01:41

skeletization detecting edges and sorting Contours now what those are is

play01:48

simply install it by writing pip install imutils

play01:55

foreign

play02:44

module so follow alongside me go to your browser and simply type

play02:51

visual Studios

play02:57

KY official websites download section say you can download the community

play03:01

version which is available for free once downloading is complete simply install

play03:06

it but installation make sure to check desktop development with C plus and

play03:12

because your body packages so it will take a while to install once it is

play03:16

installed again come back to your command prompted Yahoo installs C make

play03:21

by writing pip install cmic

play03:27

and once it is done you can install delete by writing pip install delete

play03:35

and this is the simplest way to install it

play03:39

now sorry modules install logic so let us simply import them and subscribe I'm

play03:44

importing CV2 now my important

play03:50

and from imutils foreign

play04:02

now we will import dlib and from sci-fi dot specials I'll import

play04:07

distance

play04:16

my bad this is sci-fi now once this is done I'll write cap

play04:21

equals to CV2 dot video capture 0.

play04:27

foreign

play04:42

true

play04:47

RIT comma frame equals to cap dot read

play04:54

now yeah I'm writing cap dot read Now read me

play05:00

built-in function which return two values where first one is a Boolean

play05:04

variable that Returns the value true or false based on if the frame is available

play05:08

or not just comments and second is

play05:18

uh frame and frame

play05:24

now I am showbu CV2 guide built-in method which is used to display image on

play05:28

our window here though parameters except where first one is the window name which

play05:32

means name of the window which is the image to save it display and in our

play05:39

case image is stored in frame variables now let us write cv2. wait key one

play05:47

now weight cubic built-in function which allows us to display a window for our

play05:51

given millisecond or until any case pressed here time go in millisecond as

play05:55

parameter except and it waits for the given time to destroy the

play06:00

windows so it will wait until any key is pressed so this is the standard boiler

play06:05

template of the opencv Run Guru so you can see it starts recording

play06:14

now let us start our actual coding by initializing our frontal face

play06:18

detector and for that many amperage variable here I detect and

play06:23

yeah but I'm writing delete dot get frontal face detector

play06:35

now your frontal face detector delivery built-in function and you must be

play06:38

wondering why we are using frontal face detector instead of traditional opencvs

play06:42

hard Cascade detector and guys the reason for that is that daily frontal

play06:47

face detector is quite fast and effective as compared to that one it

play06:51

does not accept any parameter and Champion so it Returns the sum of

play06:55

pre-trained hog and linear svm phase detector which is included in the dlib

play07:00

library where hog stands for histogram of oriented gradients

play07:05

now let us take one more variable in predict

play07:08

and yeah

play07:19

name of this file hook up in the description box to

play07:24

download

play07:27

I'll paste it here with DOT dat extension

play07:32

now what it does is it will detect 68 landmarks from our face these landmarks

play07:36

are some points on our face which includes eyes eyebrows jawline lips uh

play07:41

nose baby etc etc in general economic Snapchat where we have different filters

play07:50

available okay so all these filters use landmarks as a reference point

play07:55

now if you will see this image so you can see the eyes with six points

play07:59

detector this is

play08:02

point zero and point three these are known as long points or horizontal

play08:06

points which are point one and five and point

play08:11

two and four these are known as short short points or also vertical points and

play08:16

ham login points to calculate ear that is I aspect ratio now both is simple or

play08:23

effective method has to calculate the relation between horizontal and vertical

play08:26

measurements of an eye this means we don't even require anything like image

play08:30

processing now there is an equation to calculate the eer using these six points

play08:34

which is e a r equals to sum of vertical distance upon 2 into horizontal distance

play08:39

of I yeah per sum of vertical distance means the equilibrium distance between

play08:43

point one and five and between points 2 and 4 and horizontal distance means the

play08:48

equilibrium distance between point zero and three so let us Define a function

play08:52

for it and here I'm writing Define

play08:57

I aspect ratio which exception parameter I

play09:03

for you make variable around

play09:07

between point one and five

play09:21

now let me copy this and I'll paste it here

play09:27

let this be Point p ages

play09:31

distance between point two and four

play09:36

and let this be Point C is

play09:39

distance between point zero and three

play09:44

now equilibrium distance to calculate currently I have taken this built-in

play09:48

function but I gotta block manually calculated you can do it manually by

play09:52

using the formula equilibrium distance equals to under root of X2 minus x 1 cos

play09:57

square plus 2 I 2 minus y 1 square now let us calculate e a r which is sum of

play10:02

vertical distance that is a and b divided by 2 into horizontal distance

play10:08

that is point C now we will simply return this ear

play10:13

now equation arrives with I had that why is it necessary to calculate ear what is

play10:18

so magical about this ear and the answer is value of eer remains constant so B I

play10:23

opened here but it suddenly drops here and that is because I close on a

play10:30

value of a plus b decreases now there is a graph for this

play10:34

activity that the value of er was constant except one point

play10:39

or you can say I close within hence we can say this is the most simplest and

play10:44

most effective way to detect an eye blink now because we have two eyes

play10:50

that is for left and right eye individually

play10:54

to calculate the final ear and then is final eer come look check if it is less

play11:00

than a certain threshold value for a certain period of time or we can say for

play11:04

certain frames then we will play an alarm for a threshold value can be

play11:08

different for different users merely it worked between 0.25 to 0.3 but in some

play11:13

case is it can go to 0.4 or 0.5 as well it all depends on the FPS that is frame

play11:18

per second of your video webcam now average come look bad May calculate but

play11:23

before that let us come here and yeah permac

play11:26

variable declared around gray and here I'll write CV2 dot convert color

play11:34

off frame

play11:37

CV2 dot color underscore BGR to Gray

play11:44

now yeah

play11:46

now let us detect the faces from this grayscale frames and for that I'll write

play11:52

detect gray comma zero

play11:56

now Java detect is the detector this is gray is this grayscale frames and zero

play12:03

is an index now initialize in a variable name subjects

play12:08

now let us take a for Loop for subject in subjects

play12:18

now detector let us detect the landmarks on it and for that I'm writing predict

play12:24

in Gray subjects

play12:30

now yapper predict is this predictor Grays is grayscale frames and subjects

play12:36

is the detector output now is in a variable name shape

play12:41

now let us write shape equals to face underscore utils

play12:48

dot shape to NP

play12:53

shape now here we are converting shapes to a

play12:57

list of X Y coordinates now let us go back up and yeah but let

play13:02

us take two points I'll start and I'll end

play13:08

and yamara right face underscore utils dot facial

play13:15

underscore landmark underscore 68 underscore ID access

play13:23

puff left eye

play13:30

now these are the eye landmarks of the left eye

play13:34

let us copy it and let us paste it here

play13:39

to detect the eye line Mark of right eye as well

play13:44

R start comma R end and here right underscore I

play13:50

now let's go back down and yeah

play13:54

to extract the left and right eye so let us write left eye

play14:01

equals to shape I'll start

play14:07

and L end

play14:12

let me copy this and let me do the same for right eye

play14:25

now once something left and right I could detect earlier let us calculate

play14:29

the individual ear by using this I detect ratio function and for that my

play14:35

upper is

play14:38

foreign

play14:52

now let us copy this and let us do the same for write I as

play14:57

well

play15:09

that actually will be the average of this left and right ear so let us

play15:14

calculate it by writing e a r equals to left ear plus right ear

play15:26

whole divided by 2. now once this is done we will find

play15:31

something called convex Hull a convex Hull is the minimum boundary of an

play15:35

object that can completely enclose or wrap the object for example let us

play15:39

consider this rectangle this may we have two borders one in black color and one

play15:43

in red color and the convex cell of this shape will be the black border because

play15:48

that is the minimum boundary Choice object to completely wrap current

play15:52

so let us calculate its convex cell by using cb2's built-in convex cell method

play15:58

and our left eye calculate so we'll write left eye now is

play16:07

similarly let us copy this and let us do it for the right eye as well

play16:12

yeah but we'll write write and this com looks occurring in a

play16:17

variable name right now let us detect Contours and what is this Contour it is

play16:24

a curve that will join all the points lying on the boundary of that shape is

play16:28

example may you can see that the red points are the convex cells while the

play16:31

green line is the Contour I still remember advanced mathematics class we

play16:35

manage topic importance

play16:39

because the online tutor that taught me never mentioned any of the real world

play16:43

application of it how important it is in computer vision

play16:47

is example maybe you can see that this red points are the convex cell while the

play16:52

screen line is the Contour now let us draw it using cv2's built-in draw

play16:56

Contour functions okay

play17:00

now this function accepts five arguments where first one is the image which is

play17:05

second is the Contour that is the indicated contour and yeah instead of

play17:11

that so we'll write left eye hub for Left Eye

play17:18

third is the Contour index which represents the pixel coordinates that

play17:22

are listed in the Contour is say we can indicate the index of the exact point

play17:27

just comments example but this point will be index one

play17:33

this will be index two this will be index three and so on but because some

play17:37

SRA control points so we'll pass any negative value so I'm writing minus one

play17:42

now fourth argument is the color which will indicate the color of the line now

play17:47

my upper rock around green line so I'm writing 0 to 550

play17:52

and final one is the thickness of the line so this is how we will draw the

play17:56

control on the left eye now let's do the same for right eye and for that manage

play18:00

to copy here and I'm pasting it here and jumper instead of left eye here I'll

play18:04

write right I have now these four line of code is just

play18:08

returned to draw this green outline on the image now yamper my condition if e a

play18:14

r less than thresh now what thresh is

play18:27

0.25 works perfectly fine with my system but it might not work perfectly with

play18:32

your setup so block threshold different values may be 0.3 0.4 or 0.5 could be

play18:37

tricer so here I'm declining threshold equals to 0.25

play18:48

[Music]

play18:53

that is for every frame the value of flag will be incremented and now we'll

play18:58

print it

play19:02

warning message

play19:05

but it will create a small problem and that is

play19:17

I'll take one more variable which is frame underscore check

play19:25

now if the value of flag is greater than or equals to this value

play19:35

then only I'm a warning message display with the help of this conditional

play19:41

statement

play19:43

irrespect ratio continuously 20 frames

play19:49

and with the help of this system

play19:52

now let us put a warning message here CV2 dot put text we want our text on

play19:59

frame with the message as alert

play20:06

third parameter is the coordinates of X and Y axis so let it be 10 30

play20:12

fourth is the font this may have a message display

play20:16

so I'm taking font Hershey simplex

play20:21

fifth is the thickness of the font six is the color of the font

play20:29

and last is the thickness of line now again value threshold values

play20:39

now let us run it once

play20:44

so it gives an attribute error uh okay

play20:50

this is landmarks guys similarly let us do it here

play20:58

let us run it again

play21:02

okay and uh my bad uh here this is orange

play21:10

let us run it once again

play21:15

okay generates one more error ah

play21:21

again my bad that was a typo let us run it once again

play21:27

and this time it runs successfully so this is the flag value

play21:32

now let me close my eyes

play21:38

[Music] and here it is displaying an alert

play21:42

message

play21:48

let us do one more thing

play21:53

let us display one more message this time on Y axis as 325

play21:59

and yeah let us resize our frame

play22:04

using imutil's free size function

play22:10

let us set its width to 450 now is infinite gluco terminate let us

play22:18

take an if condition that is if cb2.8 q1 and 0xff

play22:26

equals to equals to ordq

play22:32

break now we'll destroy all windows

play22:42

and we will release the cap

play22:48

and with that I may almost put a project curly I accept one thing and that is

play22:53

playing music now let's do it as well and for that my pie game modules a mixer

play22:58

functions

play23:03

and before importing make sure to install it

play23:07

and for that come to your command prompt and right with install Pi games

play23:15

now buy game we made a system we already installed so it says requirement already

play23:19

satisfied but RK system is automatically installed

play23:23

now Pi game modules

play23:27

you can even refer that video after the competition of this one usually

play23:32

Implement description box now let us initialize our mixer module

play23:37

here

play23:40

and let us also load our music file using mixer dot music

play23:46

dot load and here we will write the path name as

play23:51

the extension of file and because of the music file already have my project

play23:55

folder May store here so we don't need to mention the path and look simply

play23:59

namely and the extension as well now once this is done

play24:05

and the movement warning message

play24:10

using mixers already built-in music function that is mixer dot music dot

play24:16

play let us save it let us run it once again

play24:23

let me close my eyes

play24:28

[Music] and here you can see it started playing

play24:37

the morning music so that's it from this project guys I

play24:42

hope you like this one and if you do kindly give it a thumbs up share it with

play24:45

your friends and forgot not to subscribe to our Channel if you have any doubt

play24:48

related to any of our project you can contact me on my Facebook and Instagram

play24:52

handles Link in the description box also check description box for the source

play24:57

code of this project and link to our other YouTube videos as well

play25:00

this is actually signing off

Rate This
β˜…
β˜…
β˜…
β˜…
β˜…

5.0 / 5 (0 votes)

Related Tags
Eye Blink DetectionOpenCVdlibFace DetectionAlarm SystemImage ProcessingPython CodingVideo SurveillanceSecurity MeasuresComputer Vision