tensorflow custom object detection model | raspberry pi 4 tensorflow custom object detection

Summary

In this Freedom Tech YouTube tutorial, viewers learn to train a custom TensorFlow Lite object detection model using Google Colab. The video guides through installing necessary software on a Raspberry Pi 4, capturing and labeling images, and using LabelImg software. It covers uploading the labeled data to Google Drive, accessing it via Google Colab, and training the model. Finally, it demonstrates detecting objects like ESP8266 and Raspberry Pi Pico using the trained model, a mobile camera, and IP Webcam software.

Takeaways

• 😀 The video is a tutorial on training a custom model using Google Colab for object detection with a Raspberry Pi Pico and ESP8266.
• 🛠️ The tutorial requires installing Raspbian OS Bullseye on a Raspberry Pi 4 and the latest version of OpenCV.
• 📱 The presenter uses a mobile camera with IP Webcam software to detect objects through the custom model.
• 📁 The images for training the model are stored in a folder named 'img' on the Raspberry Pi.
• 🏷️ LabelImg software is used to label the images for training the TensorFlow Lite model.
• 📂 Images are organized into 'train' and 'validate' folders for the training process.
• 🔗 A GitHub repository is mentioned for downloading necessary files and accessing a Google Colab file.
• 📦 The 'yt' folder containing the labeled images and XML files is zipped and uploaded to Google Drive.
• 💾 The zipped folder is then downloaded into Google Colab, where the model training process is initiated.
• 🎯 The trained TensorFlow Lite model is exported and downloaded to the Raspberry Pi for object detection using the IP camera.

Q & A

What is the main objective of the video?

The main objective of the video is to guide viewers on how to train a custom model using Google Colab to detect objects such as a Raspberry Pi Pico and ESP8266.

Which devices are used in the demonstration?

The devices used in the demonstration are a Raspberry Pi Pico and an ESP8266.

What software is required to be installed on the Raspberry Pi 4 for this project?

The software required includes Raspbian OS Bullseye, OpenCV version 4.5.5, TensorFlow Lite, and the IP Webcam software.

How can viewers find instructions for installing the necessary software?

Viewers can find instructions for installing the necessary software by watching the creator's previous videos on installing Raspbian OS Bullseye, OpenCV, and TensorFlow Lite.

What is the purpose of the 'img' folder mentioned in the script?

The 'img' folder contains 70 images of Raspberry Pi Pico and ESP8266 that will be used to train the custom model.

What is LabelImg and why is it used in this project?

LabelImg is a graphical image annotation tool. It is used to label the images of the objects (Raspberry Pi Pico and ESP8266) to create bounding boxes for training the TensorFlow model.

How are the images organized after labeling with LabelImg?

The labeled images are organized into 'train' and 'validate' folders, with corresponding XML files, to be used for training and validating the TensorFlow model.

What is the purpose of zipping the 'yt' folder and uploading it to Google Drive?

The 'yt' folder is zipped and uploaded to Google Drive to make the labeled images and XML files accessible to Google Colab for training the TensorFlow model.

How is the TensorFlow model trained in Google Colab?

The TensorFlow model is trained in Google Colab by using the TFLite Model Maker and the training data from the uploaded 'yt' folder.

What is the final output of the training process?

The final output of the training process is a TensorFlow Lite model that can detect the specified objects (Raspberry Pi Pico and ESP8266).

How is the trained TensorFlow Lite model used to detect objects in real-time?

The trained TensorFlow Lite model is used in conjunction with IP Webcam software and OpenCV to detect objects in real-time through a mobile camera feed.

Outlines

00:00

💻 Introduction to Training a Custom Model with TensorFlow Lite

The video begins with a warm welcome to the Freedom Tech YouTube channel. The host outlines the session's agenda, which includes training a custom model using Google Colab to detect objects like a Raspberry Pi Pico and ESP8266. The host encourages viewers to subscribe if they find the content helpful. The practical demonstration involves setting up a Raspberry Pi 4 with Raspbian OS Bullseye and installing OpenCV and TensorFlow Lite. The host also mentions using a mobile camera with IP Webcam software to detect the objects through the custom model. The video emphasizes the need to watch previous videos for installation instructions and to prepare the environment for model training.

05:04

🖼️ Preparing Images and Using LabelImg Software

The host demonstrates how to prepare images for training the TensorFlow Lite model by capturing 70 images of the Raspberry Pi Pico and ESP8266 using a mobile camera. These images are saved in a folder named 'img'. The next step involves using LabelImg software to label these images. The host guides viewers on installing LabelImg on Raspbian OS Bullseye by providing a GitHub repository link. The process includes cloning the repository and moving the necessary files to the desktop. The host then walks through the steps of using LabelImg to label the images by creating bounding boxes around the objects and assigning them labels.

10:08

📁 Organizing Data and Zipping the Dataset

After labeling the images, the host explains how to organize the data. This involves creating a 'freedom tech' folder with 'train' and 'validate' subfolders. The labeled images and their corresponding XML files are then copied into these folders. The host proceeds to zip the 'freedom tech' folder, ensuring that all the labeled images and XML files are compressed. The zipped file is then moved to Google Drive for easy access and use in Google Colab for the next steps of the model training process.

15:09

🔄 Uploading Data to Google Colab and Setting Up the Environment

The video continues with the host showing how to upload the zipped dataset to Google Colab. This is done by accessing Google Drive and uploading the 'freedom tech.zip' file. Once uploaded, the host demonstrates how to open Google Colab, upload the file there, and set up the runtime environment for model training. This includes installing the TensorFlow Lite Model Maker and TensorFlow Lite Support package, as well as mounting the Google Drive to access the uploaded dataset.

20:10

🚀 Training the Model and Evaluating its Performance

With the environment set up, the host begins the model training process in Google Colab. This involves unzipping the dataset, specifying the object labels, and initiating the training process. After the model is trained, it is evaluated using the validation data. The host then demonstrates how to export the trained model as a TensorFlow Lite model. The final steps include downloading the model to the Raspberry Pi and moving it to the appropriate TensorFlow Lite folder. The host concludes the video by showing a live demonstration of the model detecting the ESP8266 and Raspberry Pi Pico using a mobile camera and IP Webcam software.

📹 Final Demonstration and Conclusion

In the final segment, the host conducts a live test of the custom TensorFlow Lite model. They run a Python script that utilizes the model to detect the ESP8266 and Raspberry Pi Pico through a mobile camera feed. The successful detection of the objects confirms the effectiveness of the trained model. The host expresses hope that viewers have learned from the video and looks forward to the next session, bidding farewell to the audience.

Keywords

💡Google Colab

Google Colab is a free cloud-based Jupyter notebook environment that requires no setup and runs entirely in the cloud. It allows users to write and execute Python code via a web browser, which is useful for machine learning and data analysis. In the video, Google Colab is used to train a custom model with TensorFlow Lite, indicating its role in facilitating machine learning workflows without the need for local computational resources.

💡Raspberry Pi Pico

The Raspberry Pi Pico is a microcontroller board developed by Raspberry Pi Foundation. It is a compact, low-cost device that can be programmed to perform various tasks. In the video, the Pico is one of the objects to be detected using the custom model, showcasing its application in object detection scenarios.

💡ESP8266

ESP8266 is a low-cost Wi-Fi microcontroller chip that can be programmed for various network applications. It is used in the video as another object to be detected by the custom model, demonstrating its relevance in IoT projects and how it can be integrated with machine learning models for detection purposes.

💡TensorFlow Lite

TensorFlow Lite is a lightweight solution for machine learning on mobile and embedded devices. It enables on-device machine learning inference with low latency and a small binary size. In the video, TensorFlow Lite is used to create a custom model for object detection, highlighting its utility for deploying machine learning models on resource-constrained devices.

💡Object Detection

Object detection is a computer vision technology that deals with detecting and identifying multiple objects in an image or video. In the context of the video, object detection is the main goal, where the custom model is trained to detect the ESP8266 and Raspberry Pi Pico, illustrating the practical application of machine learning in recognizing specific objects.

💡Raspbian OS Bullseye

Raspbian OS Bullseye is the latest version of the Raspbian operating system based on Debian Bullseye. It is designed specifically for Raspberry Pi devices. The video mentions installing Raspbian OS Bullseye on a Raspberry Pi 4, which serves as the base for setting up the environment needed to train the custom TensorFlow Lite model.

💡OpenCV

OpenCV (Open Source Computer Vision Library) is an open-source computer vision and machine learning software library. It provides a common infrastructure for computer vision applications and is used extensively in image and video analysis. In the video, OpenCV is installed on Raspbian OS Bullseye to support the development of the object detection model.

💡LabelImg

LabelImg is a graphical image annotation tool that is widely used in machine learning to label images for object detection. It is mentioned in the video as a tool to create bounding boxes around the objects in images, which is a crucial step in preparing the dataset for training the TensorFlow Lite model.

💡IP Webcam

IP Webcam is an application that can be installed on Android devices to turn them into a network surveillance camera. In the video, the presenter uses IP Webcam to capture images from a mobile device, which are then used to test the custom TensorFlow Lite model, demonstrating a real-world application of the model.

💡Model Training

Model training is the process of teaching a machine learning model to make predictions or decisions based on input data. The video describes the steps to train a custom TensorFlow Lite model using Google Colab, emphasizing the iterative nature of model development where data is used to improve the model's accuracy.

💡Google Drive

Google Drive is a file storage and synchronization service developed by Google. It is used in the video to store and share the zipped dataset and trained model files, illustrating its utility in cloud storage and collaboration for machine learning projects.

Highlights

Training a custom model using Google Colab for TensorFlow Lite object detection.

Object detection with Raspberry Pi Pico and ESP8266 using custom-trained models.

Setting up a Raspberry Pi 4 with Raspbian OS Bullseye for machine learning tasks.

Installation of OpenCV 4.4.5 on Raspbian OS Bullseye to support object detection.

Use of mobile camera through IP webcam software for capturing training images.

Labeling images for custom object detection using LabelImg software on Raspbian OS.

Organizing training and validation datasets for custom object detection models.

Cloning a GitHub repository and using Google Colab files for the training process.

Uploading labeled data to Google Drive and preparing it for model training.

Training the TensorFlow Lite model using Google Colab with GPU support.

Evaluating the trained model with validation data to assess performance.

Exporting the trained model as a TensorFlow Lite model for further use.

Running object detection on Raspberry Pi with TensorFlow Lite and custom models.

Integration of IP webcam software and Python scripts for real-time object detection.

Successfully detecting Raspberry Pi Pico and ESP8266 objects with the custom model.

Transcripts

hello friends and welcome to youtube

channel freedom tech and in this session

what we are going to learn in this

session we are going to train our own

custom model with the help of google

collab then we are going to detect our

objects in this scenario i have here a

raspberry pi pico and

esp8266 and we are going to detect

esp8266 and pico with the help of our

own custom model okay so before we move

to our practical friends if you learn

something from our videos please

consider to subscribe our channel okay

friends thank you so much

and let's get started so friends we are

going to train our own custom model for

tensorflow lite with the help of google

collab okay so for that first of course

we need to install the raspbian os

bullseye on raspberry pi 4. i already

created the video how to install

raspbian os bullseye uh this is what a

raspbian os bull size 64-bit version

okay you name

you name hyphen m

as you can see r64 it means uh it is a

64-bit version okay then i have already

installed the opencv latest version

4.4.5 so i already created the video how

to install the opencv latest version on

raspbian os bullseye so watch the video

and install the opencv latest version

4.5.5

then we need here a tensorflow lite i

have already created that video also how

to install the light the tensorflow lite

on raspbianos 64-bit version

just watch video and install the

tensorflow lite

then i am going to use here my mobile

camera with the help of ipv webcam

software

we are going to detect our custom model

means in this scenario i am going to

detect my esp8266 and the pico with the

help of a mobile ipcam software okay so

i already created the video how to

install opencv how to install tensorflow

lite and how to use the mobile camera

with the help of ipcamp for tensorflow

lite so watch these three videos and

just

configure the basic configuration for

this particular video for

training your own custom model for

tensorflow lite okay so i have already

installed all these things

so

we are ready now and first we need here

our images so i have already captured

the images with the help of my mobile

camera so i'm going to show you here if

i open the file manager and i have

create here a folder name called img as

you can see this is what the img folder

here as you can see i have here a 70

images of raspberry pi pico and nodemc

esp8266 okay i'm going to show you here

as you can see esp8266 and the pico

board okay uh i have saved it inside the

slash home slash pi and img folder okay

so

this is what our data which basically we

are going to train here then we need

here a label mg software okay so

we are going to install the

label mg software on raspbian os

bullseye okay for that i have first

created here a

github repository okay you need to

download the repository and inside that

repository we have here a google collab

file and the text file so i will mention

the link you need to simply copy and

paste the link then open the link code

copy

minimize browser and i am going to clone

the folder inside the download so cd

downloads and come to the downloads

folder then sudo

get clone and paste the link

and just hit enter it will clone our

folder inside the downloads i'm going to

clear the screen

ls

and as you can see this is what our

folder simply cd tensorflow hit enter ls

and this is what our file which this is

basically the text file which we want to

move on our desktop so

sudo

mv

label emg slash home slash

pi

and the desktop and desktop hit enter

and as you can see we have our text file

on our now desktop so if i open the text

filter as you can see

here i have mentioned how to install the

label mg on the raspbian os bullseye

okay so first command which we want to

run here we need to install

a

py q25 hyphen dell hyphen tools package

so simply i'm going to run cd command

clear the screen and i am now

inside slash home slash pi okay here we

want to run the command first command

simply hit enter

and it will install the py q25 package

on our raspbian os boot size 64 64-bit

version okay i already installed then we

need to install our main package which

is the label emg copy it and paste

and just hit enter it will install the

label emg on our raspbian us i

already installed as you can see the

requirement is already satisfied that's

it

so now what we want to do we want to

start our label emg software inside

raspbianos so run the command line this

way l a b

and just press the tab button it will

auto complete our command label emg you

can run the command with the help of

sudo or or you can run the command or

with the help of pi user okay so i am

going to run the command as a normal

user so

label amg just hit enter it will open

label emg inside the raspbian os then

open dir and

we need to select our folder where we

save our images so

slash home slash pi emg this is what my

folder and click on choose

then it will open our images as you can

see

then we need to click on change save dir

click on here and mention the same

folder part so

slash home slash pi img

is the same path where we have our

images so same path for the change save

dir okay click on choose

so we have successfully select open dr

path also we have successfully select

the change save dir path that's it now

we are ready and then after what we want

to do we want to click on create

rectangle box just click on and then

we can basically create a bounding box

on our

object so this is what our pico

then just mention the name pico

click on ok then our second object

create rectangle box and also we want to

create the box on our esp8266

then mention esp8266

so we have our two object pico and

esp8266

click on save click on next image and

then we have here our next image which

is

only pico board

so click on pico

ok

save click on next image so this is the

way you need to first label your images

with the help of label emg software so i

have already labeled the images okay so

i am going to simply

close the software i hope so far so good

now what we want to do i am going to

first show you here

i have your already trained images as

you can see this is what the image is

and this is what the xml file if i open

the xml file with the help of text

editor

then

just terminate

as you can see this is what our xml file

as you can see xml file and inside that

we have our object names

as you can see this is what the pico and

this is what the esp8266

okay so this is the way first you need

to label your images with the help of

label emg software that's it now next

step we need to create here a folder

okay so i'm going to

this is what our file manager and here

as you can see

i have created a freedom tech folder

okay so simply you need to right click

on the file manager and create a new

folder i'm going to mention like yt

let's say yt folder

and click on ok this is the way you can

create the folder so i have created the

folder y2 folder then inside that again

we need to open the y2 folder and inside

that again we need to create the two

folder so first folder

which is the train

click on ok

then next one validate

validate that's it so

as i told you first we want to create

the folder in this case i have created a

y2 folder as you can see and inside that

we need to create a train and validate

folder that's it now what we want to do

where we save our train image in this

case i have saved my images which we

just label i have saved inside the

images as you can see we need to simply

press the ctrl a

select all images

then right click

click on copy

and go to the folder white folder

and

train and just paste here all the images

and the xml file okay as you can see

it's copied then same inside the

validate

just paste

okay

so this is how you need to create the

folder and inside that folder you need

to create train and validate folder and

inside that train and validate folder

you need to

paste all your images which you label

okay

now what we want to do we want to simply

zip this y2 folder

we want to simply zip our y2 folder okay

so what i am going to do here i am going

to minimize the browser and in this case

i have already created the folder which

is the freedom tech same folder freedom

tech and inside that i have here a train

and validate folder and inside the train

invalidate as you can see i have already

saved the images with the xml file okay

so now what we want to do we want to zip

the folder so for that we are going to

run the command so simply i'm going to

open our text file

and we need to run this one command zip

hyphen r

then the zip file name and our folder

name so in this case i have created a

freedom tech folder okay so

i'm going to show you here just a minute

so first

open the terminal

then run the command sudo apt

install zip and it will install the zip

package

which i have already installed then

we need to run this one command so

simply copy it

copy it

and just paste inside the terminal

and remember one thing friends i have

here a freedom tech folder so here you

need to mention your folder in case

as i told you we have created the folder

name called yt folder so you need to

mention here a white folder

okay so remember one thing you need to

mention here your folder name then your

folder dot zip file name then slash star

means it will

zip all the files

which we have inside the freedom tab

folder okay so simply i am going to hit

enter

and as you can see it will

now

zip all the content which basically we

have inside the freedom tech folder okay

so friends as you can see we have

successfully zip our

folder if i run the command ls

and here it is whatever zip file

freedomtech.zip now what we want to do

here we want to simply move this file

inside our

google drive okay so now i am going to

close the text file open the browser so

first you need to log in with your gmail

account for accessing your own google

drive so i am going to mention g drive

it will open our google drive

go here google drive sign in i have

already signed in with my drive okay

so here it is as you can see inside my

google drive i have already uh copy the

freedomwebtech.zip file but i am going

to simply remove it

okay remove

and then i am going to again upload the

freedom tech dot zip file so right click

file upload

go to the pi

and we need to search for our file

freedom check dot zip simply

click on open and now it's uploading our

zip file on google drive okay so

meanwhile what we want to do we want to

open our google collab so what we want

to do we want to simply open google

collab

click on here

and then here we want to click on upload

choose file

and we clone our file inside the

downloads using our repository as you

can see this is what the folder inside

that we have our google collab file so

click on

open and it will open the

file inside our google

cooler as you can see

we have our

file inside now google column okay

so

on google drive

as you can see our file uploading

process also completed okay so now what

we want to do we want to open our google

collab and here we have our file already

uploaded now what we want to do we want

to simply go here run time

then change runtime type and so select

the gpu then click on save

now what we want to do we want to simply

start our

file so click on first connect as you

can see

first we need to click on connect

connecting

so as you can see we have successfully

connected now we are ready and we are

going to run our first command so just

click on here and it will install the t

of light model maker and tflight support

package as you can see it's installing

so we have successfully installed the

packages just scroll down

then

import the required packages click on

here

so we have successfully import the

packages just scroll down

and here

these steps is basically for

the mounting our google drive okay so

just click on

and it will ask you to permit this

notebook to access your google drive

file simply click on connect to google

drive

then

it will open this kind of window here

you need to select your gmail id okay so

i'm going to select my gmail id because

i have open my google drive with this

freedom tech email id so i'm going to

select that one then scroll down click

on continue

and that's it we have successfully mount

our drive so now simply scroll down and

we are going to unzip the freedom tech

dot zip so click on here the next step

and it will unzip the

folder okay so just if you click here

then you will see the freedom check

folder train and validate folder as you

can see train and validate if you do

like this way

we have basically all the images as you

can see

okay

so as you can see we have successfully

unzip our freedom tech dot zip file now

just scroll down

scroll down

and this is what the next steps so here

what we want to do we want to mention

our label name so in this case i have

label

esp8266 and pico

okay so mention

your object label's name so esp8266

then the pico and here also

esp8266 and the pico

okay so remember these things you need

to mention your object label names like

this way esp-266 and pico esp8266 and

pico okay then simply click on here

so all the good as you can see green

tigma then just scroll down

scroll down

then next step just click on

so as you can see this step is also

completed just scroll down and then now

we are basically ready we are going to

train our model as you can see train the

tensorflow model with the training data

so now simply we need to click on here

now from here basically we are going to

train our images okay so it will take

some time okay friends

so friends as you can see the training

process has just completed okay we have

completed our training process now next

steps

okay so next step is evaluate the model

with the validation data so simply i am

going to click on here

and as you can see the model

evaluate process just started okay

so friends as you can see the model

evaluation process also completed now

next steps

export as a tensorflow lite model okay

so simply we are going to click on here

on cell

and now it will create the android dot

tf alight okay we are going to basically

uh export our tensorflow lite model okay

and the model name is android dot tf

alight okay

so friends now

we have our model ready okay this is

what the last step okay so export as a

tensorflow lite model this is what our

last step okay now if you just click on

here which is our folder this is what

our model simply we need to click here

right click then click on download and

it will download our model inside our

raspbianos bullseye okay

so as you can see we have successfully

download our model so simply i am going

to minimize the browser

and simply i am going to open the

terminal and this is what our terminal

now simply run the command cd

downloads folder because our model is

downloaded inside the downloads folder

so cd downloads hit enter if i run the

ls command as you can see android dot tf

li this is what our custom model so

clear the screen and we need to move the

model inside our tensorflow folder so

sudo

mv android dot tflight slash home

slash pi

then tensorflow tensorflow lite bullseye

this is what our folder name which we

have already cloned

okay

so let me just make like this way okay

this is what our folder which we have

already cloned in a video where i have

already explained how to install the

tensorflow lite on the bullseye so the

same folder if you don't know how to

install watch the video and first

install okay so we are going to move our

android dot ta for like folder slash

home slash pad tensorflow hyphen light

hyphen bullseye

then inside that we have example folder

then the light folder then again

examples then object detector and then

we have our raspberry pi folder so we

need to move our android dot tf a light

file inside the raspberry pi okay so

simply hit enter

clear the screen now i am going to run

the cd command

then we are going to insert our folder

so cd tensorflow lite example

slide examples object detector then the

raspberry pi

okay clear the screen

clear

ls

as you can see we have successfully move

our model inside the raspberry pi folder

now what i am going to do we are ready

and i am going to run our script with

the help of our model so i have here

file as you can see here detect ipcan.pi

this is what the file this is what the

python script for

running the ipcam with tensorflow lite

okay so what i am going to do here i

need to just start the ipcam software

from mobile then i am going to enter the

ip address inside the

ipcm.5 okay as you can see this is what

the ipcam.part so what i am going to do

i am going to minimize the terminal go

to the menu programming open the tony

python ide

and i am going to

just close the file

go to the open

and we need to go inside the pie then

our folder tensorflow lite iphone

click on it examples

then the light

examples and we need to go here object

detection and here it is what our

raspberry pi folder click on it and then

i need to open the i

ipcam.pi file because we need to change

the ip address

so open it

and this is what the ip address so i am

going to open the ipcam software for

mobile and then i'm going to enter here

my ip address okay

so friends i have started the ip camera

the ip address is same

192.168.0.102. so remember one thing you

need to mention your ipic webcam

software ip address here okay so we are

ready and i am going to start the script

so just close the tony editor open the

terminal and remember one thing we are

now inside the tensorflow lite bullseye

example slide examples object detection

raspberry pi folder and here we are

going to run our custom model so sudo

and our file name is detect ipcm dot pi

okay so

sudo python3

then detect

ipcm.pi

space hyphen hyphen model

space we need to mention our model name

which is the android.t of align and now

simply we need to hit enter then our

camera frame will be start okay so i'm

going to simply hit enter remember one

thing i have already started the ipcam

software from my mobile so if i just hit

enter

so friends as you can see we have

successfully detected our esp8266 and

the pico okay as you can see here

esp8266

and pico with the help of mobile camera

using ipcam and the opencv and also the

tensorflow okay

so this is the reference you can create

our own custom model for detecting the

object with the help of tensorflow lite

on raspbianos raspberry pi 4

so i hope you learned something from

this video we'll meet our next video

till then thank you take care and bye