Tutorial 23- Operation Of CNN(CNN vs ANN)

Summary

In this 23rd tutorial of the deep learning playlist, Krishna apologizes for the delay in uploading due to technical issues and promises continuous uploads to complete the series within two weeks. The video focuses on the operations of CNNs, contrasting them with ANNs. It explains the convolution operation, the role of filters (kernels) in edge detection, and how weights are updated during backpropagation. Krishna also hints at discussing max pooling and the concept of 'location invariance' in future videos. For those interested in transitioning to data science, Krishna recommends a resource from Springboard India.

Takeaways

• 😀 The speaker, Krishna, apologizes for the delay in uploading the 23rd tutorial in his deep learning playlist.
• 📈 The tutorial focuses on the operations of CNNs (Convolutional Neural Networks) and their differences from ANNs (Artificial Neural Networks).
• 🔍 Krishna explains the convolution operation, detailing how filters or kernels are used for tasks like edge detection in images.
• 👓 The script mentions that the speaker has previously discussed padding and convolution operations in earlier tutorials.
• 💡 The tutorial aims to clarify how weights in CNNs are updated, particularly after the convolution operation.
• 📊 Krishna uses diagrams to contrast the operations of ANNs and CNNs, highlighting the multiplication of weights and inputs followed by an activation function in both.
• 📚 The speaker plans to cover max pooling in an upcoming video, which is another important operation in CNNs.
• 🧠 The script touches on the concept of 'location invariant' in CNNs, inspired by how human brains recognize features in images regardless of their location.
• 🔗 Krishna provides a link to a resource for those interested in transitioning to data science or seeking advice from real-world data scientists.
• 🎓 The tutorial concludes with an invitation to subscribe to the channel and a promise of more in-depth discussion on max pooling in the next video.

Q & A

What is the main topic of the 23rd tutorial in Krishna's deep learning playlist?

The main topic of the 23rd tutorial is the operations of CNN (Convolutional Neural Networks), focusing on the basic differences between ANN (Artificial Neural Networks) and CNN operations.

Why did Krishna skip many days before uploading this tutorial?

Krishna skipped many days due to some problems he had to resolve, which involved setting up his recording environment properly.

What is the purpose of the filters or kernels in a CNN?

Filters or kernels in a CNN are used for tasks such as horizontal and vertical edge detection, and they help in feature extraction from images.

How does the convolution operation update the filters in a CNN?

The convolution operation updates the filters through the backpropagation process, where the weights inside the filter are learned and updated based on the output and loss calculated.

What is the role of the activation function in both ANN and CNN?

The activation function in both ANN and CNN is used to introduce non-linearity into the model. It is applied to the output of each neuron after the weighted sum and bias addition to introduce complex patterns.

What is the significance of the term 'location invariant' mentioned in the script?

The term 'location invariant' refers to the ability of a CNN to detect features at different locations in an image, making the model robust to the position of the object within the image.

Why is max pooling important in CNNs according to the script?

Max pooling is important in CNNs because it helps in reducing the spatial dimensions of the output from the convolution layer, which in turn reduces the number of parameters and computation in the network, thus controlling overfitting.

What is the difference between the operations in ANN and CNN as described in the script?

The main difference lies in how the networks process data. While ANN uses a full connection between layers, CNN uses filters that slide over the input to perform convolution operations, which is more efficient for image data.

How does the script explain the process of a convolution operation with an example?

The script explains the convolution operation by taking an example of a 3x3 filter applied to a part of an image, demonstrating how the filter values are multiplied with the image pixels and summed up to produce a single output value in the feature map.

What advice does Krishna offer for those looking to transition into data science?

Krishna suggests visiting a channel called 'Springboard India' for discussions and advice related to transitioning into data science and becoming a real-world data scientist.

What is the significance of stacking multiple convolutional layers in a CNN?

Stacking multiple convolutional layers allows the network to learn hierarchical features, where each layer can detect increasingly complex patterns, starting from edges to more abstract features like shapes and textures.

Outlines

00:00

😀 Introduction to Deep Learning and CNN Operations

Krishna introduces himself and apologizes for the delay in uploading the 23rd tutorial in his deep learning playlist. He explains the reason for the delay was to improve the setup for the tutorials. Krishna promises to upload videos continuously to complete the deep learning series within two weeks. The video focuses on the operations of Convolutional Neural Networks (CNNs) and aims to highlight the differences between CNNs and Artificial Neural Networks (ANNs). Krishna provides a visual comparison between an ANN and a CNN, explaining the convolution operation, weight updates, and the role of filters (kernels) in detecting features like edges in images. He also mentions previous videos on padding and convolution operations and encourages viewers to seek his advice on transitioning into data science, promising more information in the video.

05:00

🔍 Deep Dive into CNN's Convolution and Backpropagation

This paragraph delves deeper into the convolution operation within CNNs, explaining how filters are used to detect features like edges in images. Krishna describes the process of applying filters to an image and how the output image is generated through convolution. He emphasizes the importance of understanding what happens after the convolution operation, particularly during the backpropagation stage where weights are updated based on the loss calculated from the output. Krishna also touches on the concept of stacking convolutional layers horizontally, like in the human visual cortex, to detect more complex features in images. He assures viewers that the basic operations in CNNs, such as weight multiplication and activation function application, are similar to those in ANNs, but adapted for image data. The paragraph concludes with a teaser for the next video, which will discuss max pooling and its significance in CNNs.

10:03

👨‍🏫 Practical Advice for Aspiring Data Scientists

In the final paragraph, Krishna shifts focus to provide practical advice for those looking to transition into data science. He recommends visiting a channel called Springboard India for insightful discussions with real-world data scientists. Krishna encourages viewers to subscribe to his channel and expresses his hope that they found the video helpful. He signs off, promising to cover the max pooling layer in his next video, and wishes viewers a great day.

Keywords

💡Deep Learning

Deep Learning is a subset of machine learning that focuses on artificial neural networks with multiple layers, or 'deep' architectures. In the context of the video, the speaker is discussing a tutorial series on deep learning, indicating that the video's theme revolves around teaching viewers about the intricacies of deep learning models. The script mentions that the tutorial is part of a 'complete deep learning playlist,' suggesting a comprehensive exploration of the topic.

💡Convolutional Neural Network (CNN)

A Convolutional Neural Network is a type of deep learning model predominantly used for image and video recognition tasks. The video script highlights CNN as the main subject, contrasting it with traditional artificial neural networks. The speaker discusses the operations of CNNs, emphasizing their use in detecting features such as edges in images, which is a key aspect of image processing and computer vision.

💡Artificial Neural Network (ANN)

An Artificial Neural Network is a computational model inspired by the human brain that is designed to recognize patterns. The script uses ANN as a comparative term to CNN, illustrating the fundamental differences in how they process data. The speaker explains that while ANNs multiply weights with inputs and apply activation functions, CNNs use convolution operations and filters to detect features in images.

💡Convolution Operation

Convolution Operation is a mathematical process used in CNNs to apply filters to an image. The video script describes this process in detail, explaining how it is used to detect features like horizontal or vertical edges. The speaker uses the example of a 4x4 image and a 2x2 filter to demonstrate how the convolution operation reduces the image size to 3x3, which is a fundamental concept in understanding how CNNs process images.

💡Filters or Kernels

In the context of CNNs, filters or kernels are small matrices used to apply convolution operations on an image. The script mentions that these filters are initialized to detect features like edges in images. The speaker explains that these filters are updated through backpropagation, which is crucial for the learning process in CNNs.

💡Backpropagation

Backpropagation is a method used to update the weights of a neural network by calculating the gradient of the loss function with respect to each weight. The video script refers to backpropagation as the mechanism by which CNNs learn and update their filters. The speaker emphasizes that after the convolution operation and activation function are applied, backpropagation is used to adjust the filter values to improve the model's performance.

💡Activation Function

An Activation Function is a mathematical function used in neural networks to introduce non-linear properties to the model, allowing it to learn complex patterns. The script mentions ReLU (Rectified Linear Unit) as an example of an activation function applied after the convolution operation in CNNs. The speaker explains that applying ReLU to each field in the output of the convolution operation helps introduce non-linearity, which is essential for the network to learn from the data.

💡Max Pooling

Max Pooling is a technique used in CNNs to reduce the spatial dimensions (width and height) of the input volume, making the representation more manageable and less computationally intensive. The video script hints at a future tutorial on max pooling, suggesting that it plays a significant role in CNNs by reducing the dimensionality of the output from the convolution layers, which helps in making the model less prone to overfitting and more translation invariant.

💡Padding

Padding is a technique used in CNNs where extra pixels are added around the border of an image to control the spatial size of the output feature map. The script refers to padding in the context of convolution operations, explaining that it can affect the dimensions of the output. The speaker mentions that without padding, the output size is reduced, but with padding, the original size can be maintained, which is important for preserving the spatial information in the image.

💡Data Science Transition

Data Science Transition refers to the process of moving into a career in data science, which often involves learning new skills and gaining experience in the field. The video script ends with a mention of resources for those looking to transition into data science, suggesting that the speaker is providing guidance and advice for viewers who are interested in pursuing a career in this area. The reference to 'Springboard India' implies that there are educational resources and discussions available to help viewers make this transition.

Highlights

Introduction to the 23rd tutorial in the deep learning playlist by Krishna.

Apology for the delay in uploading the tutorial due to setup issues.

Promise to upload videos continuously and complete the playlist within two weeks.

Discussion on the operations of CNNs (Convolutional Neural Networks).

Explaining the basic differences between ANN (Artificial Neural Networks) and CNN operations.

Illustration of ANN operation with an example of inputs and weights.

Explanation of the convolution operation in CNNs, including edge detection.

Description of the convolution process using a filter on an image.

Differentiation between grayscale and RGB images in terms of pixel representation.

Importance of understanding the output image after the convolution stage.

Introduction to the concept of filters and their role in feature detection.

Explanation of how filters are updated during the backpropagation stage.

Mention of applying ReLU activation function after convolution.

Discussion on stacking convolutional layers horizontally.

Explanation of the concept of 'location invariant' in CNNs.

Promotion of a resource for data science transition and advice.

Invitation to subscribe to the channel for more tutorials.

Transcripts

hello my name is Krishna and welcome to

my youtube channel now this is my 23rd

tutorial on the complete deep learning

playlist I know guys like this was like

it I had actually skipped many days in

order to upload this particular tutorial

there were some problems I went I had to

actually make up this particular setup

and the previous setup was a little bit

rude so now I've made this setup

properly I'm extremely sorry for the

delay now I'll be continuously uploading

videos every with respect to deep

learning and will try to complete within

two weeks so today in this particular

video we'll be discussing about the

operations of CN n that is convolution

neural network and we try to see the

basic differences between an artificial

neural network operation and a

convolution neural network operation so

in this left hand side this is basically

my n and operation and in the right hand

side and basically draw my CN n

operation and we will try to discuss

this we'll try to find out the basic

difference then how the operation

usually takes place in this case how the

weights are getting updated that I have

already made a video regarding that in

my previous videos in previous in in my

past twenty second and twenty first

tutorial I have actually created a video

on padding and actually shown you what

is convolution operation so let us go

ahead and try to see because after

convolution at operation the filters

that we are choosing in convolutional

neural network how this is getting

updated will try to understand that and

one more thing guys if you are looking

for some suggestion towards the

transition of data science or if you

want some advice with related to the

real life data scientist I wish I'll be

sharing some information so make sure

that you watch this particular video

till the end so let us go ahead now over

here let me just consider an artificial

neural network and suppose this is are

my inputs X 1 X 2 X 3 and these are my

weights that gets assigned W 1 W 2 W 3

you know if you have not seen this

videos you can completely go and check

my playlist again the link will be given

in the description now we know that in

the first iteration I did but when we go

to the first hidden neuron in the first

layer we will multiply all the weights

and the inputs right and I am assigning

it as dead and after this we will also

add a bias

after adding bias we apply a raloo or

some activation functions like sigmoid

relu and

and we have already discussed about this

now similarly if I take the same

operation with respect to a CNN you know

that in CNN we have some image it may be

of any size like it may be current in

this particular example I have taken for

cross flow now when I take this image I

can initialize some filters you know

filters or corners these are called as

kernels now why this filter is basically

used it is basically used to you know to

find out the whole horizontal edge

detection vertical edge detection many

things like that right and we have also

discussed about this now the convolution

operation basically says that okay we

have understood that how the correlation

operation takes place we will take this

value suppose over here I have some

values like this 5 6 7 8 9 10 11 - and

these are our pixels right if you know

and if I'm considering this as a you

know great pixel white and black images

I'll have one pixel value right if I

consider this as RGB image then it will

have three channels one is our channel G

Channel and B Channel and each and every

field will be represented with the help

of three values you know the R value

suppose this R value is 128 this G value

is something like 100 this blue value is

basically like car 250 right so when

this all the colors have combined we

basically get a color image but in the

case of grayscale image will just have 1

pixel values now what will happen

suppose this is my this is my filter

okay so suppose this is my values of the

filter and consider that this is the

vertical filter vertical filter

basically means if I apply this

particular filter on to this image I'll

be able to get all the vertical edges in

the output image right and after that

I've also concerned right well I have

also told you if I take an image of 4

cross 4 and 2 cross 2 filter is applied

I will be getting 3 cross 3 if I am NOT

adding any padding or stripes stripes

basically means first of all I place

this filter on top of it over here okay

so this is my 3 cross 3 so this in the

first operation I'll do this I'll get

the value what kind of operation is

happening convolution will try to

multiply each and every value of this

suppose 0 into 1 is 0 okay then in the

second

1 into 2 is 2 right in the 3rd field 0

into 3 is 0 and like this all the

addition will happen and finally we'll

be getting the value for this particular

field and similarly this operation will

happen and that that usually happens in

convolution and I have discussed this in

my previous videos also now the most

important thing to understand over here

is that when we are getting the output

image right what happens after this

particular stage that is pretty much

important to us understand

and always remember guys like this kind

of filters you know suppose this is my

filter 1 we may also have another filter

like f2 we may have another filter like

f3 this filter may be an horizontal edge

detection this may be some like shape

detection something like this kind of

filters will be present now you know

that in air and we will try to update

this weight will try to learn this

weight with respect to the output in the

backpropagation stage you know that

after we get the output we calculate the

loss and after we get the loss what we

do we again back propagate we find out

all the derivative is subtract all these

weights and we update those weights

right so similarly in this particular

case we will try to learn these

sweeteners we will try to update the

values inside this particular filter

with the help of back propagation and

that is the trick over here all the

concepts are same everything is same

right there is one more like suppose I

am getting the output after one

convolution operation there are still

some more operation called as Matt's

pulling I discuss about max pooling in

my next video

but you need to understand that this

filters will have to be learned by my

convolution neural network this values

needs to be updated inside my filter now

once I am getting this particular output

after this I will again go and apply

relu activation function on each and

every field on each and every field I

have to apply the rail u activation

function like how I did it over here you

know so this was my operation right I I

multiplied rates with my inputs in the

NL after I got this value I applied it

I applied an activation function so

similarly over here first of all I

applied filter I did my convolution

operation I got the output after getting

the output for each and every field I

applied my real

activation function right now after when

I do the Ray Lu activation function you

need to understand in the

backpropagation and again I've still not

discussed about Matt's pulling but

Matt's pulling will also be play a very

important role over here so in short

when when I just go ahead and finally

when my back propagation is done this

all values are getting updated a similar

case like how we did the updation over

here considering the last function will

be considering some optimizer same thing

will get applied to this convolution

neural network also so this is pretty

much important to understand guys the

basic difference why I am showing you a

n and because the operation is almost

same you know will multiply the weights

with the inputs and then we'll try to

apply an activation function so

similarly over here in the convolution

operation what will happen we will take

this filter will apply convolution

operation whatever output will go we

will will apply an activation function

each and every band is over here and

then we'll finally get the output so

this is the basic difference in the

operation of CN N and an en n right

after this you know you may also

vertically stack any number of

convolution so this is one convolution

operation right considering the rail you

activation function is one formulation

operation and I can stack this

horizontally you know one after the

other now why why do I have to

horizontally stack also you should

understand that suppose let me take an

example we know that in our brain right

we have various for various regions to

detect some images right suppose I am

seeing a image of a cat support and I

have explained in my previous video I

may have layers like v1 v2 v3 v4 v5 v6

and this is also horizontally stacked

you know after this particular output

will be given to v2 then it will be

given to v3 v4 v5 and finally we will be

able to see the image right so similarly

in this case we can also stack this or

conditioner Electric horizontally one

after the other suppose in the v1

suppose I am seeing a cat over here okay

so this is my cat sorry for my bad

diagram but consider that this is the

cat okay suppose v1 is able to detect

the face okay then

since I have horizontally stacked it

with another convolution over here now

this this this face will go to the next

stage here the more clear phase can be

visible then inside this particular

phase what are the features that can

also be visible right and they may be

different different filters for

detecting eyes they may be different

different filters so we are we can use

maximum number of filters over here and

you know that initially all this values

inside this filter will be randomly

selected you know some values will get

selected and later on with the help of

backpropagation all these values will

get updated right so similarly we can

stack this and this one whole operation

is called as one cognition layer and we

can stack this horizontally after this

I'll also be discussing about what is

max pooling then you learn more

understand about why max pooling is also

used there is a very very good term in

one of the research paper called as

location invariant location invariant

right so very important term in one of

the research paper what it says is that

usually when our human brain see some

faces right some of the neurons get

automatically triggered right and

similarly we should try to make our

convolutional neural network also do

like that so what I'm saying is that

suppose in one of the image I have

multiple cats multiple cats faces then

automatically this convolutional neural

network or the kernels that I'm

basically using should get automatically

triggered you know where it will be able

to detect those multiple faces so that

step will be done by the max pooling

layer okay I will be trying to explain

you that in my next video but just

understand this was the basic difference

between artificial neural network and

the CNN I hope that diagram has got

completely mixed up but I hope I was

going in the right part I have explained

in the right part just keep this in mind

in my next video I'll be discussing

about the max pooling layer now I'll go

what I said in the starting life if you

are looking for transition towards data

science if you're looking for such

barrier advice if you are looking for

some help

respect to some real-world data

scientists talk you can basically go and

visit this channel called a springboard

India the link is basically given in the

description it has wonderful discussion

with respect to different different data

scientists you can have and have a look

onto that the link will basic will be

given in their description guys so I

hope you like this particular video

please do subscribe on channel if you

are not already subscribed I'll see you

in the next video have a great day and

given it up