Building a Neural Network with PyTorch in 15 Minutes | Coding Challenge
Summary
TLDRIn this fast-paced video, the host challenges himself to code a PyTorch deep learning model within 15 minutes without referring to any documentation or pre-existing code. He introduces PyTorch, an open-source machine learning library, and proceeds to build a neural network for the MNIST dataset. The video covers model creation, training, and prediction, showcasing the host's ability to meet the time constraint and successfully train a model to recognize handwritten digits.
Takeaways
- ๐ The video is a challenge to code a PyTorch deep learning model within 15 minutes.
- ๐ซ The presenter sets a rule to avoid looking at documentation or using pre-existing code, with a 1-minute penalty for violations.
- ๐ There's a stake involved: if the time limit is not met, the presenter will give a $50 Amazon gift card to the viewers.
- ๐ ๏ธ PyTorch is an open-source deep learning framework developed by Facebook, used for building deep learning models quickly.
- ๐พ The video demonstrates how to import necessary dependencies from PyTorch and TorchVision for model building.
- ๐ The presenter guides through creating a neural network class, setting up the model architecture with convolutional layers, and defining a forward method.
- ๐ The training process is explained, including data loading, setting up the optimizer, defining a loss function, and writing a training loop.
- ๐พ The video includes a practical example of training a model on the MNIST dataset, which consists of images of handwritten digits.
- ๐ The presenter successfully trains the model and saves the weights within the time limit, demonstrating the efficiency of PyTorch.
- ๐ฎ After training, the video shows how to load the saved model and make predictions on new data, showcasing the model's ability to classify images.
Q & A
What is the main challenge the video aims to address?
-The video aims to challenge the presenter to code a PyTorch deep learning model within a strict time limit of 15 minutes without referring to any documentation or pre-existing code.
What are the consequences if the presenter fails to meet the 15-minute time limit?
-If the presenter fails to meet the time limit, they will give away a $50 Amazon gift card to the viewers.
What is the purpose of using the MNIST dataset in the video?
-The MNIST dataset is used for training the deep neural network model as it is an image classification dataset with 10 classes (0-9), which is suitable for demonstrating the capabilities of PyTorch in a short timeframe.
What is the role of the 'transforms' module from 'torchvision' in the script?
-The 'transforms' module is used to convert images into tensors, which are the data format that PyTorch works with for building and training neural networks.
What is the significance of the 'nn.Sequential' API used in the video?
-The 'nn.Sequential' API in PyTorch is used to create a sequence of neural network layers, making it easier to stack layers and build the model architecture.
How does the video demonstrate the process of creating a convolutional neural network (CNN)?
-The video demonstrates creating a CNN by stacking multiple 'nn.Conv2d' layers with specific parameters, such as the number of input channels, output channels, and kernel size.
What is the purpose of the 'nn.Flatten' layer in the neural network architecture presented?
-The 'nn.Flatten' layer is used to flatten the output of the convolutional layers so that it can be fed into the fully connected 'nn.Linear' layers for classification.
How is the training process of the neural network described in the video?
-The training process involves iterating over batches of data, making predictions, calculating loss using 'nn.CrossEntropyLoss', and applying backpropagation to update the model's weights using the 'optim.Adam' optimizer.
What is the significance of the 'cuda' keyword in the script?
-The 'cuda' keyword is used to specify that the model and data should be sent to the GPU for faster computation. If a GPU is not available, it defaults to using the CPU.
How does the presenter ensure the model's performance is evaluated during the training?
-The presenter prints out the loss for each batch after each epoch to monitor the model's performance and ensure it is learning effectively.
What is the final step demonstrated in the video after training the model?
-The final step demonstrated is saving the trained model's state to a file using 'torch.save', which allows the model to be reloaded and used for making predictions without retraining.
Outlines
๐ Introduction to a PyTorch Deep Learning Challenge
The speaker introduces a challenge to code a PyTorch deep learning model within a 15-minute time limit. They outline the rules: no prior code or documentation can be referenced, and using copilot is not allowed due to an expired subscription. A one-minute penalty is applied for breaking these rules. If the time limit is not met, a 50 Amazon gift card is promised to the audience. The speaker briefly explains PyTorch as an open-source deep learning framework developed by Facebook, used for building deep learning models quickly and efficiently.
๐ป Setting Up the PyTorch Neural Network
The speaker begins coding by creating a Python file and importing necessary dependencies from PyTorch, including the neural network class, Adam optimizer, and data loader. They also import datasets and transforms from torchvision to handle image data. The speaker then proceeds to download the MNIST dataset, a collection of images for handwritten digit classification, and sets up data transformations to convert images into tensors. A data loader is created to batch the data into sets of 32 images. The speaker then defines an 'image classifier' class, inheriting from PyTorch's neural network module, and sets up the model architecture using convolutional layers and a linear layer for classification.
๐ Rapid Development of the Image Classifier
The speaker continues to develop the 'image classifier' by defining the model's layers, including convolutional layers with varying numbers of filters and kernel sizes. They calculate the output size after applying the convolutional layers and adjust the input size for the linear layer accordingly. The speaker also sets up the forward method for the model, which passes input data through the defined layers. They create an instance of the classifier, move it to a GPU if available, and define the Adam optimizer with a specified learning rate. A cross-entropy loss function is also instantiated. The speaker then outlines the training loop, which includes unpacking data batches, sending data to the GPU, making predictions, calculating loss, and performing backpropagation.
๐ Completing the Training and Saving the Model
The speaker wraps up the training process by running the training loop for 10 epochs, printing the loss for each batch, and saving the trained model weights to a file. They express excitement as the training completes without errors and within the time limit. The speaker then demonstrates how to load the saved model and make predictions on new images, showcasing the model's ability to classify handwritten digits correctly. They conclude by thanking the audience and encouraging feedback, promising to share the code on GitHub for those interested in trying the challenge themselves.
๐ Conclusion and Engagement
In the final paragraph, the speaker invites viewers to like, subscribe, and turn on notifications for more content. They express gratitude for the viewers' engagement and ask for feedback or suggestions for future challenges. The speaker also inquires if the audience enjoyed the PyTorch deep learning challenge and if they would like to see more of this type of content. The video ends with a positive note, encouraging viewers to participate and learn from the shared experience.
Mindmap
Keywords
๐กPyTorch
๐กDeep Learning
๐กNeural Network
๐กConvolutional Neural Network (CNN)
๐กOptimizer
๐กLoss Function
๐กBackpropagation
๐กEpoch
๐กBatch
๐กTensor
๐กGPU
Highlights
Introduction to a challenge of coding a PyTorch deep learning model in 15 minutes.
Setting a 15-minute time limit for building the model without referencing documentation or pre-existing code.
A one-minute penalty is applied for using documentation or pre-existing code.
The presenter is not using Copilot due to an expired subscription.
Failure to meet the time limit results in a 50 Amazon gift card giveaway.
PyTorch is an open-source deep learning framework developed by Facebook.
Importing necessary dependencies for building the neural network.
Downloading the MNIST dataset for image classification.
Creating a neural network class called 'image classifier'.
Stacking layers for the model using PyTorch's sequential API.
Using convolutional neural network layers with specific parameters.
Adjusting for the reduction in image size after applying convolutional layers.
Creating a linear layer to output predictions for the 10 classes of the MNIST dataset.
Instantiating the neural network, optimizer, and loss function.
Setting up a training loop with backpropagation.
Printing out the loss for each batch during training.
Saving the trained model to the environment.
Successfully training the model and making predictions within the time limit.
Loading the trained model and making predictions on new images.
The presenter expresses happiness and satisfaction with the successful training and prediction.
Encouragement for viewers to try the challenge and provide feedback.
Transcripts
so i looked at the pie torch
documentation for the very first time
yesterday and in this video we're going
to be trying to code a pie torch deep
learning model in 15 minutes
[Music]
this is gonna be interesting what's
happening guys welcome back to another
episode of
code that where i try to build stuff in
a ridiculously short time frame in this
episode as i mentioned we are going to
be building a deep neural network using
pie torch
very quickly so what are the rules well
first and foremost we're going to have a
time limit as per usual the time limit
is going to be 15 minutes now we need
some constraints in this particular case
i'm not going to be allowed to look at
any documentation or pre-existing code
also i'm not gonna be able to use
copilot to be honest my subscriptions
expired anyway so that doesn't really
matter if i do go and use existing code
or documentation it is a one minute time
penalty off our total time limit we're
also going to need
some steaks if i don't make that time
limit it is going to be a 50 amazon gift
card to you guys now the last two
episodes we haven't quite hit that time
limit so who knows we're gonna have to
try to make a bit of an effort in this
one before we jump right into it we
should probably take a little bit of a
look as to what is pie torch pie torch
is an open source deep learning
framework which is predominantly used in
python it's developed by the team at
facebook and it accelerates the speed at
which you can go and build deep learning
models it is used quite heavily in
state-of-the-art models as well as in
deep learning research ready to do it
let's get to it
alrighty guys let's get into it 15
minutes on the clock
let's go
okay so the first thing that we need to
do is create a new python file so i'm
just going to call it uh
torch and then dot pi
okay then we need to import a bunch of
dependencies so i've already gone
and installed pi torch on this
particular machine but we need to import
some pie torch dependencies so let's go
for it so uh from
i torch actually from torch well i'm
already screwing up from torch import
and then
from
porch.optimum import adam
from torch.utils.data
import what is it uh data loader
all right cool so those are so torch so
this most of our neural network classes
are going to be inside of the neural
network class atom is going to be our
optimizer and data loader is needed to
load a data set from pytorch then we
need a bunch of dependencies from torch
vision so from torch vision
watch oh my gosh
everything torch vision uh import data
sets and then from torch
vision so we need this next dependency
to convert our images into tensors which
are going to work with pytorch so from
torch vision dot
transforms
import uh what do we need
through tensor
boom okay let's close this for now all
right so those are our dependencies
imported all we now need to do is import
our data set so we don't have any data
sets in our folders but some images that
have already gone and downloaded which
we'll be able to test out later on so
let's download our data set so we're
going to use the mnist data set which is
an image classification data set there
are 10 classes so 0 to 9 so we're going
to be able to download that using the
data sets
class over here so train equals
datasets.mnist
and then we need to specify where we
want to download it so root equals and
then the folder so data
and then we're going to specify download
equals true because we need to download
it and then we also need to download we
want the train partition
set that equal to true
and then we need to specify any data
transformations that we want to go ahead
and apply so we're going to specify
transform
and we want to transform it to a tensor
and then we are going to create our data
set so data set equals
data loader so we need to pass our train
partition so this over here to this data
load us to train and then specify how we
want to batch up our data set so we're
going to
convert it into batches of 32 images
cool so this is
that's right two comments so import
dependencies
get data
all right now we actually need to create
our neural network class so
the first thing that we're going to do
is create a class and we're going to
call it image classifier
and we are going to subclass it from the
neural network.module class from pi
torch then we need two functions or two
methods so definite
and we also need a forward
a forward method this is akin to
the call method inside of
tensorflow
all right so we've initialized our first
method so now what we need to do is
create a model so this is where we
actually stack all our layers so we are
going to call self.models we'll create a
new attribute equals nn.sequential so
we're going to be using the sequential
api from pytorch so it's coming from up
here
and then what we want to do is we're
going to use convolutional neural
network layers so uh so nn.conf 2d
and then we need to specify what this is
intense all right so the input channel
is going to be one because our images
are black and white then we want 32
filters of shape 3x3 or 32 kernels
all right so that's our one
convolutional neural network layer then
we need some activations
to handle non-lyrics yeah
non-linearities
and then uh what are we doing so we're
going to add in a bunch more
convolutional neural network layers so
we now need 32 as our input channels and
then we're going to output 64.
and then let's do one more set of
convolutional neural network layer or
conv conv layers and then we are going
to be taking 64 from up here
now without getting too deep into the
math by applying each one of these
layers with these specific parameters
we're going to be shaving off two pixels
off the height and width of each image
so we need to adjust for that in our
output when we apply a linear layer so
first up we need to flatten these layers
and then we're going to apply uh
creating linear layers so nn dot linear
and our shape so we need to pass through
the input shape into the linear layer so
it's going to be 64 channels because
that's the final channel output from our
last convolutional neural network layer
multiplied by our images from mnist are
in the shape of one comma 28 comma 28.
so if we're applying
three convolutional neural network
layers with these specific parameters
this specific stride and whatnot we're
gonna be shaving off two pixels each
time so we're effectively going to be
shaving off two
two and then two so it's minus six so
it's going to be 68 multiplied or minus
6
sorry 28
minus 6 then 28 minus 6 over here and
then our output shape needs to be the
number of classes so out we've got 10
different classes so 0 to 9
classes 0 to 9 so we need 10 outputs
beautiful okay i think that's our neural
network layer let me just quickly take a
look at that um all right wait we need
to subclass this model
so we are going to call super
dot
init
and that looks good and then our forward
function or method is pretty simple so
it's just going to
take in our current instance and we're
going to take in our x data and we are
going to return
self.model and then we're going to pass
through our x value into that all right
how we're looking how are we looking for
time all right six minutes of we need to
accelerate okay so that's our image
classifier class
uh neural network
now what we need to do is set up a a
couple of things so we need to create an
instance
of the neural network
uh loss and our optimizer
okay so instance of our neural network
so we're going to call it classifiers to
clf equals image classifier
and then we need to say that we want to
use our gpu so i've installed the gpu
edition of torch on this so it's the
kudo equivalent so we're going to send
it to that device so we can type in dot
2
and then say cuda if you don't have a
gpu then you can set this to cpu
really need to stop talking and stop
writing okay so that's our classifier
cool i think that is good then we need
to instantiate our optimizer
and we uh imported our optimizer from up
here i don't know why this turtle thing
imports so our optimizer is atom so
we're going to be using atom
and then to that we need to pass our
classifier and the parameters
and then we need to specify our learning
rate learning rate adjusts how fast or
slow our neural network is going to
learn so we're going to specify lr
equals 1 e negative 3.
cool all right so that's our optimizer
then we need to create a loss function
so loss function equals
uh nn dot cross entropy
loss
beautiful okay so that's our
neural network done that's our optimizer
done that's our loss function done all
right now what we need is a training
function we are we're gonna make this
seven minutes oh yeah we're gonna make
this all right um
okay so training
uh flow so we're just gonna do it inside
of like our typical function or like
python function here
that is still recording cool all right
um
so if
breathe breathe
remember your pressure point
remember your pressure points okay if
name
equals equals
main i finally want to make this
hit the code that time limit all right
so if name let's get rid of that equals
main then what do we want to do so we
want to run our deep learning training
for a number of epochs that's
effectively going through all of our
batches so
if actually four epoch in range
range 10
so this means we're going to be training
for 10 epochs
train for 10 epochs
then for batch in
data set so we're going to be looping
through
this over here
then what we need to do is unpack the
data that we're getting there so x and y
are going to equal batch so we're going
to unpack that data and then we need to
send the x and y values to our gpu again
so it's going to be x comma y equals
x dot 2.
cuda again if you've got a cpu only it's
going to be true cpu
y dot 2
cuda
so this is a little different compared
to when you're working in tensorflow um
all right then we need to make a
prediction so y hat equals
clf and then pass through x
to generate a prediction yes that makes
sense then we need to calculate
loss so this is how you typically go
through and actually build or build a
flow input intense flow you just
effectively go compile and then dot fit
all right so we need to calculate our
loss so our loss is going to be our loss
function
plus function and then we are passing
through y hat and then y so that
calculates our loss and then we actually
need to go and apply backdrop apply
back prop
so first what we need to do is we need
to zero out any existing gradients so we
go opt.zero grad
then we need to take or we need to
calculate those gradients so loss got
backward
and then we need to go and take a step
and effectively uh
apply gradient descent so opt dot
visit op dot step
cool i think that's okay guys all right
so then what we want to do is we want to
print out our loss for every batch so
we are getting close print
f and then we are going to
the epoch
so i'm just going to print out the epoch
so we're going to print out what epoch
we're currently up to and then we're
just going to print out loss
loss is
and then we'll push loss dot item
here
then close that close that all right
that looks good
i think we're okay so what are we doing
so let's just quickly check so we've got
our dependencies
utensil that's fine data loader
we've gone and subclassed our model yes
that's fine
we've got our forward
method
instantiated all of this stuff
gone and sent our data to the gpu gone
and applied back prop we're then going
to print out our epoch print out our
lost oh last thing so we want to save
down this model to our
environment right so we actually want to
go and save it here so let's go and save
that so um with open we're going to call
it uh
model state
dot pt
and then we're going to write binary
as f and then we need to oh actually we
need to import some more stuff so we
need to download we need to import save
and then load from torch
so we are going to
save
uh
what's the format now
it's model it'll be clf dot state it
and then we are saving it as f i think
that's right
okay let's give it a crack all right so
then we can run to run this we're just
going to run python and then torch n.pie
so python
and if we successfully kick off training
we'll pause the timer because come on
guys you got to give me that at least
all right uh so let's run this
[Music]
okay no errors so far this is looking
promising
all right so it's downloading the data
set
so we should have our data set popping
up here you can see inside of data
if we see a lost metric i'll pause it
come on
this is killing me
yes all right
guys
we started trading
oh man
yes i think we're going to finally make
this first coat that so you can see
fewer that we are successfully printing
out at epoch loss so epoch zero loss is
0.02 then we're going to 0.01 then
0.0002
and once we finish training you should
effectively see that we get our
porch weight saved
the intensity
guys
you can see that our neural network is
currently training right
oh i'm so happy and then we'll make a
prediction i'm i'm counting this as a
dub we hit it in what
30 less than 13 minutes
so what 12 minutes and 54 seconds that's
got to be a new record
got all the the gangster deep learning
practitioners out there gonna be like
nick you skipped over a ton of details i
was like come on i had 15 minutes anyway
we are chugging along
so i effectively once our model gets to
epoch
what is that epoch 9 we should
have our weight saved out because we
went and wrote this
and i'm going to share all this code as
usual code should be in the description
and you can give this a crack yourself
all right cool we didn't get any errors
so boom
we've got to play some angel music in
here
oh guys your boy is happy
so that is successfully how to go and
build a neural network with pytorch so
we did it
i'm happy now so if we wanted to we
could actually go and load up
this um neural network and make a
prediction which is why i've got these
images here so
should we try to get this done in two
minutes
i'm a little nervous all right let's do
it so we've got two minutes left on the
clock
let's try to get the predictions done
okay so we've successfully loaded
downloaded or trained our model so now
what we can do let me just minimize this
so now what we can do is comment this
stuff out so we don't need to go and
train again
and then what we'll do is we'll load up
so we've got a classifier there so we'll
then go with
open
and then we want to load up this so
model state dot pt
so with open model state dot pt
read binary
as f and then we are going to go load
f and then we want to go clf dot load
i think it's load state dick yep cool so
that's going to load the weights into
our classifier then in order to make
predictions we need to import
torch
torch up here and we also need to import
uh pillow to load our image so then what
we can go ahead and do
is load in our image so image equals
kill i'll actually wait at some point so
from
kill import
image
so then we're going to import our image
to image
dot open
and we're just going to open up image
one right now which is a number two so
uh image underscore one
dot jpg
and then what do we need to do so we
need to convert that to a tensor so
we're going to go uh image answer
equals
true tensor
pass through our image and then we need
to unsqueeze it because we want to pass
through a single sample so we're gonna
unsqueeze and then send it to our gpu so
cuda
and then we should be able to print
classifier and then image tensor
i think that might be okay
all right let's run it
and this should print out our prediction
oh we need to uh so torch dot ugg max
i'm gonna take this as a win guys
right that's the timer up though
so if i now go and run this
boom it's predicting two
how good is that
i'm gonna take that as dub guys because
we've managed to train that model and
make predictions so if you take a look
now right i can then go and load up
image two
all i need to do is change the image
here
and it should print out the class
the tens of zero so image if we go and
open up image two
boom that's tense to zero so it's
predicting the the class is zero
we go and run it for let's go and do it
for image three
which is a number nine so if we go and
pass through image three here
boom number nine
thank you so much for tuning in guys
hopefully you've enjoyed this speed
running building pie torch neural
networks code's going to be in the
description via my github if you guys
want to take a crack i'll catch you in
the next one peace thanks so much for
tuning in guys hopefully you enjoyed
this video if you did be sure to give it
a big thumbs up hit subscribe and tick
that bell it really does help and i
really do appreciate you checking out
this video if there's any feedback any
challenges or anything you'd like to
hear a little bit more about do let me
know in the comments below thanks again
for tuning in guys also let me know if
you like this challenge and uh if we
could do better next time thanks again
for tuning in peace
[Music]
Browse More Related Video
Neural Network Python Project - Handwritten Digit Recognition
Introduction to PyTorch
ะขะฒะพั ะะะ ะะะฏ ะะะะ ะะกะะขะฌ ะฝะฐ Python ั ะฝัะปั! | ะะฐ 10 ะผะธะฝัั :3
Prepare your dataset for machine learning (Coding TensorFlow)
ใใฅใผใฉใซใใใใฏใผใฏใฎๆง่ฝใๆฑบๅฎใฅใใใใผใฟใฎ้ใจ่ณช
Welcome to the Hugging Face course
5.0 / 5 (0 votes)