Illustrated Guide to Transformers Neural Network: A step by step explanation

00:00

transformers are taking the natural

00:02

language processing world by storm these

00:04

incredible models are breaking multiple

00:06

NLP records and pushing the

00:08

state-of-the-art they are used in many

00:10

applications like machine language

00:12

translation conversational chat BOTS and

00:15

even a power better search engines

00:18

transformers are the rage and deep

00:20

learning nowadays but how do they work

00:22

why are they outperformed a previous

00:24

king of sequence problems like recurrent

00:26

neural networks gr use and LS tiens

00:29

you've probably heard of different

00:31

famous transformer models like Burt CBT

00:34

and GB t2 in this video we'll focus on

00:37

the one paper that started it all

00:39

attention is all you need to understand

00:43

transformers we first must understand

00:45

the attention mechanism to get an

00:47

intuitive understanding of the attention

00:49

mechanism let's start with a fun text

00:51

generation model that's capable of

00:53

writing its own sci-fi novel we'll need

00:56

to prime in a model with an arbitrary

00:57

input and a model will generate the rest

01:00

okay

01:01

let's make the story interesting as

01:03

aliens entered our planet and began to

01:07

colonize earth a certain group of

01:10

extraterrestrials begin to manipulate

01:12

our society through their influence of a

01:14

certain number of the elite of the

01:16

country to keep an iron grip over the

01:19

populace by the way I then just make

01:22

this up this was actually generated by

01:24

open AI is GPT to transformer model

01:27

shout out to hugging face for an awesome

01:29

interface to play with I'll provide a

01:30

link in description okay so the model is

01:33

a little dark but what's interesting is

01:36

how it works as a model generate tax

01:38

word by word it has the ability to

01:40

reference or tend to words that is

01:43

relevant to the generated word how the

01:45

model knows which were to attend to is

01:47

all learned while training with

01:49

backpropagation our intends are also

01:51

capable of looking at previous inputs

01:53

too but the power of the attention

01:55

mechanism is that it doesn't suffer from

01:57

short-term memory rnns have a shorter

02:01

window to reference from so when a story

02:03

gets longer rnns can't access word

02:06

generated earlier in the sequence

02:09

this is still true for gr use and L

02:11

STM's although they do have a bigger

02:13

capacity to achieve longer term memory

02:16

therefore having a longer window to

02:18

reference from the attention mechanism

02:20

in theory and given enough compute

02:22

resources have an infinite window to

02:25

reference from therefore being capable

02:27

of using the entire context of the story

02:29

while generating the text this power was

02:33

demonstrated in the paper attention is

02:35

all you need when the author's introduce

02:38

a new novel neural network called the

02:40

Transformers which is an attention based

02:43

encoder decoder type architecture on a

02:45

high level the encoder Maps an input

02:48

sequence into an abstract continuous

02:51

representation that holds all the

02:53

learned information of that input to

02:55

decoder then takes our continuous

02:57

representation and step by step

02:59

generates a single output while also

03:02

being fed to previous output let's walk

03:05

through an example

03:08

the attention is all you need paper

03:11

applied to transformer model on a neuro

03:13

machine translation problem our

03:15

demonstration of the transformer model

03:17

would be a conversational chat bot the

03:20

example with taking an input tax hi how

03:23

are you and generate the response I am

03:25

fine

03:26

let's break down the mechanics of the

03:29

network step by step the first step is

03:32

feeding our input into a word embedded

03:34

layer a word embedding layer can be

03:36

thought of as a lookup table to grab a

03:38

learn factor of representation of each

03:40

word neural networks learned through

03:43

numbers so each word maps to a vector

03:45

with continuous values to represent that

03:47

word

03:50

next step is to inject positional

03:52

information into the embeddings because

03:55

a transformer encoder has no recurrence

03:57

like recurrent known networks we must

04:00

add information about the positions into

04:02

the input embeddings

04:04

this is done using positional encoding

04:07

the authors came up with a clever trick

04:09

using sine and cosine functions we won't

04:12

go into the mathematical details of the

04:14

positional codings in this video but

04:16

here are the basics for every odd time

04:20

step create a vector using the cosine

04:22

function for every even time step create

04:25

a vector using the sine function then

04:28

add those vectors to their corresponding

04:30

embedding vector this successfully gives

04:33

the network information on two positions

04:35

of each vector the sine and cosine

04:39

functions were chosen in tandem because

04:41

they have linear properties the model

04:43

can easily learn to attend to now we

04:47

have the encoder layer the encoder

04:50

layers job is to map all input sequence

04:52

into an abstract continuous

04:54

representation that holds the learned

04:57

information for that entire sequence it

04:59

contains two sub modules multi-headed

05:02

attention followed by a fully connected

05:04

network there are also residual

05:07

connections around each of the two sub

05:09

modules followed by a layer

05:11

normalization to break this down let's

05:14

look at the multi headed attention

05:15

module multi-headed attention Indian

05:20

code applies a specific attention

05:22

mechanism called self attention self

05:25

attention allows a model to associate

05:27

each individual word in the input to

05:30

other words in the input so in our

05:32

example it's possible that our model can

05:34

learn to associate the word you with how

05:37

M are it's also possible that the model

05:39

learns that word structured in this

05:41

pattern are typically a question

05:43

so respond appropriately to achieve self

05:47

attention we feed the input into three

05:49

distinct fully connected layers to

05:51

create the query key and value vectors

05:54

what are these vectors exactly I found a

05:57

good explanation on stock-exchange

05:58

stating the query key and value concept

06:03

comes from the retrieval system for

06:05

example when you type a query to search

06:07

for some video on YouTube

06:08

the search engine will map your query

06:10

against a set of keys for example video

06:13

title description etc associated with

06:16

candidate videos in the database then

06:18

present you with the best match video

06:20

let's see how this relates to self

06:22

attention the queries and keys undergoes

06:27

a dot product matrix multiplication to

06:29

produce a score matrix the score matrix

06:32

determines how much focus should a word

06:34

be put on other words so each word will

06:37

have a score to correspond to other

06:39

words in the time step the higher score

06:42

the more the focus this is how queries

06:44

are mapped to keys then the scores get

06:48

scaled down by getting divided by the

06:50

square root of the dimension of the

06:52

queries and the keys this is to allow

06:56

for more stable gradients as multiplying

06:58

values can have exploding effects next

07:01

you take the softmax the scaled score to

07:04

get the attention weights which gives

07:05

you probability values between 0 & 1

07:08

by doing the softmax the higher scores

07:10

get heightened and the lower scores are

07:12

depressed this allows the model to be

07:14

more confident on which words to attend

07:16

to then you take the attention weights

07:18

and multiply it by your value vector to

07:21

get an output vector the higher softmax

07:23

scores will keep the value of the words

07:25

the model learn is more important the

07:28

lower scores will drown out their

07:29

irrelevant words you feed the output

07:31

vector into a linear layer to process to

07:35

make this a multi-headed attention

07:37

computation you need to split the query

07:39

key in value into adding vectors before

07:42

applying self attention to split vectors

07:45

that goes through the same self

07:47

attention process individually each self

07:50

attention process is called a head each

07:52

head produces an output vector that gets

07:54

concatenated into a single vector before

07:57

go through in a final linear layer in

07:59

theory each head would learn something

08:02

different therefore giving the encounter

08:03

model more representation power okay so

08:07

that's multi-headed attention to sum it

08:09

up multi-headed attention is a module in

08:12

a transformer network that

08:14

you to the attention waits for the input

08:16

and produces an output vector with

08:18

encoded information on how each word

08:20

should attend to all other words in a

08:23

sequence

08:27

next step the multi-headed attention

08:29

output vector is added to the original

08:31

input this is called a residual

08:33

connection the output of the residual

08:36

connection goes through a layer

08:38

normalization the normalized residual

08:41

output gets fed into a point-wise

08:43

feed-forward network for further

08:45

processing the point-wise feed-forward

08:47

network are a couple of linear layers

08:49

with a relict evasion in between the

08:52

output of that is again added to the

08:54

input of the point-wise feed-forward

08:56

network and further normalized the

08:59

residual connections helps the network

09:01

train by allowing gradients to flow

09:03

through the networks directly the layer

09:05

normalizations are used to stabilize the

09:07

network which results in sustained

09:09

producing the training time necessary

09:11

and a point-wise feed-forward layer are

09:14

used to further process the attention

09:16

output potentially giving it a richer

09:18

representation

09:21

and that wraps up the encoded layer all

09:24

these operations is for the purpose of

09:26

encoding the input to a continuous

09:28

representation with attention

09:30

information this will help the decoder

09:33

focus on the appropriate words in the

09:35

input during the decoding process you

09:38

can stack the encoder and times to

09:40

further encode the information where

09:42

each layer has the opportunity to learn

09:44

different attention representations

09:46

therefore potentially boosting the

09:48

predictive power of the transformer

09:50

network now we move on to the decoder

09:54

the decoders job is to generate text

09:57

sequences the decoder has similar sub

09:59

layers as the encoder it has two

10:01

multi-headed attention layers a

10:03

point-wise feed-forward layer with

10:06

residual connections and layer

10:07

normalization after each sub layer these

10:10

sub layers behave similarly to layers in

10:12

the encoder but each multi-headed

10:14

attention layer has a different job it's

10:17

capped off with a linear layer that acts

10:18

like a classifier and a soft Max to get

10:20

the word probabilities the decoder is

10:23

auto regressive it takes in the list of

10:26

previous outputs as inputs as well as

10:28

the encoder outputs that contains the

10:30

attention information from the input the

10:33

decoder stops decoding when it generates

10:34

an end token as an output let's walk

10:37

through the decoding steps the input

10:42

goes through an embedding layer in a

10:44

position on coding layer to get

10:45

positional embeddings the positional

10:48

embeddings gets fed into the first

10:50

multi-headed attention layer which

10:52

computes the attention score for the

10:54

decoders input this multi-headed

10:57

attention layer operates slightly

10:58

different since the decoders

11:01

autoregressive and generates the

11:02

sequence word-by-word you need to

11:05

prevent it from condition into future

11:06

tokens for example when computing

11:10

attention scores on the word am you

11:12

should not have access to the word fine

11:14

because our word is a future word that

11:16

was generated after the word am should

11:21

only have access to itself and the words

11:23

before this is true for all other words

11:25

where they can only attend to previous

11:27

words we need a method to prevent

11:29

computing attention scores for future

11:31

words this method is called masking

11:34

to prevent the decoder from looking at

11:36

future tokens you apply a look-ahead

11:38

mask the mask is added before

11:41

calculating the softmax and after

11:43

scaling the scores let's take a look at

11:45

how this works the mask is a matrix

11:48

that's the same size as the attention

11:50

scores filled with values of materials

11:52

and negative infinity x' when you add

11:54

the mask to the scale attention scores

11:56

you get a matrix of scores with the top

11:58

right triangle filled with negative

12:00

infinity x' the reason for this is once

12:03

you take the softmax of the mask scores

12:06

the negative infinity is get zeroed out

12:08

leaving a zero attention score for

12:10

future tokens as you can see the

12:13

attention scores for M have values for

12:15

itself and all other words before it but

12:18

zero for the word fine this essentially

12:21

tells the model to put no focus on those

12:23

words this masking is the only

12:27

difference on how the attention scores

12:29

are calculated in the first multi-headed

12:31

attention layer this layers still have

12:33

multiple heads that the masks are being

12:35

applied to before getting concatenated

12:37

and fed through a linear layer for

12:39

further processing the output of the

12:42

first multi-headed attention is a mask

12:44

output vector with information on how

12:46

the model should attend on the decoders

12:48

inputs

12:51

now on to the second multi-headed

12:53

attention layer for this layer the

12:56

encoders output are the queries in the

12:58

keys in the first multi-headed attention

13:00

layer outputs are the values this

13:02

process matches the encoders input to

13:05

the decoders input allowing the decoder

13:07

to decide which encoder input is

13:09

relevant to put focus on the output of

13:11

the second multi-headed attention goes

13:13

through a point wise feed-forward layer

13:15

for further processing the output of the

13:18

final point wise feed-forward layer goes

13:21

through a final linear layer that access

13:23

a classifier the classifier is as

13:26

biggest number of classes you have for

13:28

example if you have 10,000 classes for

13:31

10,000 words the output of that

13:33

classifier will be of size 10,000 the

13:37

output of the classifier again gets fed

13:38

into a soft max layer the soft max layer

13:42

produced probability scores between 0

13:44

and 1 for each class we take the index

13:47

of the highest probability score and

13:49

that equals our predicted word the

13:51

decoder didn't taste the output and adds

13:53

it to the list of decoder inputs and

13:55

continue decoding again until end token

13:58

is predicted for our case the highest

14:01

probability prediction is the final

14:02

class which is assigned to the end token

14:04

this is how the decoder generates the

14:07

output the decoder can be stacked n

14:10

layers high each layer taking in inputs

14:13

from the encoder and the layers before

14:15

it by stacking layers the model can

14:18

learn to extract and focus on different

14:19

combinations of attention from its

14:21

attention heads potentially boosting its

14:24

predictive power and that's it that's

14:27

the mechanics of the transformers

14:28

transformers leverage the power of the

14:31

attention mechanism to make better

14:32

predictions recur known networks trying

14:35

to achieve similar things but because

14:37

they suffer from short term memory

14:39

transformers are usually better

14:41

especially if you want to encode or

14:42

generate longer sequences because of the

14:45

transformer architecture the natural

14:48

language processing industry can now

14:49

achieve unprecedented results if you

14:52

found this helpful

14:53

hit that like and subscribe button also

14:55

let me know in comments what you'd like

14:56

to see next and until next time thanks

14:59

for watching