Hello and welcome to this path into Torro now we're going to make the forward function that will forward
propagate the signal through all the brain from the very beginning with the input images up to the outputs
which will contain the key values for the actor and the de-value that is value taken by the function
for the critic.
So it's going to be quite similar as what we did for Dume but this time something is going to change.
Amending is going to change that.
Now we have an illustration in the brain.
So we all have to do something more to propagate the signal and be careful with that.
And the other thing less important but still that changes compared to before is that we're not going
to use a real activation function as you know the nonlinear activation function but we're going to use
the loo which is kind of a more sophisticated really function.
You will see that in the past we're talking mentation.
So let's do this let's make this function we start with a death.
It's actually the last function of this execrated class.
So we're going to call it forward like Einstein and this fourth function is going to take self the object
because we're going to use the objects and the inputs.
So important to understand what will these inputs be.
This will not only be the input images these inputs will also contain the hidden nodes and the cell
nodes of the same.
So that's why I wanted to highlight that some things are going to change now.
Basically we're considering the forward function the Internet and cell nodes of the list.
And speaking of them now what we're going to do is separate these two inputs of this argument in putting
the forward function and how can we separate them.
Well we can recall a new Voivode which will be the inputs images.
So that's designed input images and we separate them with the topple H x and C X which is a temple of
the hidden states and the cell States at the Coliseum.
So H x other states and C X are the South states.
All right and that will be equal to the input that is this argument here.
So now we made that separation and therefore we can start to propagate the signal throughout the brain
and to do that we are going to get successively different layers from the first one to the last one
by using our connections that is the convolutions the LACMA connection and the linear connection here
the full connections.
So let's do this.
Now it's going to be the same as before.
We're going to get our first layer that we're going to call X and get this first layer we need to propagate
the signal from the inputs to this first layer and therefore we need to use the first convolution because
it is the first convolution that propagates the signal from the input images to the first layer.
So what we're going to do now is copy this because this is the first convolution we had here and we
apply this first convolution to our input images which are now the right inputs and that we get that
propagates the signal from the input images to the first layer.
But now remember we have to use a nonlinear activation function to break the linearity in order to be
able to learn the non-linear relationships inside images and to do this we are going to use as we said
the activation function that we're about to see right now and that by touch stuck.
But before that let's get it.
So to get it it's like really we take the functional module which has a shortcut and then that and then
a loop and then we put all this in parentheses because we want to nonlinearly activate the neurons.
This first layer here that we obtained by applying the first convolution on the inputs.
So now let's go to the PI torch Doug to understand what it is.
Here it is.
So you can access it to torch stuck or slash slash.
And then that H G and L and then you have to find non-linear activations and then the nonlinear activation
functions you will find.
Well really that's the classic when we know it's just a max of zero and X you have the graph in mind.
Then you have six which is this one.
So a little more sophisticated.
And then we go we have a look.
And as you can see blue is a redo plus an additional element.
So it's like a more sophisticated really.
And so that's the one we use to nonlinearly activate the neurons and the different layers.
And by the way this new activation function is called the exponential in their unit.
So there we go.
We apply the elu on the first convolutional layer.
And now things are going to be easy.
We're going to proceed to the next for propagation of the signal which is from the first convolutional
layer to the second convolutional layer which we are going to call X because basically we're just adding
X now x is the first convolutional there.
And by propagating the signal from the first convolutional layer to the next one X will become the next
convolutional layer.
And so to propagate the signal from the first convolutional layer to the second one we can't simply
copy this and paste it here and replace one by two.
And now of course the second convolution is not applied to the input images but to X that is the first
convolutional layer which is right here.
All right perfect.
Now we get our second convolutional there.
And now let's propagate the signal again from the second convolutional there to the third one.
And therefore we can directly copy this and paste it here and replace two by three.
There we go.
And last one.
Now to propagate the signal from the third convolutional there to the fourth one and last one we can
just copy this again here and replace three by come four.
There we go.
So let's recap.
We start with our inputs.
We apply the first convolution to get the first convolutional there.
Then we apply the second convolution to the first convolution layer to obtain the second convolutional
layer.
Then we apply this third convolution to the second convolutional there to obtain the third convolutional
layer.
And finally we apply the fourth convolution to the third convolutional there to obtain the fourth convolutional
there.
And that's how the signal is propagated throughout the eyes of the eye.
So there we go we have now the output signal after the four convolutions and now we know what to do
we need to expand this whole output signal in one dimensional vector.
That's the flattening step.
So we're going to update x again x.
Now will become this flattened one dimensional vector.
And to do this that's the same we need to take X which is so far the fourth convolutional layer X but
then we use a view function and we first put minus one to say that we want one dimensional vector and
then as a second argument we need to put the number of elements in the vector and that is remember three
two times three times three.
And therefore we can put here 32 times three times three.
There we go.
Now we have our flattened vector and the flattening step is done.
Perfect.
Now let's take care of the LCN part.
So as you understood the LSD takes as input the flattened vector.
This one dimensional vector of three two times three times three elements.
So it's already ready and well-prepared for the icn.
The team is now ready to take this Flaten vector as inputs and therefore we can take our LACMA and inputs
as argument first X are ready.
FLATOW And vector that is this x ray here that we just expanded.
But also and that's where the trouble comes into play.
We need to put H x and z x.
And we can use ha-Satan see X here because we made that separations from the original input arguments
of the forward function.
So LACMA X the platens output vector after the four convolutions and all of the hidden in the cell note.
So there we go.
Then we must not forget the self because LSD is a variable or function so a variable attached to the
object itself and a CM and this will actually return to outputs a total of two outputs which will be
the output nodes and the output sounds.
So it's actually topical and therefore we can update H.
X y and z x the cell nodes because that's exactly the output of this NCM here.
Great.
So we're almost done now.
Now that we have the outputs of the illustration we need to get the useful output because actually only
the hidden notes are useful and therefore we going to get it by then X again and X will now be equal
to atax the first element of the output sample of the US an X equals.
And we're almost done.
Remember that we have two brains one brain for the actor in one room for the critic.
And therefore we have to output signals to return the output signal of the actor and the output signal
of the critic.
And therefore now what we're going to do is return these two output signals and how can we do that.
Well that's very easy.
We simply need to take our linear connections but separately that is a linear connection of the critic
and that the full connection of the actor.
And we applied each of these connections to the output X that is a useful output of the LACMA and that
will be all that will be the output signal.
So there we go.
Let's do it.
We first take self or object then we get the linear connection of the critic which is critic and this
colinear which we apply to X the output signal the CM and then same we take self again then that and
then we take the linear connection of the actor which is actor and it's colinear are which same we apply
to x.
There we go so that's the main thing we need.
But then we're also going to return the top of Ajax to a node and see X to sell node because we will
be using them later and the retro look of the LCN.
All right perfect.
So now we're done with the brain.
Or should I say the brains because we actually made two brains one for the actor and one critic.
So congratulations for making the eighth with the brains.
I hope that wasn't too overwhelming to combine at CNN and in Anniston but at least the good news is
that we're really working with the best and most powerful model.
So there we go.
We're actually done with this first family model that.
Y.
And so in the next two toile we'll take care of the optimizer because we're going to make a separate
optimizer.
We're not going to cut each line of code because a lot of it comes from the research papers and this
is actually pretty specific.
And if we go into the deep details of what's going on with this optimizer this might be a little too
overwhelming for what's going to happen next because we still have the train function to make which
will be a huge function and that contains the algorithm that we see.
So trust me you want to keep some energy for that.
And therefore we will not spend too much time on this.
But still I will expand the code and you will understand the whole idea behind this optimizing.
So congrats again for making this activity class and I'll see you in the next Statoil 290 optimizer.
Until then enjoy I.