Hello and welcome to this tutorial.
All right so we're going to start with the first fellow of our moral and the most important fowl that's
muddled up.
Why.
And this is in this file that we will implement the brain of the homo you know the brain at the heart
of the 8:3 Simmo that's in this file that will make the neural network which will of course contain
some convolutional will know when that works because of course we're still doing some deep reinforcement
learning.
So are able to have eyes and inside this neural network will integrate everything that is related to
the active Craigmile and there is a bonus.
As I told you we are implementing one of the most powerful A-3 morals and what makes it that powerful
is that it will contain a record a new network and more precisely in LCN long term memory so that we
can learn the temporal properties of what's going on.
Again that is actually the temporal properties of the input so that the predictions can be even better.
So there we go.
We are implementing a very powerful model that combines basically all the neural networks that we saw
in the deepening course.
That is an artificial neural network a convolutional neural network and a record new network.
And at the heart of all these networks there is of course the A-380 model that will make the AI AI very
powerful.
So let's do this let's attack this model and implemented.
So we're going to start by making two functions.
There are just some functions that will take care of how we can initialize the weights because you know
we're going to have some new networks and therefore we're going to have weights and we just want to
make these two functions for us so that we already have a tool to integrate very easily inside the home
so the neural network.
So these two functions are going to be normalized columns.
Initialiser that is basically a function that can not only initialize some weights but sets a specific
variance of tensor weights.
So that's exactly what we're about to implement right now.
And then we will implement the second function which will be the weights in that function and that will
basically initialize the weights in enough time a way for the learning.
All right.
And then once we're done with these two functions we will start implementing the neural network.
So let's do it let's quickly make these two functions so I'm starting with a def here then I'm going
to give the name of this function which is normalized and these core columns underscore initialiser.
There we go.
And this function is going to take just two inputs.
First it's going to be the weights we want to initialize and the standard deviation because as I just
said we want to set a specific variance for our tensor of weights.
And if you want to understand why we have to do this it's because you know when we make the neural network
that will be the actor and the critic according to the through tomorrow and we'll make two separate
fully connected layers one for the actor and one for the critic.
And these two fully connected layers will have weights and we will set a standard deviation for each
of these two groups of weights.
And so what we'll do is we will set a small standard deviation for the actor.
It will be around 0.01 and a big standard deviation for the critic which will be around 1 I think.
So that's why we're making this function so that we can very easily set the standard deviation for the
weights we will initialize later for the actor and the craic.
That's why we're doing this.
So now we're just going to set a default value but this will change afterwards when we initialize the
weights.
So let's use so for 1.0.
All right.
And now we're ready to define what's inside this function.
So what will first prepare is the output that we're going to call out.
So this out for all is what will be returned by this function.
And so at first what we're going to do is it.
So as you understood this output will be a tensor of weights that will have a specific standard deviation.
But before we take care of setting the standard deviation we just want to initialize it and then we'll
set the standard deviation here which is the argument which is the input of this function.
So out into initialize tensor weights you might know how to do it.
We already did it.
We're going to use our torche library and from this torche library we will take the round and function
which will initialize the torched answer with random weights that follow a normal distribution.
So that's why it is called rant.
And as for normal.
And so now what we simply need to input is the number of elements that distention will contain.
And this number of elements is of course the number of weights because we were actually initializing
a tensor for these weights here.
And so to get this number of elements we can simply take our weights and add dot to get size with parenthesis.
And this will give the number of elements and weights so that it will create the torch tensor of the
same number of elements of our weights and it will be initialized with random weights following a normal
distribution.
All right.
And now it is time to set the standard deviation we want to have that is the standard deviation here.
So we're going to do now is a simple normalization.
We have a tortured sense of weights and now we want to normalize it.
And so to normalize it will simply write the explicit computation.
And so what we simply need to do here is take our output then update it by multiplying it by the standard
deviation we want to have divided by the sum I have just mentioned.
And so to get the sum we're going to use the square root function by torch.
And so that's when taken here torche that s q r t.
That's the square root function.
And inside we are going to input the square some of the weights of a vector.
And so we take our outputs.
Then we use the power function to which we input to because we want to take the square of the sum and
then we take therefore the sun.
And inside we are going to specify the index of the cone that contains the weight.
Want to some and then to get these weights separately because you want to sum them well we usually expand
on the school as a function of our output out.
All right.
So this will get the weights out which so far was initialized as a Torchin sort of weights that gets
all these weights.
We take the sum of square and then we take the square root to apply the normalization.
And the fact that we have this standard deviation at the numerator we'll make sure that we can write
it here.
Variants out will be equal to the square of the standard deviation.
This formula here will make sure that this tensor of weights that we initialized will have a variance
that will be equal to the square of the standard deviation that we put as arguments.
And that is how we can set a specific standard deviation for the future actually and critic that will
make soon and we will choose a small standard deviation for the actor and a larger one for the critic
and we will do this very easily since this function.
All right.
And so now we have only one thing to do left.
It's of course to return the output that is now normalized with this specific standard deviation.
All right so perfect.
That's the first function we had to make.
That's the first tool we will be very happy to use tonight.
H-3 sea brain.
We have one more function to make now.
It's going to be the weight function.
And that's just a function that will remind initialize the weights to make the learning optimal.
So let's do this in the next tutorial and until then I.