Hello and welcome to this Python tutorial.
So now that we are ready to train the network to minimize the squared distance between the outputs and
the target thanks to what we did with eligibility trace in the previous section.
Well basically we were ready to start the whole training by you know getting our input our target our
predictions then computing the last error between the predictions and the target and then doing the
backward propagation was to get a grid in the center of data weights.
So we were ready to do all this but since we want to compute the moving average and 100 steps you know
to keep track of the average during the training.
Well just before we do this whole training we are going to make a class right now that we'll get this
moving average of 100 steps.
So no worries we will do it quickly.
We'll make it class with three functions what we will do all this and the single tutorial so we'll do
it quickly.
We already did it.
And besides we want to focus on the training right now because that's the most important.
So let's make this class right now in this single tutorial.
All right so we are going to introduce a new class which we're going to call and a for moving average
and then here we go with our first function.
So that of course the innate function that never changes in it and this and that function is going to
take two arguments.
The first one is self for the moving average future object and size which will correspond to the size
of the list of the words of which we're going to compute the average.
So this is going to be 100.
All right so we have arguments for the function.
Now let's go inside the function.
Now you know what to do we have to initialize the variables specific to the object.
And these are.
Well first the first one is going to be a list of words that's going to be the list containing the 100
words of which we're going to compute the average.
So here right now we're just simply initializing this list with this empty list here.
So list every word and then the second variable of our future object is going to be of course the size
and the size is going to be equal to the arguments who will input when creating the future moving average
objects.
So sightseer.
And already we are ready to move on to the next function which is going to be the add function and that
will add the cumulative rewards.
Be careful it's not the simple reward it's the cumulative reward.
And that's because you know we are doing eligibility trace and therefore learn every 10 steps.
And therefore learning with cumulative reward and not a simple reward.
So this adds function that we're about to make will add the cumulative reward to that list of rewards.
So Jeff we're going to call it ad of course and this function is going to take two arguments.
The first one is self because we're going to use this list of words here because simply we are going
to append the cumulative reward to this list of words.
So we need the self to be able to get this to self.
And the second one is going to be the rewards which will represent the cumulative reward.
All right so there are two arguments of the function.
Now let's go inside the function and let's define what it has to do.
Ok so very simply the first thing has to do is whenever we get accumulative we want a new one.
You know when we progress on tenue steps Well what do we have to do is add this cumulatively words to
the list.
And that's exactly what we're going to do we're going to write a line of code that will add this new
cumulative reward that we're getting after progressing on ten steps to this list of words here.
And to do this we have to separate two conditions because since he will be working with batches Well
we want to be in some lists but in some other cases do we words can also be as a single element and
the syntax to add an element to a list which is the list of words here is not the same whether you're
adding a list or a single element.
So we just have to make this a condition that will separate these two cases.
And let's start with the first case which is the case when what we're adding to this list of words is
a list and to do this we're going to add is instance in in parenthesis we put two arguments the first
one is are we words that we're adding.
So we weren't.
And the second one is LIST.
And so if instance words list means if the rewards are into a list.
And so if the rewards are into a list what we do is very simply self that we take our list of reward
and we are going to add this list because since this is a list what we can do is use a simple addition
operation because we can some tulis to get the rewards.
Here is a list because this will be called to true meaning for in this case.
And so we can simply sum this list to our list of words and therefore we can simply have your list of
rewards plus equals rewards.
And by doing this we're just extending the list by some in these two lists together.
All right.
And then second condition.
So we can simply add else so that if the rewards is not a list and therefore if it's a single element
and so else what happens in that case.
Well that's the same we want to add two words to our list of words.
But we cannot use the syntax because we words will no longer be a list.
It will be a single element.
And so what we need to use is another syntax which is the append function.
When you want to add a single element to a list you can add something to.
You have to use the append function.
And so this is exactly what we're going to do now.
We are going to take our list of words of the object and paste that here and then add dot and then we
go first one.
And of course in parenthesis we put the elements we want to append.
And this is of course b word but words in that case will not be a list.
It will be a single element like a single cumulative word not into a list.
All right and then we want to do this but now we have to add something more.
It's what does happen when this list of words gets more than 100 elements.
Well in that case what we have to do is delete the first element of this list every word to make sure
that this list of words always contains no more than 100 elements.
So exactly like what we did for the south driving car when making this go window and so to make sure
of this we we're going to add a while condition specifying that whenever the length of our list of words
that is number of elements in our list of words whenever this number is larger than self that size that
is the size that we said here and which later will be equal to 100.
When we create the object.
Well as soon as the number of elements of this list of words is larger than 100.
Well what we want to do is delete the first elements of our list of words which we can get by taking
the index 0 that is the first index of our list of.
This is the first element of our list of words and we want to delete it whenever our list of words contains
more than 100 elements.
So that with this with this conditional here we make sure that our list of words never contains more
than 100 elements and therefore Now what we can do is make a new function to compute the average of
our list of word which will contain on the run one hundred elements.
And therefore we will compute the moving average of 100 steps each time.
So let's make this the function it's going to be very easy because there is the main function in Python
which is a function from non-pilot to compute the average of a list.
And so let's introduce our last function here which we're going to call average and this function is
going to take one argument which is going to be self because we're going to use of course still our
list of words which is a variable of our object to self and coloring.
And now let's compare the average and so directly we will return the average because we can get it with
the mean function to which of course we're playing.
Well what we want to compute the mean of that is our list of what I think I still copy that.
Yes there we go.
So we simply return the mean of our list of words and the mean.
As I said is a function by non-Thai.
So here and Anglish look at that mean self list of words.
And then we go we have our average on 100 steps perfect.
So we made that class very efficiently.
Now we get the instructions on how to obtain a moving average of 100 steps.
And since we're going to use one moving average object when doing the training well let's already create
this moving average object.
And so we're going to call it an A and simply and is going to be an object and a class.
And as we said we want the size to be 100 because we want to compute the moving average on one 100 steps.
So perfect.
There we go.
We are now ready to train our AI to finally be intelligent.
It's about time it is from this point that our age will become smart.
So I can't wait to train it.
It's going to be quite easy because this is something we are dead.
But this is going to be fun.
And besides after that it will be time to have even more fun because basically our age I will be fully
ready that is built and also intelligence and therefore we will execute the code.
And then I will play Doom and eventually we'll watch the videos of our AI AI playing Doom and we will
see if it manages to reach the vest.
So I can't wait.
Let's do that training.
And until then I.