Hello and welcome to the very last step of this part one building the.
Now the only thing that we have to do left is to make this the Ford function that will propagate the
signal from the very beginning when the brain is getting the image to the very end when I place the
action so we're going to make this whole function and that's going to be our last step before we move
on to part to training our AI with deep convolutional kulang.
So let's do this.
We are going to take the function call which actually is similar to the init function that is it's an
existing function but this time we use it to call some other functions.
The ones that we made before because you know we're going to use the forward function from the brain
and the port function from the body.
And so we're using this function now to basically call these functions.
So call is going to take two arguments.
The first one is self.
Of course the object and the second argument which according to you what is it going to be.
Well we are doing the whole propagation this time.
So what we want to take as input is of course the input images because of course that's the starting
point when the AI is playing the game.
It is first visualizing the images of the game then propagates the signals in the brain and then plays
the action.
Therefore the second argument is going to be inputs.
And now we are ready to make this whole propagation.
So let's do this again.
So the first step where is it the first step is receiving the input images from the game.
And since these images are going to enter the neural network Well you can imagine that we have to format
them in a special structure and the structure is of course a torch structure.
So the first thing that will happen is that we will convert these images into an umpire array then we
will convert it into a torch tensor and then finally we will put the torch tensor inside a torch variable
that will contain both the tensor and a gradient.
That's for our dynamic graphs to compute very efficiently the gradients later faster Kattie great in
the sense.
So that's our first step.
And then once we get the right format of our images well they will be able to enter the neural network
and then that's where we'll do the whole propagation of the signals.
So let's do this first that converting the image into the right format.
So our images are so for inputs.
Now we're going to create a new variable which I'm calling input.
So that's the real input of the neural network and this input.
Where is it going to be.
Well first we need to take our inputs.
That is our original images.
Then as we said we want to convert these images into Nampa arrays.
So to do this we can simply take none by which has a shortcut and then the function array.
So we put in put in the parentheses of the function array.
There we go.
Now it is converted into something and by arrays.
But then since the cells of the numpad arrays will contain the pixels it is actually safer to specify
the type float.
It's better to make sure we have some floats right now to make sure that we can use that float.
Sorry to hear.
All right so now we still have an umpire.
But with the tablet.
All right and that's also another reason it's that tensors are by definition arrays of a single type.
And so we choose the single type to be a float float 32.
All right.
Now that we have our non-bio raise the next step is to convert that into a tortoise sensor and to do
this we can use for example torch.
And then from underscore non-pilot function that will convert that into a torch sensor.
There we go.
And now the last step is to put these torch sensors into a torch variable containing both the tensor
and the agreement.
And you know how to do it of course we take our variable class because actually everything that is inside
this variable is actually the input of the variable class.
But I wanted to show that to you this way because you know we start with our input images then we convert
them into numbered arrays then to torch tensors and then tomorrow.
And now we're good.
They are allowed to enter the neural network that is first the eyes of the eye and then the fully connected
layers to lead to the predictions.
So speaking of the eyes of the eyes that's exactly what we're going to do now.
We're going to propagate these allowed images now into the eyes of the eyes that is through the three
convolutional layers.
And to do this you're going to see now how it's so simple.
That's because we already have our brain in our body from the init function.
We simply need to take our brain self that brain and apply this brain to the input images and that will
propagate things to the food function here from the brain.
That will propagate the signals inside the brain and since the forward function of the brain returns
the output signals that the neurons of the output layer containing Q values Well this self the brain
input here will return this output signal and therefore we're going to put here whether it be turned
into a variable and we're going to call it very simply outwits this output is the output signal of the
brain.
And now now that we have the output signal of the brain Well we have to propagate this output signal
to the body and to do this we're going to use the second forward function from the body and to do this.
We simply need to take our body and apply it to of course the output because the Ford function of the
body takes as input the output signals of the brain.
So that's exactly what output is right now and returns the actions.
And therefore since it returns the actions.
Well here we are going to add actions to cause self that very output.
All right so now you can see that very simply we propagated the signals inside the brain and then from
the brain to the very first by using the form function from the brain which takes us and put the input
images and then propagate them into the brain to retain the key values.
And then we propagate this output signal into the body where the forward function of our body to get
the action to play.
And so now the only remaining thing that we have to do and that's the very last line of code of this
part one building the.
We have to return the action to play and that is actions.
However right now the actions have to torch format and we need to convert them back into them by right
and to do this we're going to take the data structure of these actions and then add here the non-pilot
function and then we go.
Now we have the actions we turned in the right format.
So congratulations.
We are now done with this first part 1.
We build the AI in three steps.
First we made the brain second we made the body.
And third we assembles the brain in the body and we propagated the whole signal from the eyes to the
moment we play the action.
So that's a first step done.
That was a huge step but now as you understood we built an AI AI but it is still stupid.
We need to train it to be intelligent.
So we need to train it to do what we wanted to do and to do this we're going to use the word have to
do environments you know because it's learning from the world by being reinforced when it gets a good
reward.
And by being punished or weakened when it's getting a bad word that's where the cue learning will come
into play.
And so that's exactly what we'll do in this part to train in the eye with deep convolutional Q learning.
I can't wait to start and until then I.