Hello and welcome to this Python tutorial.
All right so now we're going to make the forward function which will propagate the output signals of
our brain to the body of the AI so that it will play the right action to reach the vest.
But there is no reaction yet because there is no training that we have not trained the AI yet but this
is exactly what we will do in part to implement in deep convolutional curin which by the way I will
rename training the AI with deep convolutional Killary.
But right now we need to forward the signal from the output layer of the brain to the body.
And that's exactly what we're going to do with this forward function which is the last function of our
body.
So let's do this.
We start with Teff forward and according to you what arguments is it going to take.
Well it's going to take of force first self and then is there another one.
Well yes there is.
And what is it going to be.
Well very naturally we want to forward the output signal of the brain to the body and therefore the
input will be the output signal of the brain.
And so now we need to give a name to these outward signals.
And so I'm going to add here the argument out put.
All right so that corresponds to the output signals of the brain after the input images are propagated
through all the brain to reach the output later which is X here returns by the forward function of the
brain.
And now this output signal of the brain will be forwarded to the body with this new forward function
that we make in the next class.
So let's do this let's add some color in here and now as you understood it we're going to use soughed
next method to play the action.
That means that the body of our AI after receiving the output signals of the brain will play the actions
with the next technique.
So basically now what we have to do is exactly the same as what we did for the car.
We're going to get our distribution of probabilities.
That's the first step and then we're going to sample an action according to this distribution of probabilities.
So basically what we could do now is get our self-driving car file and copy paste what we implemented
for the select actually function in the self-driving car.
But let's do it again.
It will be good practice and actually you can try to type it before me.
OK so first what we're going to do is get our probabilities.
So I remind this is a distribution of probabilities for each of the q values which depend on the input
image and each action.
So we have one key value for each of the six or seven possible actions and therefore we get a distribution
of seven probabilities and then 7 because I think there are seven actions instead of six.
Because besides moving for left right or shooting we can also run that makes seven possible actions
and therefore we get a distribution of seven probabilities one for each q value associated to each action.
So Propst equals.
And now remember what we had to do.
Well basically we have to use the soughed max function from the functional module.
That's very simple we take our functional module first then do that and then we take our soughed next
function.
Here it is.
We press enter and now we put the arguments of the next function which I remind are the elements for
which you want to create a distribution of probabilities.
And so that's of course the q values that is the outputs of the neural network.
That's the output of the neural network for which you want to create a distribution of probabilities.
Now remind us we want to create this distribution of probabilities to be able to explore the different
actions instead of directly picking the one that has the maximum Q value.
If we directly pick the one that has the maximum Q value where we don't explore much the other actions
and we might miss something.
But with this soughed next method we can do some more exploration and therefore maybe find some hidden
solutions in the patterns that might be much better.
So again I highly recommend stuff Nax and then from now what we have to do is input the values that
is our output here.
The outputs of our brain so outputs that we go.
But then we have this temperature parameter that we can use that we can configure to customize the exploration.
Remember that the higher we set the temperature the less exploration of the other actions will do because
the best action will be selected with a higher probability as opposed to the other actions which will
be selected with lower probabilities.
That's exactly like with a in car and therefore we have to multiply the output here by our temperature
parameter t that we get perfect now we get a little warning because we haven't used preps yet but we
are about to use it now.
And so that brings us to the next thing we have to do.
How are we going to use these probabilities.
Well we're going to sample the final action to play from this distribution of probabilities and therefore
what we have to do now is use the multinomial function to sample the action according to this distribution
of probabilities.
So now we're ready to get our actions.
So I'm creating a new Voivode here because that will become actions that will be played by the body
of our AI.
And so now we take our distribution of probabilities probs to which we add dots and then the multi normal
method.
All right and now we get our final actions to play there assembled from our props distribution.
Okay perfect.
So now we were ready to return what we want.
That is the action to play.
And these are of course actions and now the warning should disappear.
We use everything we want.
There we go.
Perfect.
So now the forward function is ready.
And congratulations.
The body is also ready.
So now we have our brain.
We have our body and therefore we're ready to assemble them to make the future AI our future AI I will
be composed of nothing else than a brain and a body.
And so it's what have intelligence and a body to play the actions which will be the right actions to
play thanks to its intelligence.
But remember before we have to train it's intelligence and that's what we'll do in part to training
the eye with the convolutional cool learning.
All right.
So let's make the AI in the next toils.
It's again going to be a class of two functions I think.
And so this one requires two or three tutorials.
So I can't wait.
This will be exciting.
And until then I.