Hello and welcome to this Python tutorial.
All right so we just build the architecture of a neural network with the function of our network class
and then we're going to make a second function which is going to be the forward function and that's
the function that will activate the neurons.
That is the function that will perform for propagation.
So let's do this let's make this function let's call it forward as we just said and this function is
going to take two arguments.
First is as usual self you know to be able to use the variables of the object because we're going to
use SE1 NFC to.
So we need the self to be able to use these variables and then we're going to need a second argument
which is our input and we're going to call it state because state is exactly the input of our neural
networks.
You know that's the States.
There are other inputs entering the neural network and then as outputs will have the q values of the
three possible actions go left go straight or go right.
But we don't need to put it as an argument here because that's exactly what we want to return to this
forward function is not only going to activate the neurons but also and mostly it will return the cube
values for each possible action depending on the input state here.
All right.
So that's the two arguments we need.
And now let's go inside the function and let's specify what we wanted to do.
OK so the first thing we're going to do is activate the hidden neurons and we we're going to call the
hidden neurons by the variable x so x represents the hidden neurons.
And so how to activate them.
Well of course we're going to take our input neurons.
We're going to use our first full connection if you want to get the hidden neurons and then we're going
to apply inactivation function on them which will be the rectified function.
So how are you going to do that.
Remember we imported the torch.
And it's that functional module that contains all the functions in order to implement the neural network.
And we gave it the shortcut.
So actually what we're going to do now is we're going to use one of these functions from the functional
module and this function is to really function.
So what is really relevant is the rectifier function that you saw in the intuition lectures.
That's just and then given to the rectifier function.
But since this function is taken from and in that functional which was given the shortcut we need to
type here first f thoughts and then that's where we can take this function.
And actually if I type are we here we go we have the real function.
So that's directed by a function that will activate the hidden neurons that is x.
So in this really function now we understand perfectly what we have to input that's the neurons that
we want to activate.
That is the hidden neurons and so to get these hidden neurons we are going to take our first full connection
one which we will apply to our input neurons to go from the input neurons to the neurons.
So let's take our first full connection one.
But our first full connection is a variable of our object.
Therefore we need to type here first self.
FC one here we go.
That's the first full connection of our neural network.
And inside this personal connection we are going to input our input states to go from the input neurons
to the hidden neurons.
And so since we gave it the name STATE Well here we have two input state and there we go.
We now get activated hidden neurons.
All right.
And now that we have the hidden neurons we are going to return the output neurons.
So next line and as you understood the output neurons correspond to our actions.
But these are not the actions directly.
These are the q values because we're building a digital model that combines a deeper model to Q learning
and therefore we use q learning here to get the q values for each of our actions and then later using
soughed Max or in our Max we will get the final action so here the Voivode I'm about to introduce will
correspond to the APA nuance and since the output neurons are the key values.
Well I'm going to call this variable.
Q That means that we go to q values and now we directly take our food connection which is the variable.
To put a variable from our object.
So we take here self taught F C to and of course here we input the neurons of the left side of this
connection.
That is what we got from the first line which is x.
So x There we go.
We now get our Q values.
That's the output neurons of our neural network.
OK.
And then the last line of code of course this forward function is used to return these values.
So we just have to add a return and simply Q values and that will return the key values for each possible
action go left go straight or go right.
All right.
So congratulations.
We're done with our first class and actually we were done making the architecture of the neural network.
Remember this is not finished job.
You can always improve the architecture of the neural network by trying different ones.
So feel free to do that by adding more neurons here.
For example if you want to add 50 hidden neurons you can just replace the 30 here and the 30 here by
50 50 and 50 and then you can add some more hidden layers by making some useful connections.
Well that's really the job of an artist.
There is no general rule of what would be the best architecture in each situation.
So that's why we have to experiment.
Let's let's try with that you will see that we'll get eventually a pretty good self-driving car.
All right.
And now we're going to make the next class which is about experience replay and we will be making that
in the next three to two hours.
Until then enjoy AI.