Hello and welcome to part to training the eye with deep convolutional kuning.
That's right.
Now that we built the AI AI with the architecture of the neural network the body the way the actions
are played and everything.
It's time to train say I with deep convolutional Cuellar.
So that's from now that we will implement you know experience replay.
Working with the values working with words.
And there's even going to be a bonus which will improve a lot.
The training process and that is called eligibility trace eligibility trace is a powerful technique
which consists of accumulating the word over several steps and the cube values are learned on this accumulation
of words as opposed to before where the cube values were learned after each transition.
Therefore after getting each word this time we will be learning the values after getting several rewards
instead of just one word.
So instead of having one transition after the other and you know updating the Q value each time.
Well the q values are going to be updated every steps because eligibility traces radicals and steps
eligibility trace and end is this number after which the q values are going to be updated and in our
model.
Here we are going to have any calls then.
So that means that will be 10 steps eligibility trace and therefore we will update and learn the q values
every 10 steps.
After accumulating the words on these 10 steps so that's a bonus.
That will make our model even more powerful and you will see that in the end we will get outstanding
results.
I was really amazed when I saw the final result.
I used to work on Merle's that took a lot of time to execute.
You know the AI took a lot of time to train but you will see that with this one plus the neural network
that we made.
That is our brain and our body here with stuff.
Next we will get a very powerful mob and therefore very powerful AI because you will see that it will
really cure lies.
Do you understand what I'm talking about.
So as you can see in this part 2 we are starting by getting the Dume environment and actually prepared
the lines of code for you.
We are just using the image pre-processing external file from a working directory folder.
So basically the order is rather to first take this line of code doesn't that make it in court.
0 0 is the name of the environment of the game we're playing.
So first we import the environment with this gem that make That's what you can find on the open page
and tutorials.
But then we use this preprocess image class which is a class from Image pre-processing to pre-processed
the images that will come into the new one that work and we preprocess them so that they have a square
format with the dominations 80 by 80 and that remember is because in our new one that work well we set
our input images to have the time engines one by 80 by 80.
Remember one is the number of channels and so one means that we're working with black and white images.
So that's the greyscale here.
And 80 by 80 means that the dominance of our input images will be 80 by 80 and that is what we set in
the neural network but of course then we need to specify this when putting the images which is exactly
what we do here with this preprocess image class.
And then after we import the environment with the right format of the input images while we import the
whole game with the videos with this line of code and remember the cool thing about this is that in
the end we'll see the videos are playing Doom So we will see how it will kill the monsters try to reach
the best and everything will be super exciting.
And remember that these videos will go into these videos folder.
All right.
And last line here but I want to show it to you because that's important that's now more related to
the AI AI that we're building.
Well remember that our neural network takes as input number actions.
That's because you know we want to make an AI AI that we can test easily on several environments and
several Dume environments.
And since the different environments have different number of actions Well we specified this number
actions variable as the input of the CNN the brain.
And therefore now what we're going to do is get this number actions variable using the Dumah environment
that we just important and created into this variable.
And later this number actually is very well that we're about to create will be the input of the brain.
So let's do this.
I'm introducing this real now variable number actions number actions equals.
Now we're going to take our doom environment that is the variable that we created.
So do my.
Then we add here that and then well here we go we take the first here action space that's the set of
your actions.
I encourage you to have a look at the opening at Horrible's to see how it works.
You know to understand how to open the gym environments work.
But basically this is the set of actions.
And from this set of actions we can access the number of actions in the environment and to do this data
here and then.
And that's the number of actions and therefore doom and that action space will return 7 6:48 because
there are seven actions.
I know that we can see six actions in the two environments on the opening page but I think we can also
run.
And so you know we can move forward move left move right turn left turn right and shoot.
And besides we can run that makes seven actions all right and that's it for getting to do environment
we have to do our environment.
We have a number of actions so we have so far everything that we need for our brain.
We will then just create an object or brain object which we'll call CNN minimal letters.
And since any function takes a number of actions as argument well we will put the number of actions
in the scene an object that we will create and then of course we'll create the body and eventually the
AI.
And that's why the next section I'm going to call it building an AI AI because now we can build as many
eyes as we want.
That's the awesome thing about object oriented programming.
We can build any AI's as we want.
And so we're going to build our AI that has the sophisticated brain and that's exactly what we'll do
in the next tutorial.
Until then enjoy AI.