Hello and welcome to the second module of this course building an AI to beat the game Doom.
So yes that's right.
We're going to play Doom with an artificial intelligence which will try to kill some monsters to reach
some goal in a 3D environment.
You know 3-D map exactly like when you're playing video game on encounter differences.
You are not going to play this game the way you're going to build and code we'll be playing it for you.
So there we go we have another very exciting challenge ahead of us and besides we're going to take it
to the next level now because we're going to make a more complex model which is the deep convolutional
Cuellar.
So I hope after this intuition lectures you're now very excited to implement it.
I am definitely very excited to implement it.
So let's start right now.
And as you can see I'm starting with Google because I want to show you the opening eigene environment
and walk you through the game we're going to play that is of course do.
All right so starting with Google we're going to type here open a gym.
So there we go opening a gym.
Then you click on the link and you will find a page that page and on the upper left corner of this page
you're going to click on environments and that is where you have all the environments which are actually
environments of games.
And for each of these games you can implement an AI to be the game.
And each time it is specified which goal you have to accomplish what are the actions and what are the
rewards and know that for most of the games the input states are actually the input images.
That is it's not like before with the self-driving car where we had you know a vector and coding One
state of the environment for most of the games here the input state is instead the image.
So it's exactly like our future I will have eyes like as humans do.
In fact this AI we're about to implement is to be the closest to a human brain because it is going to
observe a series of images then it is going to detect where it has to do in the images and then using
a classic neural network.
Well we'll know which action it should play.
And so now let's move on to the game.
We want to beat which is the last games on this list do.
So let's click on it and there we go.
Welcome to the doom and violence.
So as you can notice we have several versions of doom.
And these are just different maps with different levels.
So for example let's have a look at the first one to basic zero which is this game that you see here.
And as you can see for this game and for any of the games it is specified to go so for this game the
goal is to kill this monster here in three seconds with one shot.
Then it is specified the rewards.
So you get plus 101 points for killing this monster here.
Then you get a bad word with minus five points for missing a shot.
So actually right now it is getting some that want because it's missing a lot of shots and you get minus
one every point oh 20 seconds 20 seconds that's all the rewards.
And then you have how the game ends.
So it ends when the monster is dead.
When the player is dead or when there is time out.
And you also have the actions so you have three actions for this one attack.
So when that when you shoot move right and move left.
OK but that's not the game we're going to play.
Actually this one is too simple.
That's my favorite.
I prefer the one where we can move forward move back turn left turn right and shoot many monsters.
This one is much more exciting than actually this is this one corner.
Let's click on it.
That's the one.
So as you can see now we can actually move forward.
We can move left we can move right turn left turn right and we can shoot again.
And we have many more monsters like these guys here.
And our goal of course is to not only to shoot them but that's because the main goal is to reach this
tip of the map here which they call the vest.
The goal the main goal and that gets us 1000 points is to reach the vest over there and to do this well
we'll have to kill the monsters because if we don't kill the monsters we will get killed and we will
not be able to reach the vest.
All right.
And so that's our main goal which of us then let's see what our rewards are we get a first word when
we are getting closer to the vest.
So that plus distance the words is exactly the distance we complete while getting closer to the vest
than mine is distance reading further from the vest and minus 100 points for getting killed.
So here the reward is just measured with the distance.
When you get closer to the vest you get a reward that is equal to the number of meters you've got closer
to the vest.
So that's a continuous positive reward.
And when you get further from the vest you get a reward that is equal to the number of metres completed
by getting further from this list.
And then if you get cancer you get a bad word of minus 100 points then the game can end when the play
touches the rest.
That's when we win.
When the player is dead that's when we lose.
And when there is time that game over and here are our allowed actions attack shoot right and left forward
turn right and turn left.
We have more actions than before.
We now have six sections.
That's OK.
This will not increase the complexity so bad.
All right so I hope you're excited with this game.
I actually think this is the most exciting game.
So we're going to build an AI to beat that game and to do this we will implement a deep convolutional
neural network.