Hello and welcome again to our self-driving car Maggio.
So in just a toile I'm going to explain the environment on which we will implement our artificial intelligence
and that will contain Of course the car that will train to drive itself and to avoid obstacles and on
which will draw some roads and some blocks for our cars to navigate around them.
So we will later build this artificial intelligence to train this car to drive on the road you know
without crossing the limits and avoiding some obstacles that will put inside the road.
So this is a pretty exciting challenge.
And actually there are two separate files as you can see there is the apex and fowl that are artificial
intelligence that will do all the training to train the car how to sell drive.
And we have a map that we found that is the code that makes all this environment.
So here is that code that's actually 200 lines of code a little more.
So this code is not typically related to AI.
It is just a code to make the environment make the map.
So I'm going to go through each of the sections one by one to explain but we're not going to implement
this code line by line from scratch because we want to focus on artificial intelligence.
But let's still go through the sections one by one to understand what's happening.
So first we import the central libraries that's for any code.
We need some libraries to perform some task more efficiently than we import all the cavy packages.
So that's not very important because this is all specific to Cavey we're using cavy to make the map.
And so we're importing a lot of classes and objects to be able to make this map and add some tools in
the map.
All right then this line is important.
This line is related because basically this is where we import our brain the brain of the car which
will be another jet of this did you in class and the teacher in class.
Is our artificial intelligence itself.
You will see we will implement the dequeue in class in the following tutorials.
And as you might have guessed dequeue instance for deep Q networks.
So we will implement a dequeue learning network and then once it's ready we will be importing it here
with this line from the AI and the eyes of course are a python file.
All right.
So can't wait to implement this.
This is going to be quite a journey but you will see this is going to be very exciting because thanks
to the I the car will be able to drive itself.
All right.
And now before I move on to the next sections we have to explain how we will train this car.
I'm not going to explain the neural network right now but I'm going to explain the idea of how we can
train the car to drive itself and to avoid obstacles.
So you know in real life if you want to train a real car to avoid some walls or some obstacles.
Well what would you do.
You would definitely not take real walls or real big obstacles and smash your car onto them.
That would cost you a lot of money.
Instead a more intelligent idea would be to punish your car not when it smashes a wall or an obstacle
but when it goes onto some sand.
So it's like you have a field.
This field has some roads on which the car has to stay and the roads are delimited by some sand.
And each time the car goes into the center it's like it's going on to an obstacle because once the car
goes into some sand it will be slowed down and we will make sure that the car is penalized is punished
for that.
And that is one essential point of artificial intelligence.
The bad word comes whenever the car goes into some sense and is slowed down.
All right.
And therefore here I'm introducing last text and that's why.
Which are the coordinates of the last point in memory when we draw some scent on the map.
All right.
And then we get our artificial intelligence which we call brain and that contains our neural network
and we will call it brain because this is actually the brain of the car and that contains our Noodle
Network.
All right.
So in this line of code as you can see I'm creating objects at the teacher in class.
I will remind you that classes and objects are but Brenna's the object dehumanises the class and 5:3
and 0.9 are the inputs of the class.
So that's very simple five corresponds to the states that are encoded vectors of five dimensions we
will see what they are perfectly describing what's happening in the environment on the map.
Then three is the number of actions there will be three possible actions go left go straight or go right
and 0.9 is again a parameter in the dequeue learning algorithm.
All right and then we have the action through rotation.
So action to rotation is a vector of three elements.
020 and minus 20.
And so we have to do this because the actions are encoded by three numbers 0 1 and 2 and that corresponds
to the indexes of this action two rotation vector.
So for example if the action that is selected at time t is zero.
Well zero corresponds to the index of this action two rotation vector and the value of index 0 is zero.
And therefore we will go straight then if the action selected is 1.
Well one corresponds to the index of this action two additional vector and the value of this vector
that has a next one is 20.
So 20 corresponds to a rotation of 20 degrees and that means the code will go 20 degrees to the right
and then if the action selected is to well to correspond to the index to this action to rotation vector.
And therefore the code will do a rotation of minus 20 degrees and therefore it will go to the left.
All right then we introduce the left reward Voivode because at each stage will be getting the last word.
So remember if the card doesn't go into some sense then the reward will be positive.
And if the code goes on to some sense well it will get the bad reward and each time this variable will
contain this reward that it gets H20.
And then we initialize the scores which is a vector that will contain the reward.
Not all of them but the reward.
Onto a sliding window so that you know we can make a curve of the mean squat the reward with respect
to time.
All right then in this code section we initialize the map so we initialize.
For example the said variable.
So that's important.
The second variable is actually going to be an array in which the cells will be the pixels of the map
and in each cell we will have a 1.
If there is some sense in a zero if there is no center at the beginning we will not be drawing anything.
So there will be no sense at all and therefore all the cells of the center array will have a zero.
There will be zeros everywhere and as soon as we draw some sense while the cells on which we draw the
sense will get one and we initialize the race with all the zeroes right here sent equals and zeros.
Then we have this important thing which is the goal.
So the goal is a point in the map which will train the car to reach.
So it's like a destination.
So what is this goal going to be.
Well this is going to be the upper left corner of the map to all train the car to go to the upper left
corner of the map.
And then once it reaches the upper left corner of the map then we will trend to go to the bottom right
corner of the map so we can imagine the following scenario.
The upper left corner of the map is the airport of a city and the bottom right corner of the map is
the downtown of the city.
And we will train a taxi or Uber to do some round trips between the airport and the downtown.
And of course we'll make the task difficult to this taxi by drawing some more and more difficult roads
and adding more and more obstacles on the street to see if the taxi can still manage to go from the
airport to downtown.
So this is going to be fun.
And so that's why here I'm setting the coordinates of the first goal that is the airport which is at
the upper left of the screen.
So the map will be like a square like this and the coordinates of the origin that is the coordinates
00 is right here.
And then larger is this distance here.
So the coordinates 20 and Najah minus 20 will therefore be right here the upper left corner of the map.
And why did I choose 20 and not zero.
Well that's because we want to train the car not to rush into the walls.
You know we want to train into the walls as well and therefore it's not be zero because we don't want
the car to touch to where we want to touch to go.
So we have to put it right here and then I'm just introducing the last distance variable which just
gives the current distance from the car to the road and that I'm initializing to zero.
All right.
And now time to make the car and the game.
So we're going to make two classes one class for the car and one class for the game.
And inside these classes will already make some connections with our AI.
So we'll do that in the next Statoil.
And until then enjoy I.