Hello and welcome to the Statoil and the to talk we initialized the map.
And now time for the exciting stuff we create the car and we do that with a class of course.
You will see that the class is very practical to create some things that have a lot of properties because
as you can see not only finding some variables from my car but also some functions which of course is
the function that will make the car move to the left to the right or going straight.
So we have a couple of variables that are important to describe the environment we have for example
the angle which is the angle between the x axis and the axis of the direction of the car.
Then we have the rotation which is its last rotation which remember is either 0 degree 20 degrees or
minus 20 degrees.
Then we have the velocity the x coordinate of the velocity vector and the y coordinates of the velocity
vector.
And then the vector of coordinates velocity x and velocity Y then we have the sensors and the signals.
And that's very important the call that we're making will have three sensors sensor one sensor two and
sensors three sensors one will be detecting if there is any sense in front of the car.
Then sensor 2 is the sensor that will detect if there is any sense at the left of the car and sensors
3 is the sensor that will detect if there is any sense at the right of the car.
And then from these three sensors we get the signals that the signals received by each of the sensors.
So signal one is the signal received by sensor one signal to is the signal received by sensor 2 and
signal 3 is the single received by sensors 3.
And so how does it work.
Signal 1 is the density of send around sensor one signal to is the density of center around sensor to
signal three is the density of sent around sensitivity and how do we compute this density of sand.
Well that's very simple.
We take some big squares around each of the sensors.
These are actually squares of 200 by 200 and for each of the squares we divide the number of ones in
the square by the total number of cells in the square that is 22 and 20 equals 400.
And that gives us the density of sand because the ones correspond to the center.
We do this for each sensor and that gives us the density of sand around each sensor.
That is the signals.
All right so now we have everything to detect the sensor and then we have the move function.
And of course the move function is what will allow the core to go to the left going straight or going
to the right.
So let's go through it quickly.
We have here the update of the position of the car with its last position which is self the pass here
and the velocity vector.
So thanks to the velocity vector the position will be updated in the direction of the velocity vector.
Then we get the rotation which we will get further down in the code right here.
Rotation equals action to rotation action.
He will select the action and then get in the rotation.
And so this self-protection equals rotation in here.
It's this rotation that we get to know how we need to rotate the car that is going to the left or to
the right.
Then we update the angle which I remind is the angle between the x axis and the axis of the direction
of the car.
And then once the car has moved then we have to update the sensors and the signal because of course
when the car has just rotated while the sensors have rotated as well and therefore we need to rotate
them by using the rotate function and to which we add the new position.
And why do we have this vector of 30 0.
Well that's simply because 30 is the distance between the car and the sensor.
You know is the distance between the car and what the car detects.
And then once the sensors are updated Well then it's time to update the signals.
And so here we do exactly what I explained to you computer signals.
We give the x coordinates of our sensor then we take all the cells from Manston to plus 10 then we do
the same for the y coordinate taking all the cells from Madison to plus 10.
So therefore we get the square of 20 by 20 pixels surrounding the sensor and inside the square we some
older ones.
So basically with some of the cells because the cells contain 0 or 1 and since in a 20 by 20 square
there is 20 times 20 equals 400 cells.
Well we divide it by 400 to get the density of ones inside the square and that's how we get the signal
of the density of centers around the sensor and we do the same for the second sensor and the third sensor
to get the second signal and the third signal.
OK.
So that's to take the sensor and then these three lines of code here are very important.
It's another that we want that we want to get to our car when it's reaching one of the edges of the
map.
You know we don't want the car to rush into some walls and therefore we want to penalize it to punish
it when it's getting too close to Wall and therefore that's what we do here.
If the first sensor is larger than longer minus 10 that is larger than here because longer is this distance
here so longer minus 10 is right here.
So if sensor one acts larger than longer Manison concerns all the points are here.
That is if the car is getting closer to the right edge of the map or if cell sensor one acts lower than
10 that's right here if the car is getting closer to the left edge of the map or if sensor Y is larger
than larger minus 10 that's the upper edge of the map and or if self-censored Y is lower than 10 that
is the lower edge of the map.
And so if the sensor one is reaching any of these four edges Well we will put the signal of the sensor
signal when the signal sent one.
We will set it to be one.
And what does that mean.
That means full sound like the full density of sand.
It's like the worst and you could get.
There's so much sense that it's going to stop your car.
So signal will be one and therefore the car will get a terribly bad reward.
All right.
And then we do the same for signal too and signals three from sensor two instances three.
All right.
And then we create the game class so that's basically the class to create the game because so far we
have only created the car and now of course we have to create the map.
We have to create the game itself.
So we will not be playing the game it's our AI that will be playing the game and the game is actually
to avoid the obstacles and to go from the airport to downtown and vice versa.
So in this game class we need to create some objects like the car then we need to define the update
function.
That is the most important and actually we will focus on that right now because that's in this update
function that will select the action that the car has to do and each time to accomplish its goal.
And this action is exactly the output of our neural network.
The new will that work that will be at the heart of our artificial intelligence.
And so this action is returned by the brain of the car which remains the object of our human class that
will be making in our AI file.
And this object has a method that is called update and it takes as input the last word and the last
signal.
So the last word is of course the last word attained by the car and the last signal is of course the
last signal of the three sensors signal one from sensor one signal two from sensor to signal three from
sensor 3.
But then I'm adding two other inputs which is the orientation of the car with respect to the goal.
So for example if the car is heading towards the goal then the orientation will be equal to zero.
If it goes slightly to the right then the orientation will be close to 45 degrees.
And if it goes slightly to the left the orientation will be close to minus 45 degrees.
So that's the force input of our input states.
And then there is a lesson put which is minus orientation.
So usually the inputs of neural network are independent There is no multicollinearity but it doesn't
really matter if we are this is because the neural network will just fix that with the weights.
But still I notice that by adding this minus orientation well that allows the car the training of the
car to stabilize the acceleration.
You know we're doing this is that the AI doesn't always explore in the same direction by adding this
minus orientation.
We make sure that it explores in both directions right or left.
And so this the three signals Plus the orientation and minus orientation are the five inputs of our
encoded vector which will go into the network that our input vector that will go into the network and
after it goes into the network Well the network returns the outputs which is the action to play.
And each time and the output is returned by this update function that contains the network itself and
the output of the network and therefore that's why we have to input the last signal that is the input
state and also the last word because the action to play also depends on the last word.
All right and then we update the main score of the words we update the rotation we use the move function
to rotate the car according to the action that was selected.
We take the distance of the car to the road and we will get the positions of the sensors ball one ball
two in both Rieker response to the balls that will represent the sensors on the map.
You'll see that very quickly.
And then here that part is very important because that's where we penalize the car if it goes into some
sense because as you can see this means if the car is onto some sense well it will be slowed down.
So that's where we reduce its velocity.
You know its velocity is usually 6 as you can see here.
And if it goes on to some sense it will be one which will be slowed down to 1.
You'll see how the car will be slowed down once it goes into some sense.
So it is slow down.
And besides it gets a bad reward it gets a minus one we want.
And that's actually the worst reward you could get.
The best reward is one the worst we word is minus one and the reward is between minus 1 and plus 1.
And then otherwise.
And the car isn't onto some sand.
Well it keeps its usual speed speed of sex and then we add something else.
If it's getting closer to the goal then it will get a slightly positive reward.
And if it's getting further away from the goal Well it gets a slightly negative reward minus 0.2.
And then last.
Conditions that are related to the reward.
Well that's if the car is getting too close to one of the edges as we spoke of earlier.
Remember when we talked about fuel sensor.
Well if the car is getting too close to the left edge of the map it gets minus one word if it gets too
close to the right edge of the map it gets rewarded minus one.
And if it gets too close to the bottom edge of the map it gets you or minus one.
And if it gets too close to the upper left of the map it gets rewards minus one.
So that's a terrible punishment.
And so you will see how it's full and fast not to rush in to something else.
All right and then this is to update the go when the goal is reached.
So you know when the car reaches the airport which is the first go there is the upper left corner of
the map.
While the goal changes to the bottom right corner of the map which is downtown.
And that's exactly what we do here we update the x coordinate of the Crow and the white coordinates
of the call and then we update the distance from the car to the car.
All right.
And then that's less important.
That's just a class that will add the painting tools you know for us to be able to paint some roads
or some obstacles on the map that's more related to K.V. can have a look if you want I'll provide the
condensed version of this code and I'll provide some reference if you want to go deeper on how to do
that with skivvy.
But we're getting further from artificial intelligence so I'm not going to go into the details of it.
And that's the same for the last code section with the car class that is just as the API buttons clear
save and load.
So that's what we do here.
Clear canvas safe.
And that's actually very important that's for us to be able to save the day I know to save the brain
so that you can reuse it later by taking the load function which is another tool we add on the map to
load the brain of the car that is to load the memory of the car how to navigate in the map.
And then finally we have the last of the last code section which runs the whole thing that is which
runs the map and the AI itself.
And actually that's what we're going to do right now.
Let's have a look at everything we made in this code.
So right now the AI is not implemented.
So the code will have a very random movement.
It will actually look like an insect but don't worry we will fix that.
Normally we will train it to move like a real car and train it to navigate following some roads and
avoiding some obstacles.
Let's do this.
I'm going to select everything and execute.
And here is the map.
And here is the car.
All right.
So that's the little thing here that you see that looks like an insect in our car.
So as I told you the actions are totally random.
So each time the car selects randomly an action whether to go straight to the left or to the right.
So that's why it is making some nonsense movements and that's why it's looking like an insect.
So we will fix that of course.
And of course since the AI is activated Well it is not going to the goal which is the airport here or
to downtown at the bottom right of the map.
And we will fix all this by making the AIs who will implement the AI into this car or this insect so
you can see the three balls here the yellow one the red one and the white one.
That's our three sensors.
So that's what we'll detect.
There is some scent around it.
And speaking of scent Well let's throw some.
So to do so I just need to do a click left here and you know in some sense by still teaching left.
So right now I'm adding some sand we can add some more.
So each time in insane as you can see that putting ones in the center array that's to send array that's
to 00 coordinates of the origin.
And here there are a lot of ones.
And as you can see well that's good to see the car.
We just went on to descent and was slow down.
So as you can see right right now it is really slow down because it's going into the sand.
And right now it's trying to escape.
And so you know what we'll do is we will draw some roads and we will draw some roads from the airport
to downtown and we will train the car to stay on the road and to avoid the obstacles.
All right.
And as you can see there is the clear button to clear the sand there is the save button to save the
brain of the car.
And actually there is this core curve that we spoke of that safety that saves your model actually the
brain of your car and then you can you know when you leave your cold or turn off your computer and you
want to go back to it again.
You can use the load button to load your mode that is to load the brain and that will get the trains
AI of your car.
All right so now I can't wait to start making the AI.
This will be a lot of fun.
We'll make our neural network and we will punish the car as soon as it doesn't do what we want.
So let's do that from the next tutorial.
And until then enjoy a.