Hello and welcome to the fun tutorial of this first module self-driving car it's going to be epic.
We're going to test our eye on the environment and we're going to test it on four different levels.
That is we're going to play a game the game will have four levels of difficulty and the aim will have
to pass these four levels.
So we're going to be these four levels.
First Level One the first level is going to be to reach the airport and then do some round trips between
the airport and the downtown.
So as soon as we see the car do these round trips Well we pass level 1 then Level 2 Level 2 will be
to still do these round trips.
But on the specific road that we draw ourselves but it's going to be an easy road because it's level
2.
And of course the car will have to self-drive by staying on that road.
So it will be a road that goes from the airport to downtown and then the other way.
And so the car will have to do these round trips by sitting on that road.
If it does we will pass level 2 and level 3.
Level 3 will be to draw some obstacles on the map to see if the car manages to avoid the obstacles and
still reaching its goal.
So no worries will draw some difficult obstacles that the car will have to avoid.
And we'll see if it managed to reach the airport and the downtown and finally level for the most challenging
level for the car will be to draw a very difficult road to reach the downtown.
So I don't know you know it will be a road like some zigzag.
I'm not a brilliant architect but I'll try to make a challenging road.
So let's hope we pass at least the first level that would be great then let's hope we can also pass
level two and three.
And if we passed level 4 that would be wonderful.
So let's do this let's take the challenge.
Well actually the self-driving car is going to take the challenge but we are the brains behind this.
So let's tell how that works.
All right so the first thing I'm going to do is just to give you a quick reminder about the map.
So that's the map.
And first we're going to look at the map we're going to look at the self-driving car without the AI
to it will just be a car having those random actions that you saw at the beginning of this model.
So how can we look at that.
Well we have to deactivate the AI and the activity.
We simply need to put a temperature equal to zero.
Remember that parameter here is the temperature.
And right now it is equal to seven.
That's a low temperature.
We will increase that afterward.
But if we don't want the car to have a brain that is if we don't want to activate the AI we simply need
to set the temperature to zero equals zero.
And same here of course that's the real temperature in the cold too.
There we go.
And then we must not forget to say because otherwise that won't include the change.
I guess now we don't have any.
I see I used to activate it.
So let's have a look at the map just to give us a quick refresher.
A quick reminder about what it looks like.
So I'm going to select everything and press enter.
All right.
And there's our map and there is our car.
So as you can see the car is having totally random actions you know to go left to go straight to the
right and therefore it is not reaching the airport which is I remind at the upper left of the map and
not reach it.
Well it just did.
That's totally random.
You see it right now it is at the airport and it is not reaching the other goal which is downtown at
the bottom right of the map.
So we were just like here but we can clearly see now that the actions are totally random.
It is going nowhere and there is definitely no artificial intelligence but no worries.
We will activate it right now.
I'm going to close the map and then I'm going to restart the kernel restart DeCarlo.
You click on this tool button here and then yes.
And now time for the show we're finally going to put this train we made in the car and activates the
AI.
I'm super excited to see what's going to happen.
We're going to activate the air right now and to do this we need to raise the temperature so to change
the temperature which is it replaced at zero by well let's start with seven as we had before.
So let's specify seven here.
All right let's not forget to say.
And now let's get back to our map and now we can just re execute this again because we restarted the
kernel.
So let's say it.
And there we go we have the car.
And what is it doing.
Well it is trying to find its way its exploring its understanding what it has to do and its about to
reach the airport and there we go first go reached wonderful and now the next goal is to reach downtown
and there dead just reach downtown.
And now its trying to find the airport back going to the airport and there there again Wonderfalls that
works.
It didn't take time actually to explore learn from the mistakes you know the mistake here is to get
further from the go.
Thats where we punish the car by giving it a slightly negative reward.
You know its minus 0.2.
So it learn from that mistake and by learning from that mistake it managed to get the positive rewards
by getting closer to the goal.
And now it finally understood what it has to do.
Its definitely reaching the airport and then reaching the downtown and then doing these round trips.
That's perfect.
We have a self-driving car but I cannot help but notice it is looking like an insect.
The car doesn't really seem sure of itself.
No it doesn't have a very confident movement.
It's like your left and right that's not moving like a car movement.
It looks more like a bug.
So we're going to fix that.
And as you might have guessed the way to fix that is increase the temperature because remember the temperature
is the parameter in the soft Mac function that we can increase so that the action is returned with more
certainty.
So that makes sense that if we increase temperature well we might end up getting a car more sure of
itself because the AI will be more sure of which action it should play.
And that remember is because the action will be played with a higher probability.
The only problem with this increasing the temperature is that remember the AI is less exploring the
other actions because by increasing the temperature of the other actions we'll have low probabilities.
But right now that doesn't seem to be a problem because the car seems to have no problem reaching its
goals.
The airport into downtown so we can totally increase the temperature if we want this thing that so far
looks like an insect looks like a car.
So let's do this.
I'm going to close this now.
There we go.
Restart the kernel again.
And press yes.
And now we're going to increase the temperature.
So let's do this.
I'm going back to my file then replaces seven by 100 There we go.
Then we save.
And now we have a self-driving car sure of itself.
So we might get better results and we might get something that looks more like a car.
Let's take a map and then let's to that again.
All right what's happened.
OK.
It did some kind of a burnout.
Not sure why.
But anyway now we have something that looks more like a car.
You can see that it is going more straight.
It is not doing these quick left and right movements.
That's because now the car is more sure of which direction to take and each time you know it wants to
take the best direction going to the airport and then to downtown.
So clearly we can now say that we passed level one.
The car is doing these round trips between the airport and the downtown.
So we are going to save that that's I'm going to show you how to save the brain.
We just need to click on this save button and if we look at what happens here.
Well we have the curve of the we were at the beginning we can observe some mistakes that it made.
So that's where the reward is negative.
But then it learned from its mistakes and the reward increased little by little until reaching a constant
positive reward equals to open one but that's the maximum reward we set.
And that's because it ended up exploring.
That's the expiration phase.
And then it just knew what it had to do.
That's where it was doing these round Rountree between the airport and the downtown without any mistake.
So there we go.
We passed level one.
Congratulations.
Now let's get things more challenging.
Let's take things at the next level.
Let's try to pass level to which I remind you we do these round trips on a specific road.
We're going to draw ourselves.
So let's check that out in the next a toil.
And until then I.