It all three.
OK.
Hi guys.
I'm the first to talk because I'm on the subject.
Hi guys.
Hey hey.
Tell me when you're ready.
Hi guys.
Welcome to the Statoil.
As I told you in the last it will surprise you.
What is it.
Hi guys.
Curious I get to say hey guys there is a surprise which is this life I guess.
Hi guys.
Welcome to the Statoil.
And as I told you in the previous Statoil I have a surprise for you.
Carol is here with me to surprise right next to me so.
Hey guys.
Surprise here.
Are you excited to see the results.
Subrace I learned together talking about it for a long time now.
Yeah.
And just so you know the students have been really struggling with me to write these lines of code.
We have 156 lines of girl now we're so relieved to finally watch the results.
Great job.
OK.
So to get the results was very simply we're going to select the whole code just before we execute.
And as a reminder you have the videos folder and do much too.
And right now this for a reason it's going to be populated as soon as we see us with some videos of
so it's happening in real time yes real time.
OK.
Good stuff.
OK.
All right.
Are we ready.
Yes let's do it.
Execute.
And there we go starting with the first video.
So first we are are you recording that's your first attempt at playing the game.
And there we go the first average we were negative.
That's pretty bad but that's totally normal.
Right now the eye is exploring and train line 41.
Yeah and remember we have to watch the 1500 too.
I think we were lucky on this one.
This is the first time I see this you know reaching such we want to as you see just do it just like
he was just like you.
OK.
Interesting.
Yeah.
So what does it mean if it reaches 1500 Oh.
So if it reaches 1500 means that we know 100 percent that the eye will have reached the best.
This is the one way it's got to get to that best as soon as possible.
Yes.
Actually we.
We have time to explore opening night.
Yeah.
You know we're playing this.
Do we ever do as you can see this mission to run as fast as possible to grab a target so you know it's
like we were super positive when it's reach you that's pretty fast.
Yeah.
So it really tried to reach us that doesn't like this is a special one because it doesn't really have
to even kill the monsters it has to figure out that it needs to run.
It has to figure out the best strategy to reach as soon as possible.
Maybe it's best just to kill the monsters.
But I think so yeah maybe it's to avoid the monster maybe it's to run very fast labeling the monster
I don't know.
Yeah but yes that's definitely a goal to reach the best as fast.
OK that's interesting interesting.
There seems to be going down.
Is it going down.
No it's normal.
I think we're just at the beginning.
And it's you know you know there is the training and it's doing some stuff and I think you know it it
went far too far but it got killed.
And so it was not a good strategy because it took some directions you know towards the monsters that
got killed but still it got form because it reached more than 1000.
But we're going to see that video actually you know the video or recording so we're going to see that
to watch each one of them.
OK.
So unless they get points for stuff.
But the biggest Poinsett Yes the most prestigious for the best of us.
Well it's a continuous war.
You know when it's approaching the west it's getting a continuous OK.
If it's getting further away from the base it's getting it's getting the negative rewards.
For example at one Mannis 41 it got further away from the vest or it got killed.
Yes getting killed also gets you back to work.
Gotcha.
OK so walk me through the main section of the code.
OK.
So so at first we made the brain the brain yeah.
Yes the convolutional areas which are the eyes of the eye and the forward function that propagates the
signal inside the brain.
OK.
OK then we have to we made the body and that is the part of the eye that plays the action.
OK.
So we see this we used the soft Max way of getting the airplane reaction of.
What's the best action you know.
Yes.
And so the cool thing about this next method is that definitely the best action is playing.
But the other actions are explored.
Yeah.
Because it works on a probabilistic approach.
You know the other actions have low probability to get played.
But still it's probably a yes or no no.
So sometimes since then we know we take a random drop from a distribution of probabilities of the actions.
Well sometimes these other actions are played.
And so that's a way of exploring that.
You know that's something we talked about on the Internet or else the fact that you got to combine exploitation
and exploration.
Right.
Exactly.
Otherwise you might get stuck and walk on that black smoke and you'll never find the other Maximals.
Exactly right.
And by the way guys as you can see I think that's the expression face being over because right now the
average word is definitely increasing.
Look you know we went up to 697 But then we went down then went down again.
But now we reach 525 then eight hundred and eighteen.
I think the ice starts to get smart and I think the average word might be increasing.
That's great either.
Take my soul.
Me too me too.
I was very surprised at the beginning because actually we oh I see.
Next ever increasing.
So I think now the exploration phase is over exploring and I think it's it might be reaching very soon
when exploration phase is over.
Is it fully over or is it so randomly sometimes not it's not fully over but is less likely to explore.
Yes that's likely to explore.
That's right.
Now I bet that the average work is going to keep increasing.
That's a reach you know the last checkpoint which is this 1500 rewards buys me off.
It's like when you come to your country you go to because they have new groceries different groceries
which you said you've got your grocery shopping experimental something something something you don't
like in your life.
And then sometimes figure out oh this is what I like to have for lunch and then if you're just buying
power.
But occasionally you'll go out and buy something new to see how you like it.
Yes I bet you do that often.
Exactly.
And then once in a while you'll find all are so cool and you'll change your pattern or have a little
bit yes.
I think that's that's kind of like something that's a good analogy.
Yes.
All right so we might reach the final average soon but let's continue on the cold.
Yes.
That's a great way to summarize it.
So as we made the brain in the body we assemble the brain in the body to make the whole day.
And that's the section Yeah.
And we have this whole fourth function that propagates the whole thing from you know the original input
image then through the brain and then through the body and then the actual space.
So it was actually an analogy you know to make your brain and thinking is actually a body.
Right now you would never see the eye with a brain in the body that any of the resorts.
It's just the way we're seeing it.
So the brain makes decision.
The body then acts on decision.
Yeah exactly.
That was just a way to play with the objects and object oriented programming to make it into his head.
OK.
So that's part one making the hand with the brain and the body.
And then part two is of course is of course the training of the.
So in the training of course the employment eligibility trace which is an instant cure and then the
deep convolutional Kiran is that instead of learning every step you're getting.
We want every step and getting to every step we compute accumulate if we win and cumulative targets
and steps.
And we did it on 10 steps and that improved the training a lot.
That's pretty good.
Yeah.
And then we make a movie average and that's.
It's like a school function.
And finally of course we train the eyes.
That's the cutting thing.
I know we have our inputs they go into the new one at work and then we get in there because we also
have a target we can get a prediction to the target that generates a loss than the last areas back propagate
into new network.
And then we place this into data weights to try to minimize the distance.
Well actually the square distance between the predictions and the time it's yes.
So that's exactly what we do here.
Plus again we're getting deeper in working with our networks basically.
And then finally we print this out to our inbox at each epoch so that we can see and keep track of how
the ice cream.
And finally we print if we reach 1500 We print that's something we still need to get to yes.
OK.
So how's it going right now.
Okay.
So you were right.
It is still exploring a little bit because it's definitely reaching an average where the largest one
thousand year.
But still you know it's still exploring because we want it doesn't want to get to 1500 yet and do it
soon.
OK to start looking at the first video.
Okay.
And that's a good idea.
So there's video so we all we have four videos when you have so many more because it's not recording
that you frequent.
So it's got to zero.
But the first the age when you see a new one is just randomly.
Yes.
OK.
So here's the first video.
No doesn't know what it's doing right.
OK.
Then secondly it might not be much better.
Oh yes.
Maybe because it's nine seconds now.
So old Kilborne.
Yes.
Good.
Two to three.
All you got to the last.
I get to do that again Larry.
I know what it is.
This is a lucky one.
The Lucky One played played again.
OK.
So go book you it say that they're not ready for war.
That's that's that's the perfect catch.
But as we said is this the best strategy to reach to this is nicest possible.
Probably not.
So let's see how how the carrot is going.
OK.
Still so sick.
All right let's do the next one.
OK.
Next one
or two seconds.
He stopped shooting.
You decided to explore the pacifist version.
I get to make friends.
I was going to befriend them all to the vast majority.
So the next one.
Once again we all got there.
OK.
You got it.
Yeah but what if if you got the rest pretty much right away.
Yes.
Then how much like how would you get more points because you know the faster you get to divest the more
positive is your reward ouster.
Can you go even faster than that.
Yes I think so let's watch the next one.
Say he understood it he's not OK.
He was running towards the best as you can see we see some of the directions left and right.
I think the best best course when we just try right strips.
OK.
OK interesting.
But you can see that he's getting play play 27 on here.
Let's see.
You know this one's on
right now.
OK so they're pretty simple.
So guys I think you know 1500 was too much.
Yeah I think you know it might be a good average two to research.
Once you choose this 1500 by 1000 then maybe you'll get to video that with your winning.
Yeah I think you've got to experiment that as well.
So it looks like because it's like converging.
Right.
They go down 300 1300.
Yeah.
Looks like he's figuring it out.
But we can see that he's figured out that he's on to the play.
Let's wait until I put 25 maybe he reaches 1500.
But anyway you're right that was really quick.
That's.
The training didn't take much time.
That's also why you think that is because I think it's because of the LGBT to trace their back into
the method.
OK.
There are actually other bodies that can play the action.
There is also as we saw in the paper which is this paper you implement this algorithm.
I think is and.
That's actually we see paper from deep mine that's going to be critical.
But as you can see in this algorithm they take action according to Neptune reports.
So that's another that's another body that's another way of playing the action.
So that's also good.
But I wanted to end too many soft necks because I thought the results would be better.
But guys if you want to practice try implementing Ipsden grip.
OK.
So how are you going 23 so far.
So the 100 is going to go another video.
Twenty five.
Yes.
OK.
Interesting.
Well it's doing you know what I read recently.
No.
Why.
Computer games don't use artificial intelligence like we created.
You know like for example those monsters that are killing him.
Why.
Well that's an old game but even in New Games they don't use these algorithms full of monsters.
You know what.
No.
Why why why.
Well I think I need some more powerful algorithms in that because he's talking about the video when
it's killing over and over again.
Which one.
You know the video of the airplane.
Yes.
No no no no no.
I mean like if when you as a human you play a game.
Yes.
And against you there's AI AI that against you in a computer.
OK.
Why they don't use these algorithms.
Oh we can truly use these algorithms.
For example I almost implemented an algorithm to play against me.
I had the pong game yeah.
So we can.
But the thing is they don't on purpose that was impossible to beat them.
I think they do.
There is a soccer player.
Think about is a good game if they use something like this.
It just gets so good that you will never win.
Yeah.
Yeah that's why they have to really dumb it down.
Yeah.
So any artificial televisions playing even in the most basic game like chess and whatever.
But like especially in the more advanced himself going to be like this album because it's just got to
be impossible to.
This is going to kill you right.
Exactly.
Because I also get suspicious suspicion online gaming gets so much access to so many human players.
So you can train all different human play behaviors and you can understand what humans judiciously actually
instantly.
The guys in the video games are nothing like this.
Yeah.
Because as you said we would have no chance.
There's funny is like something somewhat post-box.
OK.
So how are we doing.
Well definitely increasing strength.
So he found a way right.
Look look at that like between 13 and 24 7 airports you're stuck at a thousand.
Yes the threat of thousand one hundred and now he finally found a break.
Maybe he's like Yavanna way John Foster somewhere maybe.
I think I just can't imagine how he can even improve even further.
Yes.
OK.
I remember he reached 1900 1900 when I went out there.
I was definitely running fast but then I said you know I don't think there's much difference.
And there we go.
Finally he recorded this video as well.
Yes it did actually add records to the air every five six months.
So wait see.
Just like that where we got this video.
Yes we are.
But anyway we already this guy.
So I thought yes I was going to ask you like what if he.
But we don't recall the winning.
Well we would like to but I guess I would have won it.
Let's watch the last one.
OK.
So congratulations guy.
Cissoko Fiat 1500
I can't really call.
That's the one right.
Yes.
And you know I don't think that's too busy 236 and right now is 2:43.
Where's the video.
Good question.
The video.
It should be very soon.
You can.
Or maybe dinner with you.
No it didn't actually I didn't finish writing results.
You know like I can die.
Well that's it it's all good.
Yeah this one is good enough.
That would be really very fast.
My friend that was on the peaceful way in trying to make friends this one I think is a shallow are not
a good idea actually.
The next time that's next and to really understand the difference.
Yeah.
OK.
That's pretty cool.
Yeah.
That's so exciting.
Hope Hope you guys got the same results.
And I think you guys might get the results before the epic's 26 because I actually tried it several
times.
And sometimes you can get them at number 10 or 15 but 26 is the first.
OK.
So that's the end of this module to know what is going on to the next module about celebrating your
breakout.
Yeah.
You guys excited for a while is break out the last one because it's the most difficult one is the most
challenging one and the simple reason for that's the main reason for that is that the little ball that
deep convolutional neural network has to detect is much smaller than the big monsters do much more much
easier to detect.
That's the reason.
And actually we once implemented the congressional Kulemin will implement the Botha's algorithm of course.
Yeah.
What we see.
Yay.
So exciting.
OK.
Good stuff.
So I hope you like it.
And congratulations for building your second act up.
Thanks guys.
See you in the next section here.