Hello and welcome back to the course on artificial intelligence.
In today's tutorial we're starting off the section on deep convolutional cool learning.
So let's have a look at what it's all about.
Previously we talked about deep cool learning.
So we had an environment where an agent and we had a vector describing that environment which was fed
into a neural network and at the end we got the q values as our outputs.
And then of course we found out how to restrain the learning part.
We found out how actions are decided based on those values that's an action part.
And we talked about action selection policies and different things about how deep learning works.
But here the key concept for all of this is how do we get from this from the actual environment and
the states to the neural network.
Well the transition is over here.
The input vectors so the input Lehre of our neural network and it is a vector.
So what we're looking at is OK so we're actually edging on the curve.
That's not the correct term we're not looking at anything.
The agent basically has this information.
So the environment is parsing it this information saying OK you the agent you're currently in this your
state is described by the sector in this simplified example it's described by this vector X-1 of 1 x
2 of 2 so your coordinates are 1 2 and that is your whole state in a in a more complex environment.
The statement and all other things that the agent can be observing but the point here is that it is
possed as a vector.
And the thing is that that doesn't happen in real life in real life except for GPS systems and other
things like that.
But in real life what do we use most of the time we use our senses we use our eyes even in GPS.
It's not built into our brain.
It's not telling us the coordinates through our brain.
So we're still using our eyes to look at at the GPS and understand what's going on there.
And so this is kind of cheating for AI to be able to get like information about the environment as a
vector.
It's too simple it's not how it works in real life.
That's not how we as humans operate and ultimately we want to create artificial intelligence which can
operate in a similar fashion to humans which is as it can take on the same challenges as humans.
And so in the human world we don't have that we don't have that we don't have these coordinates or other
types of vectors that are passed to us that explain the state we're in in that environment.
So we're going to have to remove that to make it more realistic.
And then what can we replace it with what do we see or what do we do as a human to get information.
Well most of the time we see of course all our senses but most of the information that we're getting
about the out the world around us comes through our sight.
And that is why we're going to change that little arrow which we had into a whole convolutional neural
network.
So this is from our Onix.
Number two we've got the convulsion Larry and that's why it's important to be quite comfortable with
evolution conditional neural networks and how that works are you if you've done ODP intercourse and
you should be comfortable with that.
Or you can just have a look at the next number two we've got some very good intuition tutorials there.
So here we've got the convolutional operation which happens so we're actually going to be looking at
this as an image.
So this is an image of Net environment.
And so the agent is actually looking at the environment.
So in this case not that he's like looking from within there he's like looking like that.
Let's say he's playing this on a computer and he can see this environment and therefore he can see like
where this figure representing agent is actually actually is.
You can see his whole in viral or whatever a human would see if it's actual maze and the human would
see the maze from inside.
And so the agent should be able to exactly the same thing.
So what he says is done goes through a commotion lair you go sort of fool pulling a leg goes there flattening
again you can find out about more about these different parts of convolutional a neural network in the
annex and then officers flattened.
Then we have inputs which go into the neural network and this is a way more realistic because the agent
has to use their sites and or has to process images which the environment is supplying to the agent
just as a human would be processing images.
And the beauty of this is not just that it's more realistic and it's kind of like more as a hue.
The age is actually the more as a human would be but it allows us to process much more complex environments.
For instance this is how we can play Doom or other games like that because instead of just getting a
vector of information which like somebody would have created for us in this environment we can just
hook up artificial intelligence to any environment which as humans have a vision of is and.
As a human when you're playing this game you can see exactly this picture and that's exactly what the
artificial neural network or the agent would see now.
So in this part of the course when you're going to probing the practical Tournelles the agent will actually
see this exact picture it'll see the pixels it will get this exact picture full of the pixels of this
person with this we've gone with this face with this percentage with everything exactly what we see
here that's exactly what the agent will see.
Then it will have to dissect that through pulling liff fattening and then it'll go into a neural net.
And needless to say that the neural networks actually be much more complex than that so let's replace
it with something like this.
This is not much more complex.
This is looks a little bit more complex but in reality the neural networks are going to be working with
and creating if you are going to be quite interesting and going to be much more complex than this.
But as you can see already here even if you just have five inputs and sort of two things become a much
more complex and here you can see we have many more actions that the agent can take.
So in the game of doom turn left and right look down look up Ron's shoot reload.
Or you know all those different actions that are possible in first person should like like do.
And moreover it doesn't have to be that you can.
You can touch this agent to another type of game.
That's the beauty of it that it's then realizes that it can now operate any kind of environment that
you attach it to because as long as there's like a visual representation of environment of that environment
it's already got the whole infrastructure the whole structure is ready to process that.
So that's what deep convolutional CULE learning is all about.
So we're taking it even to the next step we're adding convolutions into or convolutional Lares into
our agents brain now and we're making it even more complex and therefore we can we're bored with being
able to solve even more complex challenges.
So I hope you're very excited about this is going to be in an epic section and we're going to create
some amazing things and I can't wait to see you on that next tutorial.
And until then enjoy AI.