Hello and welcome to this story almost the final title of this module.
I'm just going to explain the main code that will execute the whole thing before we get the exciting
results and watch the videos.
So this is the main code and as you can see this is pretty short.
We start by importing the libraries and the modules and also the different classes and functions we
made like Altikriti from our model file.
The train function from the train file and the test function from the test file and of course we import
our optimizer.
Then we start with the first section where we get into a class all the parameters.
And this is this perhaps.
Remember this is BRAMs object that we created from this harams class.
Each time we got a parameter like the learning rate we are not without.
So let's go through them quickly.
This first one or here is the learning rates.
So as you can see we choose a small and you rate the second one is the parameter.
Again we take it as 0.39 we take a OneTel parameter a set of 1 16 processes 20 steps and max length
of 10000.
And we spoke about that this is the parameter we set to make sure the agent doesn't get stuck indefinitely
into a state of the environment so this will stop the game.
If the episode length goes over this maximum length and eventually of course we get the name of our
environment.
Break up is zero.
And by the way you can also play on some other environments just by changing the name of the environment
here.
So if you want to play to some other breakout versions or even some other Atari games well you can simply
replace this breaking of zero here by some other games.
But I can tell you that breakout video is already very challenging.
All right so all the parameters here.
And then there is the main code for the main run.
And so here lets see what we do in this first line.
We set one thread per core.
Then in the second line we get all up renderers by you know creating a new object of the Paramjit class
which will get initialize all these parameters here because there are variables attached to this BRAMs
object.
Then we set the seed.
Then we get our environment using the Create tree and function with the name of our environment which
is break out of zero.
You see cells that name and therefore parameter and name is pretty of zero.
So that will get us the environment of break out.
And by the way this is not the usual way of creating an environment but you know to improve the whole
process and to improve the performance.
Well we use this to actually create an optimized environment and this we do this things the universe
universe is a package that comes with all the packages you installed on open engine.
Well thanks to universe we get an optimized environment.
This is what its all about here.
Then we get our shared model by creating an object of the active critic class.
And so here it's important to understand that this shared model is the model shared by the different
agents.
So we have different threads in different course.
And speaking of threats at the next line here shirred model that share memory what we do is we store
the model in the shared memory of the computer so that all the threads can get access to it even if
they are in different course.
So that's what we do here.
This is to enable this then we get our optimizer linked to the parameters of our shared model and with
a learning rate of one point or one.
And again it's important to understand that the optimizer is also shared because it's going to act on
the shared model and saying that the next line optimized that share memory we store the optimizer into
shared memory so that all the agents can get access to it to optimize the model.
Then we initialize our processes so the test process doesn't update the shared model but it just uses
it to try it in one part and print the score and record the videos.
So that's exactly what is done here with Target equals test.
That's the test process and this process here is cut from torture that multi pre-processing.
So here and what it does is that it basically runs a function on an independent thread.
So then when we do restart we start a new process which was the initial Last year at this time.
And then with this process to append P we add the process in the list of the processes.
And finally in this loop here we just do a loop to run all the other processes that will be trained
by updating the shared model.
And that's basically what happens in the last lines of code here.
So if you don't want to get into the details of it the important thing to understand is that this will
run the processes in an optimal way and therefore we should all be good to execute this code and to
have a trained model and eventually watch the results.
So I can't wait to do that.
This is going to be pretty exciting.
I will try to find people now so that we can all watch it together.
And so until next time I.