Hello and welcome to this tutorial.
Congratulations again for being done with it.
See we made it we made the brains and we trained them.
But now we still have to make a test agent which will not a date tomorrow at all but it will just use
the share model to make its own explorations.
And of course in this code we will record some videos and these will be the videos test agents break
out with a certain score.
So let's go through this code.
The most important is done so as I told you we're not going to code it line by line but I think it's
important that you understand what's going on here.
So there we go with this code in the first section as you noticed.
We import the libraries and then we did find this test function which will make this test agent to do
its own exploration and play the breakout game.
So we get this test function takes three arguments the first one is ranking that is still to do synchronize
the test agent as we did for the training agents.
Then we have our parameters of course because you need some.
And of course we have the shared model because this test agent will use a shared model to do its own
exploration.
All right then we go inside the function and this line of code we synchronize to test agent.
Exactly as we did before then we import the environment.
So I remind that in the main code which will be in the next tutorial.
Well and name here will be replaced by break of zero so that we can go into the break zero environment
and play the game and the Red Cross Trumans that will get the videos of our evening break out.
So basically this line of code in total means that we run one environment with video.
Then at the next line of code we synchronize this environment so exact same principle as the Trend function.
Then we get our model and to do this we create an object of the activity class and we put the input
shape with our environment observation space and shape zeroes are exactly like the train function and
our outputs which are the actions with an action space.
So exactly like before then something new here since we're done with the training.
We don't want to put the model in train mode because simply we don't want it to train we want to put
it in development.
And that's what we do here with model that evolved.
So that's just basically to put the test agent in a mode that will basically test it tested performance
evaluated.
Then here we get our input states which are the input images from the game which at this point are an
entire race.
Then we convert them into torch dancers.
Here we initialize some of the words here.
We initialize down to true.
So still just like last time then something new again we introduce this third viable with a time function
to measure the time of computations.
And that's because you want to get the starting point.
Then here the actions we use a very practical type of cue that allows to add an element to the cue from
the right or from the left.
So that's very practical and I'll give you the reference I think in decremented version of the code.
So you'll have a look at what this dequeue is and that's what allows to do that.
Then we initialize the length of an episode with zero of course and then we will increment the size
in this well loop.
So we use the same trick here.
While true and in the loop we increments the length of the episode by one.
When the game is done when the game is over we reload the last set of the shared model the share model
that the dated by the other models.
Remember that here the shared model is no longer dated then.
Still if the game is over if the game is done we Reinette we re-initialize the cell states see X and
then States H x and else if the game is not over well we keep the same cell States and in states.
But to make sure they are taught variable so that they can be attached to a gradient.
OK so that's something we already dead in the trend function and then still in the while loop and after
having a data that states in the hidden states the right way depending on the two cases here.
Well what do we do we get the predictions of tomorrow.
That's exactly what we do here with this line of code.
So we get the value which is the output of the critic the actual value which is the output of the actor.
And then it's up all of the hidden states H x and the cell States the X then we generate a distribution
of probabilities of the actions that is at the Q values action value here.
And we do this with the next function.
And of course we don't need to get the luck probabilities here because this is just for the training
for the test agent.
It will just play the actions we will just use it you know like doom a certain activity to play it but
we're not doing any training here.
So we have just a prop and from this we play the action by taking directly to RMX of these probabilities
that is it takes the action that has the highest probability.
And the reason is that the test agent doesn't do any exploration.
Remember that we want have a chance to pick up some actions that have low probabilities when you want
to do some exploration of these other actions and you know not taking each time the action that has
the highest probability but here test agent can do any exploration and therefore that's why we directly
take the action that has the maximum probability again then once we play the action we reach the next
state and we get the next word.
And that is a day dated whether or not the game is over.
So this we get all this with this line of code by playing the action after having selected it with our
Max here.
So we play the action here and we get the state we get the reward and done it again and then since we
just got a new reward We're going to update some of the reward by simply adding this new word.
And finally whenever the game is done.
So if that means when the game is done when I finishes playing the game well we're going to print the
results with the time the opposite.
We wanted the length of the episode that is how much time did it last.
Playing great out and this is just how we print all these variables using these tiny tricks.
That's for the time then we want some it's just a variable.
Some of the words and ideas at length is the of the length of the present OK.
And then once we printed all the results well since the game is over and we want to start a new game
we're going to re-initialize everything.
That is the sum of two words zero the length of an episode to zero.
We're going to re-enact all the actions by using this key function resets the input images you know
by repeating all the breaks altogether.
And finally we use this time that sleep 60 seconds to do a break of one minute to let the other agents
practice.
And that if the game is over.
OK.
And finally we have this last line of code which will get us the new state and then we can move forward.
We can continue in this new game.
So there we go.
That's the test function.
Things to which you will see the videos in one or two tutorials.
I hope you will be altogether like last time to watch the results that is with you.
Curial and me that will be fun.
And I'm telling you.
Expect to see good results.
But keep in mind that this breakout game was super challenging.
We thought it was a simple game to play first but not at all.
It actually turned out to be much more difficult than doom.
And that's why we put it in the last module.
But anyway let's make this main function in the next tutorial.
Same This is not the most important here.
Now that the A-380 is demented we will not code it line by line it will expand code and very quickly
we will get the results.
Until then enjoy AI.