‫Welcome back.
‫And in the last video, we checked out how to tune a model's hyper parameters by hand.
‫And we figured out by the end of it once we'd written some code ones, we've split our data into train
‫validation and test sets manually.
‫We created an evaluation function.
‫Yeah.
‫And at the end we figured out tuning him all by hand.
‫That sounds like a lot of work.
‫And you're right.
‫It would be if we went through and tried a whole bunch of different parameters.
‫But luckily the developer's psychic loan have also come across this problem and have created Randomized
‫Search CV, which stands for Randomized Search Cross Validation.
‫Let's see how we'll tune high parameters.
‫So AKA adjusts the settings on our models to make better predictions or hopefully make better predictions
‫using randomized search.
‫CV So let's go here and create another heading.
‫This can be 5.2, so hyper parameter tuning with randomized search.
‫See v beautiful.
‫First thing we'll do is we'll import it.
‫So from SRC learn dot model selection import randomize name.
‫We might be able to press tab here.
‫Randomize search CV.
‫Beautiful.
‫Let's see an example of how we use it before we check out the docstring.
‫So what we'll do is we'll create a grid of hyper parameters and by grid I mean dictionary.
‫And I've typed Grodd here a dictionary of hyper parameters.
‫We'd like to adjust.
‫So if we come back up here, these are the parameters we're going to adjust.
‫So what we're going to do is create a dictionary with the type of parameters we'd like to adjust as
‫the keys and then the values we'd like to try as the values of the dictionary.
‫So let's do that.
‫So in estimate as that's a key, we're going to get a positive list.
‫So 102 hundred, 500, 1012 hundred.
‫Beautiful.
‫Now, if you're wondering, Daniel, like, where are you getting these values?
‫Well, remember, as always, with any model or estimate or in psychic line, there's extensive documentation
‫on it which tells you some of the settings that you can change on the model.
‫So what I've done is I've read through here, read through the examples, done some research and found
‫different values for the high parameters we can try.
‫So go here.
‫None.
‫Five.
‫Ten.
‫So if it looks like I'm just throwing out values here, trust me, they're not just sort of throwing
‫out values.
‫They are based upon some research and some experience.
‫And don't worry, you won't begin with that.
‫No one begins with that.
‫It takes a little bit of practice to go through it.
‫That's why we're having a look hands on at all these different functions that we can use to improve
‫our models.
‫Wonderful.
‫And then we'll do two more, which is min samples split.
‫Two, four, six.
‫Come here.
‫Men samples leave.
‫And let's see how we'd implement randomized search.
‫CV So we'll set up a random seed.
‫Well split into X and Y.
‫So you want heart disease?
‫Shuffled, actually.
‫Because that's what we've used before.
‫Dot drop.
‫Oh, actually, we need to make X here.
‫Don't drop.
‫Target access equals one we'll make why here so hot disease shuffled.
‫Target.
‫Beautiful.
‫And we're going to go split into train and test sets.
‫Huh?
‫Now, I'd be wondering, why are we only using train and test here?
‫We've just created a validation set.
‫Now we'll get to that.
‫Why train?
‫Why test, train, test, split x, y.
‫And we use a normal split here.
‫So test size equals 0.2.
‫Wonderful.
‫And then we're going to instantiate.
‫Random forest classifier.
‫Self equals random forest classifier and jobs equals.
‫We could do negative one here, but at the moment negative one is broken for me.
‫Now that'll make sense in a in a second.
‫But in jobs stands for how much of your computer process are you going to dedicate towards this machine
‫learning model?
‫And negative one means all of it.
‫So by default end jobs is one are actually what is the default in jobs?
‫Let's go here.
‫End jobs equals none.
‫Okay, so we'll set a lesson on actually, I'll just settle this one.
‫So different values you pass to end jobs will dictate how much of your computer processor that you want
‫to dedicate towards a machine learning model.
‫And now, once we've got a classifier instantiated, we'll set up randomized.
‫Search.
‫So we're going to pass it.
‫We're going to call it RS Clef.
‫So what that's going to do is it's just appending RS to it for randomized search and now randomized
‫search.
‫CV The first parameter it takes is estimator equals clef.
‫So we're passing it this this random forest classifier.
‫We've, we've instantiated parameter distribution is the next one.
‫We're going to pass it out grid.
‫So this is grid up here.
‫And the next thing is we're going to define an ETA, so equals ten.
‫And this is the number of models.
‫To try.
‫We'll do five fold cross-validation and then we'll set verbosity.
‫For both eagles to.
‫Whoa.
‫This is some code we haven't seen before.
‫So what is happening when we run randomised search?
‫CV we look at the docstring.
‫Press shift tab in here.
‫It's because we have an imported.
‫That's right.
‫We'll press shift tab here now.
‫So what's the docstring here?
‫Randomized search on hyper parameters.
‫Randomized search.
‫CV implements a fit and a score method.
‫It also implements predict, predict, probate decision function transform and inverse transform if
‫they implemented in the estimated used.
‫So randomized search on hyper parameters.
‫Hmm.
‫That can sound a little bit confusing.
‫What does it mean by randomized searched?
‫So.
‫That's in the name randomised search CV.
‫CV is for cross-validation, which is where you might recognise this parameter.
‫See vehicles five.
‫So that means we're using five fold cross-validation.
‫So five fold cross-validation.
‫This is why we don't necessarily have to create a validation set here.
‫So what randomized search CV will do is it'll take our classifier.
‫And it will take a parameter distribution's grid, which is this, and it's going to search over this
‫grid.
‫410 different times, different combinations of these parameters at random.
‫So for example, on its first iteration, so because it's doing NIDA equals ten, it's going to do this
‫ten times on its first iteration.
‫It might try a model with ten estimators, a max depth of nine max features set to order min samples
‫split set to two min samples, leaf set to one and then on the next iteration.
‫So iteration two out of ten, it might try 1000 estimators 30 as the max depth auto is, the max features
‫four is the min sample split and then four is a min samples leave.
‫Then it's going to keep going that up to ten.
‫Now we could set this to 100 to 1000.
‫I don't think there's even a thousand combinations in here.
‫If you went through, you could go six times five.
‫That's 30 times two.
‫That's 60 times three.
‫That's 180 times three.
‫That's 460 odd.
‫And the math may be a little bit off there, but you get the point, right?
‫If you went through all of these and tried to combine them in every single combination, that's going
‫to be a lot of different models.
‫So let's see this in action.
‫So once we've set it up instantiated randomize search CV, we'll go here fit the randomized
‫search CV version of self.
‫Don't ask r's self don't fit x train the y train.
‫Now the reason we only have to fit an x train and y train is because the CV and randomised search CV
‫stands for cross validation.
‫So that is what it's going to do.
‫It's going to automatically make our validation sets for us, which is a beautiful thing.
‫So if we come back to our diagram, it's going to take our data and then it's going to try different
‫hyper parameters cross validated on different hyper parameter settings.
‫So that's what it's going to return.
‫It's going to figure out which combination of these high parameters is the best, up to ten different
‫models.
‫And again, we can increase this if we wanted to try more combinations.
‫So let's see it in action.
‫So this is where we've set verbose to to it's going to output and tell us what's going on so fitting
‫five folds for each of ten candidates totaling 50 fits.
‫A.K.A. it's trying ten iterations of different combinations of parameters in this grid and splitting
‫each combination five times because see vehicles five totaling 50 fits on the data.
‫So it's going to run this fit function 50 different times using different hyper parameters on different
‫splits of the data.
‫So we can see here the first split it's trying is an estimate as equals 1200 because this is one of
‫our hyper parameters up here.
‫And remember, each split, each iteration, it's going to pick combinations of these at random.
‫It's also set min sample split to six year min samples leaf to two max features the square root and
‫max depth to five OC.
‫And then if we were to scroll through here, we can see all of the different combinations it's tried.
‫So let's pick one at random.
‫This one here.
‫So this combination is trying an estimate as equals.
‫200 men's samples split equals six min samples leaf equals two max features equals square root and max
‫depth equals none.
‫Then it's going to keep going until it's finished.
‫It's going to give us a warning here.
‫So let's see.
‫Once it's finished, what we'll be able to do is call best params on it.
‫So if we go rs seal clf best params and this is going to show us the combination of hyper parameters,
‫which combination of these got the best results.
‫Wonderful.
‫So we're an estimate is equals 200 min sample split equals six min samples leaf equals two max features
‫equals square root and max depth equals none.
‫They were the best cross validated results across ten different models.
‫And now when we call predict on our randomized search classifier by default, it's going to use these
‫parameters.
‫So instead of finding these by hand, randomized search CV has found them for us.
‫So let's do it.
‫Let's make predictions with the best hyper parameters.
‫It's what we might do.
‫RS Y Preds Eagles.
‫RC Left predict and we use.
‫We could use the validation set here, but in our case we're going to use the test set and then we'll
‫go evaluate the predictions.
‫RS matrix equals evaluate.
‫Why test?
‫RS Why Prince?
‫Well, now function is supposed to be evaluate Prince.
‫There we go.
‫Wonderful.
‫So if we come back up here.
‫Did we see an improvement here?
‫No, we didn't.
‫And so this is this sort of where the experimentation comes in, hyper parameter tuning.
‫You won't always find an improvement after running something like this.
‫Maybe we could run it for longer and try 50 different combinations.
‫And after it's tried 50 different combinations, it will find some parameters which end up being better
‫than our manually tuned result.
‫But what I hope you're starting to see is that using randomized search KV rather than running through
‫all these different settings by hand, it gives us a way to codify or function ais the tuning of hyper
‫parameters.
‫And so now there's one more way we can use to improve our model's hyper parameters and it's with grid
‫search.
‫KV So it's kind of similar to randomized search, but it's got one, one key difference.
‫We'll have a look at that in the next video.