‫Welcome back.
‫How did you go?
‫Did you try and use the score method on a regression problem using a regression estimator?
‫I hope you gave it a shot.
‫If not, let's check it out.
‫And I just want to highlight something before we get into that, is that we have a training data set
‫here and of course, where we're concerned with how a model performs on the training data set.
‫But if you're looking to use your model outside of a Jupyter Notebook, which you most certainly probably
‫will be as you move forward in machine learning and whatnot, you want to really pay attention to the
‫test score.
‫That's the real score that that actually matters when you start to export your models, save them and
‫use them in an application.
‫Yeah.
‫Of course training is yeah.
‫You want it to perform well on training, but the test is really the unseen data.
‫That's the kind of.
‫And of course, the real test set is when you actually deploy your models into an application.
‫But for now, we can kind of simulate our model being in an application by hiding some data from it.
‫And so this score, I would pay more attention to the test score than the training score.
‫So just keep that in mind.
‫Now let's grab our regression code because we're not going to.
‫We've already seen this a few times, so we'll bring this down here.
‫We're going to practice the score method.
‫We don't need predictions.
‫So we've got import random forest regression because we're predicting a number.
‫We've got the housing data frame, the X, the features are everything but the target variable, the
‫labels is the target variable.
‫We're going to split it into train and test sets.
‫We'll create the model instance, which is the random forest regression.
‫Remember, we can simplify our model by setting an estimators so the default is 100.
‫How about we start with two and see how poor our model performs so fit?
‫The model should be pretty quick because I just want you to let you know as well where we're fitting
‫on 20,000 samples here.
‫But the higher this number goes, as you could imagine, the longer the model takes to fit Y.
‫Because remember, with an ensemble, it's a combination of many different kinds of models.
‫So if we're only fitting two, it's only going to take 2% of the time it takes to fit.
‫100 and a thousand is going to take how long?
‫Ten times as long as 100.
‫Now, of course, those numbers might not be 100% correct because of how computing scales and multi
‫processing and whatnot.
‫But just rule of thumb, the more estimate as you have here, the longer your model is going to take
‫to fit.
‫So let's try that.
‫We'll fit it again.
‫And now we're going to score it.
‫Using the school method on the test data.
‫How does it perform?
‫0.7.
‫Huh?
‫Now let's rewrite this so we can just store it there for the time being.
‫Why test now?
‫That's with two estimators.
‫Let's.
‫Let's upgrade this to ten.
‫What happens all we get a boost in performance.
‫Look at that.
‫We get eight points there.
‫Now, if we want to have a look at what the metric here that's being used for the score method, how
‫might we do that?
‫We can check the docstring docstring for a regression algorithm.
‫And so I can't learn the default score method.
‫Evaluation metric uses the coefficient of determination, which is another word for another series of
‫words for R-squared value.
‫If you'd like to read more into that, the Wikipedia page is great.
‫I'm sure there are some fantastic YouTube videos out there, but essentially it is also about coefficient
‫denoted R squared or R squared.
‫Pronounced R squared is the proportion of the variation in the dependent variable that is predictable
‫by the independent variable.
‫And again, these there's a lot of different terminology in the data science world, but we have the
‫dependent variable is our target because let's have a look at the housing dataframe housing.
‫DF So this is the dependent variable Y because our assumption is that this variable is dependent on
‫these other variables.
‫And so these are also referred to, I'm calling them features, but they're also called referred to
‫as independent variables because these independently combine them make the dependent variable.
‫So that's how I interpret that.
‫You might have a different interpretation, but that's how I interpret these things.
‫And you're going to build up your own intuition over time with different names of things.
‫But I like the name features because these are all like medium income.
‫That's a feature that you use to predict the target variable in combination with all of these.
‫So this is the r squared value.
‫Let's now upgrade it.
‫How about we try 50 estimates?
‫What happens there?
‫So it was seven eight before.
‫See how it took a little longer that time to score again.
‫So we pushed it up to 80.
‫Remember here as well.
‫I just want to remember remind you the default score evaluation metric is R squared for regression algorithms
‫and highest equals 1.0 lowest equals 0.0.
‫So if our model perfectly predicted the.
‫Test values, we'd get a score of 1.0 if it predicted the mean of the test values, we'd get a score
‫of 0.0.
‫That's just how the R squared function works.
‫So if we have, why test y test?
‫There's all those values there.
‫If we got the mean of that mean, if every single sample in our predictions predicted that we'd get
‫an R-squared value of zero.
‫And I'll let you test that out if you like.
‫But let's push this up a bit.
‫Let's go to 100 and see what we can get.
‫We'll score it out again.
‫It takes a little bit longer than when we just had 50 takes a lot longer than when we just had two.
‫Okay.
‫So we don't get much higher.
‫So it looks like 80 is about the tipping point there.
‫I'll let you explore with different values.
‫But that's we've just covered a major component of evaluating our model's metrics and scoring, quantifying
‫the quality of predictions.
‫We've gone through this one.
‫So the next thing that we're going to do is start looking into how we can use the scoring parameter.
‫So if you want to jump ahead, you can read here the scoring parameter and just see what different metrics
‫there are and see if you can figure out which ones are for regression and which ones are for classification.
‫I'll see you in the next video.