‫Welcome back.
‫And the last video we saw are to score or the coefficient of determination or r squared, a few different
‫names for the same thing.
‫And I've just put a little note up here just to clarify what we've discussed.
‫So what R-squared does compares your model's predictions to the mean of the targets.
‫Values can range from negative infinity, a very poor model.
‫So a model that predicts wild things will be a very low score to one for a perfect model.
‫For example, if all your model does is predict the mean of the targets, its R-squared value would
‫be zero as we saw.
‫And if your model perfectly predicts a range of numbers, its r squared score would be one.
‫So if it perfectly predicts the target values, it gets an r squared score of one.
‫Let's go down.
‫And now I issued the challenge of checking out mean absolute error and mean squared error, which if
‫we go back to regression matrix main square error, mean absolute error.
‫There's a few more that we could look, but I'll leave those as extra curricular.
‫These are some of the most common ones you're going to come across.
‫So let's dive in now to mean absolute error, which is Maya.
‫And then Maya is the average of the absolute differences between predictions and actual values.
‫We'll see this in practice.
‫It gives you an idea of how wrong your model's predictions are.
‫So let's say this.
‫See it in action.
‫We can go.
‫Maya, very commonly abbreviated or mean absolute error is very commonly abbreviated to me from silent
‫metrics, import, mean, absolute error.
‫And then we're going to set up Y threads.
‫We'll use our model that we created before model up predict on the test data set Y Because evaluating
‫a machine model, we want it to predict on data.
‫Of course we can evaluate over the training set, but most often you'll be more concerned with how it
‫performs on the test data, set data it hasn't seen before.
‫So make predictions on the test data set and we can set up Maya as being equal to mean absolute error.
‫This is another one of those beautiful functions that psychic loan has built in, and the premise of
‫every or many evaluation functions in psychic loan is to compare the true values to the predicted values.
‫So then if we go down to MIA, and so we just figured out that mean absolute error is the average of
‫the absolute differences between predictions and actual values.
‫We wrote this beautiful little function or we imported it actually.
‫So I can learn right that for us.
‫Thank you, psychic loan.
‫And we're going to compare we're going to get the mean absolute error of the Y test values to our model's
‫predictions that it made on the test features.
‫Let's have a look why pred is not defined of course y parens.
‫Oh, so the ma is 0.32.
‫Now this is just numbers on a screen.
‫What does this actually mean now this value?
‫Because of the nature of ma, it means that on average let's have a look at why spreads and let's have
‫a look at y test.
‫On average, each one of our predictions is 0.3, two plus or minus the Y test value because the absolute
‫in there gets rid of the negative sign.
‫So it makes all the negatives positive.
‫So you're going to get a positive value here.
‫Now let's have a look.
‫How about we make it so it's a little bit more visual?
‫Let's compare them.
‫We'll create a dataframe, PD dot dataframe.
‫We'll set the data equal to a dictionary equals actual values.
‫So this will be the I'll let you guess, the test data of course, and then we'll do the predicted values.
‫And this will be why reds and then we'll create a differences column by going DF differences equals.
‫So this has just be the difference between our true values and the predicted values.
‫So predicted values.
‫We might turn that into absolute though actual values before we do that.
‫But why don't we just run that and then have a look at our DF and we'll get the first ten.
‫Let's visualize me.
‫All right, there we go.
‫So we have the actual values and now the index here, of course, is going to be shuffled because we
‫shuffled our test data.
‫So the actual values and the predicted values.
‫So this is the differences before they've been absolute ID.
‫Is that a word?
‫Absolutely.
‫Well, we'll treat it as a word.
‫We'll see what that means in a second.
‫So we've got a difference.
‫This one was very close.
‫So 0.01 difference.
‫This is more closer to the actual match there.
‫That's a bit lower.
‫That's a bit higher than the mar value.
‫So again, it might it means on average that difference.
‫So some will be smaller, some will be larger, like this one is quite large here.
‫That's nearly 1.0 difference.
‫Now what if we took the average so dif differences.
‫What if we took the average of that column mean oh, we get a different value.
‫Right.
‫But the reason why why do you think that that's different to the MIA?
‫We hinted at it before.
‫Is that because the main absolute error turns these gets rid of the negative.
‫So if we just go the differences main, it's going to include the negative there and it's going to include
‫the positive and we'll get a slightly different value.
‫But what if we go maybe NP is it ABS absolute for MP Absolute, then get the main.
‫There we go.
‫Look at that.
‫Beautiful.
‫So that's just the formula for main absolute error.
‫We've got our actual values y test, we've got our predicted values.
‫We calculate the difference.
‫So we go actual values, minus predicted values, get the absolute of that and then we take the mean.
‫So that's just MIA using formulas and differences.
‫Now knowing this, how do you think we could do mean squared error?
‫That's your challenge for the next video we've done main absolute error.
‫I encourage you to go back to the documentation, look up the docs for a mean squared error.
‫It's going to be very similar to what we've just covered.
‫So just to reiterate, my E is how different our model's prediction is on average to the actual value.
‫Now you'll have to remember that this is in a scale of hundreds of thousands.
‫So even though this is 0.29, that's $29,000 different.
‫So just keep that in mind and Maya will give you that value.
‫But the value is going to be in the same scale as the data you're working with.
‫So give that a shot fit mse mean squared error and we'll do it together in the next video.