‫Okay.
‫So we've got a confusion matrix and we've got a rock curve as well as an AUC metric.
‫Let's write this down.
‫Actually, let's communicate with ourselves.
‫So now we've got a rock curve, an asymmetric and a confusion matrix.
‫So we've got a rock curve.
‫An asymmetric and a confusion matrix.
‫Let's get a classification report as well as.
‫Cross validated.
‫This is a key one here.
‫Cross validated precision recall and f one score.
‫Beautiful.
‫So if we want a classification report, we've already imported the function there, so we can do it.
‫We go print classification report.
‫Now Classification Report is going to give us the precision recall and f one score.
‫But let's see it first classification report.
‫We can find it.
‫I'm passing it.
‫Why test and why?
‫Preds Boom.
‫This has given us everything that we need, right?
‫Of course.
‫Validated precision recall, fine score.
‫Well, not really.
‫The reason being is because this classification report.
‫And it's been on Y spreads.
‫It's only being done on one set of test data.
‫So we've got a classification report.
‫It's been found by this.
‫And we can tell that by coming up here back to where we made predictions.
‫So we only made predictions on the test data set.
‫So if we come back, we only made predictions on here.
‫So what we need to do is a cross validation score of precision recall and f one, which is the values.
‫That we've got here, precision recall and f one.
‫Now, he might be looking at this going, hmm.
‫Daniels telling me we need to cross validate precision recall and F1.
‫Still not exactly sure.
‫I kind of forgot what's actually going on here, but that's okay.
‫Let's get a refresh of what's happening.
‫We've got precision.
‫So precision indicates the proportion of positive ID.
‫So a.k.a the model predicts class one which is has heart disease which were actually correct.
‫So a model which produces no false positives.
‫Right.
‫So a false positive up here is false positive predicted one when it's supposed to be zero.
‫A model which predicts none of these has a precision of 1.0.
‫All right.
‫So that's precision.
‫Now, recall indicates the proportion of actual positives.
‫Which were calorically classified.
‫So a model which produces no false negatives has a recall of one.
‫So a model which produces no false negative.
‫So remember, false negative is where the model predicts zero in a binary classification, predicts
‫zero where it should have predicted one.
‫So a model with zero in this box will have a recall of one and a model with a precision of one will
‫have a zero in this box.
‫So if our model was perfect, these would both be zeros, and our precision and recall would both be
‫1.0.
‫And now f one is a combination of precision and recall.
‫So if our model was perfect and had precision recall, scores of 1.0 F one would also be 1.0.
‫Now support.
‫Support is the number of samples each metric was calculated on in our test data set.
‫There are 29 samples with a value of zero and 32 samples of the value of one.
‫So has heart disease.
‫And now accuracy.
‫We've seen that before.
‫And macro average this is short for average is the average precision recall and F1 one score between
‫classes so between these two.
‫So the macro average doesn't take class imbalance into account.
‫So if you have class imbalances, in our case we don't because we have relatively similar numbers for
‫values here.
‫So say for example, this was two and this was 59, then you would have class imbalances.
‫If you do have class imbalances, pay attention to the macro average.
‫But for the weighted average is the weighted average precision recall and F1 score.
‫So weighted is a key here between classes.
‫So weighted means each metric is calculated with respect to how many samples there are in each class.
‫So a.k.a. this metric will favor the majority class e.g. this will be a high value when one class outperforms
‫another due to having more samples because that class has more samples and it will be given a higher
‫weight.
‫Whew.
‫That was a bit of a mouthful, but you can look up this information in the classification report documentation.
‫So Classification report SRC line.
‫You can also look up each of those metrics that we just covered individually.
‫Right.
‫So if you wanted to find out more.
‫Actually, we want the user guide.
‫There we go.
‫This is going to tell you some information about the classification report.
‫If you want to search for more precision, recall f one and you could look these up macro average weighted
‫average in the cyclone documentation.
‫But remember, we just said before that these metrics here are only calculated if we look at our cross-validation
‫diagram are only calculated on one test split.
‫So what we want to do is recalculate these metrics, but with using cross-validation, which makes them
‫a little bit more robust.
‫So how can we do that?
‫Hmm.
‫What we might do is wait till the next video.
‫So spoiler alert in the next video, we're going to calculate these metrics here using cross-validation.