‫So we left off in the last video calculating our model's false positive rate, true positive rate,
‫and at what thresholds?
‫Using the rock curve function.
‫Now, we kind of got back the false positive rate, but it's in an array of numbers.
‫So looking at this again doesn't really make much sense, but plotting it and seeing the ROC curve,
‫the actual ROC curve will make a bit more sense.
‫What we're going to do is we're going to create a function for plotting risk curves.
‫Now, in your practice, when you need to make multiple plots, as we saw in the map plot lib section,
‫you might want to create a function that does plots for you if you want to make more of the same plot.
‫So that's what we're going to do here.
‫Just see an example of how we might create ROC Curve plotting function.
‫So first of all, we'll import map plot lib not pie plot.
‫We're going to go as pelt beautiful and we're going to go call it nice and simple plot rock curve.
‫It's going to take the false positive rate and the true positive rate.
‫Then we're going to come here.
‫We'll leave a nice docstring.
‫So it makes it makes a bit of communicative sense.
‫Plots are rock curve given the false positive.
‫Right.
‫PR and true positive rate TPR of model wonderful nice and simple go plot raw curve plot dot plot.
‫We're going to pass it the false positive rate and the true positive rate and we'll give it a color
‫school.
‫Got to remember that these are these libraries.
‫Take American spelling.
‫We go here orange and then label equals rosy for Roc curve.
‫And then we're going to go plot line with no predictive power.
‫This is going to be a baseline.
‫You'll see it when the plot comes out.
‫It'll make a bit more sense.
‫That way we can compare our model to some some other arbitrary model which just predicts nothing.
‫Zero one.
‫Wonderful.
‫And the color is going to be dark blue line style.
‫We'll keep it dotted.
‫Actually, this is only two, so it looks different to our ROC curve and the label is guessing.
‫So this is this line you'll see when we plot.
‫It is basically like our model was just guessing.
‫Then we're going to go customize a plot.
‫The pale blue dot x label.
‫This is going to be the false positive right f PR and then we're going to get a potato y label.
‫You might be able to guess what this is.
‫This is the true positive rate.
‫This is because we got PR is the X label and TPR as the the Y data true positive rate.
‫And then we're going to go receiver operating characteristic.
‫Ah.
‫I see.
‫Now, this is the type of plot you might give someone when they're asking for the rock curve.
‫Right?
‫So this is why I'm showing you this show.
‫And you'll probably also come along, this sort of plot when you have a look at different classification
‫problems online.
‫They might use the rock curve as an example of how the model is doing.
‫We want April 1st TPR.
‫Beautiful.
‫So this should work.
‫Wonderful.
‫What is going on here?
‫We finally get to see what a rock curve looks like.
‫If we go to here, if the false positive rate is 0.6, the true positive rate is going to be 1.0.
‫So the maximum score it can get is 1.0 up here.
‫And so this model here going from corner to corner is guessing.
‫Can you guess where the most ideal rock curve might end up?
‫If this is guessing and our model is doing far better than guessing by getting about 80%, 85%, 82%,
‫something like that.
‫Can you guess where the most ideal curve will go?
‫That's all right.
‫If you come, we're going to have a look at it.
‫So before we do, actually, is let's have a look at the AUC score.
‫We can do this from Skyline Import ROC, AUC score.
‫And so this is where the AUC comes into play from ROC.
‫So when you hear Arianda curve or receiver operating characteristic, what is AUC score?
‫And so we can do that, let's just see it in action.
‫This takes the test labels as well as the probabilities.
‫Well.
‫So we can do y probs.
‫What we need a.
‫Positives.
‫There we go.
‫So how did I know this again?
‫Docstring y true y score.
‫Wonderful.
‫And so what is this metric you might be asking?
‫Well, this is remember, AUC stands for area under curve.
‫And so if we were to calculate, take our curve, ignore this one for a second.
‫Let's just get rid of that.
‫So if we were to take our curve.
‫This year.
‫It's a bit of a jagged curve, but it is a curve.
‫And if we were to measure the area, so shade in everything under here, shade in all of this and figure
‫it out.
‫Then we've got one across here.
‫We're missing some space up here.
‫So our area is definitely not going to be one.
‫Our score is going to get 0.84, so 0.849.
‫Now the maximum score you can get, can you guess with an AUC score?
‫So area under the curve, the maximum score you can get is 1.0.
‫And so that means that the curve goes straight up here to this corner and straight across here to that
‫corner.
‫Now let's see it in action.
‫So if we're going to go plot a perfect rock curve and a U.S. score, so we're going to go f pr tpr thresholds
‫equals rock curve.
‫Go.
‫Why test?
‫Now we're going to plot what you're going to use our function here, rock curve.
‫See where it comes in handy because we want to plot multiple rock curves.
‫TPR fpr TPR let's do that.
‫So this is a perfect rock curve here.
‫The area under the curve here.
‫Can you guess what it would be?
‫We just discussed it.
‫So perfect.
‫AUC score is ROC.
‫AUC score.
‫Why test?
‫Now that's 1.0.
‫And so in reality, a perfect rock curve is very unlikely.
‫That means you've got a perfect model.
‫It's got no false positives.
‫Everything's a true positive.
‫If we saw a 1.0, here's a rock IAC score.
‫Is that going to sort of maybe raise a flag to see if your model is predicting the right thing?
‫The main details here that a rock curve is predicting is a true positive rate versus a false positive
‫rate.
‫And the main metric you can use to to boil it down, rather than just being a curve, you can use the
‫AUC score.
‫So that's what you do to calculate a rock curve.
‫If someone asks you to calculate a rock curve for your model and it's going to compare the false positive
‫rate versus the true positive rate.
‫So now we've seen rock curves, an area under the curve.
‫Let's have a look at the next classification metric or the next way to evaluate a classification model,
‫and that is with a confusion matrix.