‫So we've seen how we can evaluate our model's performance or at least initial performance on something
‫like epoch accuracy and epoch loss and see how we can track different experiments.
‫But this is just on the models training and validation sets, and it's only really on what the model
‫has learned in the data set.
‫What are other really good way to evaluate what our model has learned is to use it for what we're building
‫it for.
‫If we're building dog vision.
‫And we always got to keep in mind what the end goal is.
‫With whatever project we're working on, we're building dog vision to see if we can build a machine
‫learning model that can identify a dog breed in a photo.
‫So it's going to make some predictions giving a photo.
‫So let's do that.
‫If we come back to have a look at our workflow, well, we fit the model to the data.
‫Now it's time to make a prediction.
‫So we're in we're still in step three.
‫We've done a little bit.
‫We've touched on step four.
‫And remember, these aren't really linear steps.
‫So these are just I've just put them here because it's easy to understand it like this.
‫But remember, they aren't necessarily linear, so you can do them out of order.
‫We come back, we'll make a little heading, making and evaluating predictions using a trained model.
‫Commander, Ma'am, for turning it into markdown shift and enter.
‫Okay.
‫So making predictions with a trained model is very similar to how we did it in psychic loan, so we
‫can call the predict function and posit data in the same format the model was trained on.
‫That's the important thing.
‫Let's see the code first and then we'll discuss what's happening.
‫So make predictions on the validation data.
‫So remember we created a validation data batch.
‫Now, this was not used to train on.
‫That's the important point.
‫If we come back to our three sets, how we evaluate a model is because it trains on the training set.
‫We want to check it out.
‫It's initial results on the validation set and then both of these exams.
‫So the practice exam and the final exam are data sets that the model hasn't seen before.
‫So we come back.
‫Let's go.
‫Model.
‫Oh actually we'll save it to predictions predictions eco model predict val data because let's just remind
‫ourselves of what val data is.
‫Validator.
‫Batch data set.
‫There we go.
‫We've got some images and some labels so we know what the true labels are of the validation.
‫So when we pass this batch data set to model or predict because of the nature of the predict function,
‫it's only going to look at the images in validator and then make predictions based on those images.
‫So then what we can do is compare those predictions which are going to be in the form of labels.
‫Well, not exactly.
‫We're going to see what they look like in a second.
‫We can compare the model's predictions that it makes on the validation images to the actual labels from
‫the validation images.
‫Now, that was a lot of talking.
‫It's much better to see this in code.
‫So verbose is just going to say, hey, when you're making predictions, show me your progress.
‫Well, that was really quick because we've got 200 images.
‫So that's the beauty of working on a GPU.
‫Predictions are really fast as well, and so that only took about 187 milliseconds, so not even a whole
‫second.
‫And if we didn't set verbose to equal one, we wouldn't get this little progress bar outputting.
‫You might be looking at this and going, what is going on here?
‫Lots of just different numbers.
‫So let's check out the shape here.
‫Prediction shape.
‫For remember?
‫Where does this line up?
‫So if we go, Len, why?
‫Well.
‫200.
‫So there's the first shape.
‫So 200 images.
‫Where do you think the second shape is coming from?
‫120.
‫Where have we seen that before?
‫Len Unique breeds.
‫120.
‫So this means that we've got an array of 200 by 120.
‫So we have 200.
‫Let's see the first element.
‫So we have 200.
‫We'll go.
‫Predictions zero.
‫200 arrays.
‫Of 120 different numbers that are really small.
‫And we're wondering what is going on here?
‫Hmm.
‫Well, let's find out the length of this.
‫120.
‫Wonderful.
‫So what this actually is.
‫Is an associated probability.
‫For likeliness.
‫So basically what our model thinks a certain image is.
‫So there's a probability value here for every single label.
‫So the value here, the highest value in this predictions array.
‫So the zeroth predictions array.
‫This is for one image, the first image in the validation data.
‫So the highest value in here is going to correspond to the index of the label that the model thinks
‫is most likely.
‫And again, this is a lot of talking.
‫It's going to make a bit more sense once we start to put it together with code.
‫But just bear with me for a second.
‫If we sum up all of these, they're going to equal very close to one or maybe one.
‫Exactly.
‫And I say very close because the way computers store numbers is not exactly.
‫So if you keep going decimal points.
‫Right.
‫It's going to get very close to one, but it might not be exactly one.
‫And you might be saying, Daniel, what are you telling me?
‫This is the way the computer store numbers is not exact.
‫Well, I don't even fully understand it, but you just have to know that when a computer stores a number
‫that has lots of decimals because of the way it's most efficient to actually store it on a computer
‫chip, it's not going to be perfectly exact.
‫So that's why I say when you sum up a single prediction array, it's going to be very close to one.
‫And now I'm going to tie this back up into where we've come back into our model.
‫This is why we've used Soft Macs activation.
‫You might be saying.
‫Daniel Farrell, this is a videos ago you've already told us about soft Macs.
‫It didn't really make sense and it's really not making sense now.
‫But if we go back to soft Macs, we just go, Yeah, how about we go, What is Soft Macs if we come
‫back to our friendly Wikipedia page?
‫It's going to tell us.
‫That.
‫But after applying soft Macs, each component will be in the interval between 0 to 1 and the components
‫will add up to one.
‫So that is what we're getting here.
‫We're getting our components because we've used a soft max activation in the last layer of our network.
‫It's outputting an array, a prediction array that is 120 in length, and all of these values in prediction
‫zero and each other prediction here.
‫So predictions one will be very close to one as well.
‫So predictions one add up to one.
‫Or very close to 100 predictions.
‫There we go.
‫See, this is just over one, so very close to one.
‫The more decimals there are, the less exact a number is with the computer.
‫And so let's put this all together.
‫Because right now it's just been a lot of talking.
‫I want to show you a concrete example.
‫Let's take predictions, zero.
‫So we're going to remove these cells.
‫And just one little tidbit as well.
‫Remember our mobile net to architecture, all that that our soft max layer is doing is taking these
‫1280 output this array of numbers and changing it into an array of numbers like this of length 120 using
‫a soft max activation.
‫Because remember, for, for multiclass classification problems, we want a soft max activation.
‫But if we were doing only single classification, we'd use sigmoid.
‫So we come back here.
‫Let's check it out.
‫So the first prediction, I'm going to type out a few little indexing tricks here, and I just want
‫you to just watch on I'll talk through it a little bit as it goes, but I want you to start thinking
‫about how these things all tie together.
‫So let's do it.
‫Because what our goal now is, our model has made some predictions here, but our goal is to understand
‫these predictions further.
‫Our computer understands these, right?
‫Because they're all numeric.
‫So remember at the start how we spent a lot of time turning our data into numbers, kind of when you
‫get to the output of a machine learning model, right?
‫So this is what we're focused on.
‫We spent a lot of time preparing our inputs.
‫Now we're going to spend a lot of time preparing our outputs, which is what we're doing now.
‫So let's go.
‫Print prediction zero.
‫And then what we're going to do is go find the max value probability.
‫So remember the function in psychic loan you might not call predict proba which outputs a prediction
‫probability.
‫This is what predict does by default in tensorflow.
‫We go here prediction and p max predictions zero.
‫So we're finding.
‫Let's remind ourselves.
‫Prediction zero.
‫What?
‫It looks like we're finding the max value in here.
‫That's all that line does.
‫And then what are we going to do next?
‫So print we want to.
‫What else do we want?
‫We want to find the some.
‫So NTP or actually we might set this up so we can do multiple.
‫So index equals zero.
‫Just so we can see it with multiple examples.
‫So the sum sum is going to be np some predictions.
‫Index nice and simple.
‫Then we're going to go.
‫Wow, that was a terrible typo, Daniel.
‫Come on, we're going to go to the max index.
‫So find the index in this array, which has the max value.
‫So which is equal to max value.
‫NP.
‫Org max predictions.
‫Index.
‫And of course, what we're doing now is a little bit tedious.
‫We're going to function as this later on.
‫So you could imagine if we just kept writing cells like this, this is where we're functional, using
‫all of our code as much as we can so that we don't just have to continually write these type of print
‫statements to explore our data or explore our predictions.
‫Options Index.
‫Wonderful.
‫Predicted label.
‫Now, before we run this cell, we'll go against our cardinal rule of, if in doubt, run the code.
‫I want you to just have a think about what this one is doing.
‫Let me remind you of what unique breeds is.
‫So these are all our dog breeds.
‫So if we have predictions and we find the mass index of it and then we find where that index occurs
‫in unique breeds, what do you think that will return?
‫Hint.
‫We've typed it out here.
‫All right, let's delete this bad boy, and then we'll shift and enter.
‫Wonderful.
‫So here's index zero.
‫This is that array.
‫Just the exact same here.
‫The output of our model when we call predict.
‫So the max value, the probability of prediction, the maximum this can be is one.
‫Because remember the total of a soft max function is between zero and one.
‫So the highest a single value can be is one.
‫So this is saying we're predicting the highest label in here with a 21.6% prediction probability.
‫Now, the sum of all of these values is very close to one again.
‫Aligning with the soft max rule between zero and one and will add up to one.
‫But because of how computers calculate numbers, not exactly one and the maximum index.
‫So where this value occurs is 17.
‫So if we counted three, we find 17.
‫It's somewhere.
‫Maybe there.
‫That's it.
‫Now the predicted label is border area.
‫So for index zero.
‫Or for validation.
‫Image zero.
‫The predicted label is Border Terrier.
‫Now let's change this up.
‫What if we did?
‫Index.
‫What's our favorite number?
‫42.
‫So we got max value probability of prediction.
‫So this is a lot higher.
‫This is 75%.
‫So it's pretty confident with this one.
‫So you could say the higher this value is to one, the closer it is to one, the more confident a model
‫is about a certain prediction.
‫The sum is very close to one.
‫The maximum index is 113 and the predicted label is walkaround.
‫So if we go here, unique grades.
‫If we were to find index number 113.
‫We would find.
‫Walk around.
‫There we go.
‫Walk around.
‫113.
‫Walk around.
‫So this is how we convert our prediction probabilities in numeric form, something that our computer
‫understands to something that we understand a predicted label.
‫So now we've got a kind of way to turn our predictions into something that we can understand.
‫What will start to do next is we'll create a bit of a function that's going to allow us to find the
‫predicted labels for every sample in the validation set.
‫So about 200 or so images.
‫And then we're going to UN batch this data, set the validation data and compare the predictions in
‫a visual way to the truth labels.
‫Now again, a lot of talking, but in the next video, we'll start to build out that functionality and
‫we'll see what it looks like in reality.
‫And I'm so excited.
‫Right.
‫Because one of the funnest parts is comparing your model's predictions with what's actually going on.
‫Dog vision is coming to life.