‫How exciting is this?
‫Oh, we've gone a long way from battling to get our images into tenses, our labels into tenses, preparing
‫our data, choosing a model on TensorFlow hub.
‫Creating model callbacks.
‫We're finally at the stage where we're ready to train our first deep learning model.
‫So let's create a heading training, a model.
‫Now we're going to put on a subset of data, because if you remember back when we split our data up,
‫we're only going to start training on 1000 images because why do we want to do that?
‫So let's right here.
‫Our first model is only going to train on 1000 images to make sure everything is working.
‫So that's what we're trying to do.
‫We're trying to minimize our time between experiments.
‫So we begin by starting on a subset of the data to make sure all of this code, I mean, it might break.
‫Right?
‫So we want to make sure that it's working before we spend a long time training on 10,000 images because
‫training on a thousand images is going to go a lot faster than 10,000 images.
‫And you could imagine the same goes if you increase it to 100,000 images or in the case of image, net
‫14.2 million.
‫So there's one more variable that we have to define.
‫Before we can get into training a model and that is number of epochs.
‫And so the number of epochs is how many passes of the data we'd like our model to do.
‫And a pass is, you can imagine, is equivalent to our model trying to find patterns in each dog image
‫and see which patterns relate to each label.
‫Let's go on to here num epochs.
‫Equals 100.
‫And we're going to do something cool here.
‫We're going to create a little slider like we have before Harambe.
‫And then we're going to create type slider and then the min will be ten, the max can be 100 and then
‫the step can be ten.
‫Look at that.
‫How cool is that?
‫So if we wanted to do ten epochs, we could do ten.
‫But we're going to stick it to 100 because we've got our early stopping callback and we're going to
‫see all of this in action in a second.
‫But essentially what a training pass is going to do.
‫If we go back to our keynote, one epoch would be giving our model a single chance to look at all of
‫the training data and then validating it from there.
‫100 epochs.
‫So we're giving our model up to 100 chances to go through the training data set and figure out the patterns.
‫So it's going to what a pass is.
‫It's going to go through these layers.
‫We're going to pass our training images to our model.
‫It's going to go through these layers.
‫And then our model, as it learns patterns, is going to make guesses on which label belongs to which
‫image.
‫And the worse off the guess is, the higher the loss will be.
‫So that's where Adam comes in.
‫Remember that guy down the bottom of the hill at the International Hill Descent Championships is telling
‫our model how it can improve its guesses with each epoch.
‫And then we're going to see how our model is doing on an accuracy level, because we've got that spectator
‫judge at the bottom of the hill watching how well our model is performing.
‫So again, if all this doesn't make sense, it's going to make more sense once we actually start to
‫run a model.
‫So let's do one final check.
‫The final check is one last time to make sure we're using a GPU because if we're not using a GPU, this
‫training our model on images is going to take a very, very long time.
‫You saw right at the beginning a GPU can speed up our code by up to 30 times and sometimes more.
‫Let's write that code.
‫Check to make sure we're still running on a JPG view.
‫So print.
‫GPU.
‫Available.
‫And then we're going to go.
‫Yes.
‫If TPH config list physical devices.
‫And then we want to check cpu else.
‫Not available.
‫So this is just the same code we wrote above.
‫But we need a GPU to be able to run this in a considerable amount of time.
‫Yes.
‫We could also have checked here.
‫Remember, that's a little shortcut trick.
‫But I love seeing this print out here.
‫Now.
‫Boom.
‫We've got a GPU running, we've got a number of epochs set up.
‫Let's create a simple function which trains a model.
‫So let's write that here.
‫Let's create a function which trains a model.
‫And the function is going to create a model using create model.
‫And then we want to go set up a tense board callback.
‫Using our tensor board function.
‫So create C.
‫This is where creating functions has really come into play for us here.
‫Otherwise we'd have to be writing code ad hoc, but writing functions to begin with takes a little longer,
‫but in the long run saves us time.
‫So call.
‫The fit function on our model passing it the training data validation data.
‫A number of epochs to train for.
‫Which is not a pox.
‫We just define that with our sick little slider and the callbacks which are our helper functions we'd
‫like to use.
‫And then finally we want it to return the model.
‫We're going to change this into markdown.
‫That is looking phenomenal.
‫Okay, let's go here.
‫Build a function to train and return a trained model.
‫So here we are.
‫Def train.
‫I can't believe we're finally up to this.
‫This is amazing.
‫We're training our first deep learning and neural network.
‫So trains are given model and returns the trained version.
‫Nice and simple.
‫Remember, even though we're getting excited doesn't mean that we're losing our communicative habits.
‫So we'll create a model.
‫So we're going model equals.
‫Say, this is where our create model function comes in.
‫Oh yeah, that's so satisfying to write.
‫And then we go create new tense board session every time we train a model.
‫Because remember, with our tensor board, with our tensor board callback, we are actually we'll just
‫call this tensor board.
‫We don't need the callback there.
‫We set up our function.
‫So if we come back up here every time it's called, it's going to create a new folder in logs with the
‫current date time.
‫So this is important because every time we run our train model function, so every time we try to train
‫a new model, a new experiment, we're creating a logs folder which we can then track our model's performance.
‫So that's a real helpful thing later on when we're trying to evaluate which experiment did better than
‫another, and then we go fit the model to the data passing it, the callbacks we created.
‫Now again, that's probably a better way to do this function, but we're just trying to make it so that
‫it works and then we can go back after we've sort of got through the phase of fitting a model for the
‫first time and refine it.
‫So let's go model fit and we're going to pass it.
‫X equals the train data, which is a data batch, if you remember, which contains the images and labels.
‫So epochs is going to be num epochs a.k.a.
‫How many times is our model allowed to look at the training data before it stops?
‫Or how many chances did our model have to pass over the entire training data set to find patterns and
‫then the validation data?
‫Is going to be VAL data, which is a data bunch as well or data batch.
‫Sorry.
‫And the validation frequency is how often we want to test the patterns that our model has found on our
‫validation set.
‫And we set it to one because we want it to test its patterns that it's found in the training set on
‫the validation data, every epoch.
‫So once an epoch and then our callbacks are ten to board, which we just created above with our function
‫and early stopping.
‫Woohoo.
‫And then we want to return model and put a little.
‫Return the fitted model.
‫How cool is that?
‫And now we can fit our model to the data by as simple as going.
‫Model equals train model.
‫You see the benefits in creating functions so we could run this function after doing a bunch of different
‫experiments later on.
‫And so are you ready?
‫Now, a little tidbit here.
‫When training a model for the first time, especially on image data or any other kind of large scale
‫data, the first epoch will usually take the longest compared to the rest.
‫Now, this is because the functions that we've written is getting the data and it's been initialized.
‫So aka loading the data into the memory of our GPU.
‫Now using more data will generally take longer, which is why we've started with 1000 images.
‫And so after the first epoch.
‫Right.
‫We've set it up so that it's capped at 100.
‫The first one might take a couple of minutes.
‫After the first epoch, a subsequent epoch should only take a couple of seconds.
‫So you're ready.
‫We're going to do this together.
‫Fingers crossed.
‫All of our functions, because this actually depends on a fair few lines of code up here.
‫Let's say if it works, 3 to 1.
‫Oh, look at that.
‫Building a model with mobile net v2.
‫I got my fingers crossed.
‫I hope you do, too.
‫Oh, yes.
‫It's starting up.
‫See?
‫There we go.
‫Epoch one of 100.
‫Because epochs is set to 100.
‫So it's going to train for 25 steps, which means our batch size.
‫So if we go, we have 800 images in our training set because remember we started with 1000.
‫So if we divide that by 32, we get 25.
‫So that's why training is 25 steps because there's batch size 32 and validation.
‫If we go 200 divided by 32, it's going to round it up.
‫So it's rounded it up to seven.
‫That's why we have validation.
‫So 200 images over a bachelor's is 32.
‫And check this out.
‫There we go.
‫At about 5 minutes.
‫And so here's what I'm talking about with the loss.
‫So now all these things that we created right up here.
‫So the loss.
‫So our goal is remember to minimize the loss.
‫Because we're at the top of the hill.
‫We're afraid of heights.
‫We want to get to the bottom of the hill.
‫You want to minimize the loss function?
‫The loss is the height of the hill.
‫We come down here.
‫So hopefully this number goes down.
‫But we want to maximize accuracy.
‫So that's how we'll know if our model is training or not.
‫If it's learning patterns, this loss should go down and this accuracy should go up.
‫So this is going to take a few minutes to get through the first epoch.
‫I'm going to wait until it's gone through the first epoch.
‫And then so I'll speed this video up and then I'll see you once it's past the first epoch because you're
‫going to see subsequent epochs after this.
‫So Epoch two out of 100 is going to be pretty quick because we're using a GPU.
‫The only reason the first one takes a while is because it has to load all of those images into the GPU
‫memory.
‫I'll see you pretty instantaneously, but for me it'll be about 3 minutes.
‫Alrighty, I'm back.
‫So we've got an ETA here, which stands for Estimated Time of Arrival.
‫It's about 12 seconds left in this first epoch, as long as everything goes to plan.
‫And so you can see the loss has reduced slightly and the accuracy has increased.
‫So this is a good thing.
‫Now, what we should see at the end of this first epoch, if we settle up correctly, we should see
‫this is loss on the training set and accuracy on the training set.
‫We should see the loss on the validation data and the accuracy on the validation set pop up any second
‫now.
‫So we're just waiting.
‫So probably what it's doing now is it's been a little bit longer than 12 seconds, but it's loading
‫the validation data into memory so it can calculate and evaluate how it's performing on that data set.
‫I can't believe we're doing this.
‫We're, like, training a model together in real time.
‫This is phenomenal.
‫I've never done this before.
‫That is as long as your code is working.
‫Here we go.
‫Look at that.
‫So now we have this.
‫What I was talking about, we have this metric is on the training data and this metric is on the validation.
‫So see how quickly our accuracy is improving.
‫Now, I told you epochs are going to speed up.
‫Look at that.
‫The first one took about 5 minutes, but these are taking 5 seconds.
‫We're already at 1.0 accuracy.
‫That's 100% accuracy on the training data.
‫That is insane.
‫So what this is telling us actually is it's a good thing our model is overfitting because it's performing
‫way better on the training data than it is on the validation data.
‫But this is great.
‫Our model is working, it's finding patterns.
‫It's using what mobile net Vertu has learned in image net and it's applying it to our dogs data set.
‫Dog vision is coming to life.
‫I'm so stoked.
‫Now I hope yours is running like this as well because we're training a model together in real time.
‫This is beautiful.
‫So what we should see is once the validation, accuracy stops improving for a number of epochs.
‫So three, because of our early stopping callback, it's going to stop training.
‫So we actually I don't think we'll reach the 100 epochs, maybe not even close because see here we're
‫getting 1.0 accuracy with a tiny loss on our training data.
‫So here we go.
‫It stopped.
‫For.
‫My heart is racing.
‫That was phenomenal.
‫So see what I meant.
‫The first epoch takes a while because it's loading data into memory.
‫Once it's loaded.
‫That initial data in the memory, the GPU is just going to go.
‫All right.
‫Time to step on the gas and we'll turn on the afterburners.
‫All gas.
‫No brakes, baby.
‫So we reached 18 out of 100 epochs and we can see that our model on our training data set is performing
‫at 100% accuracy.
‫That's pretty crazy.
‫1.0.
‫You've got two times this number by 100.
‫The loss is fairly low.
‫I mean a loss of zero is is perfect.
‫But we can tell that our model is overfitting because it's performing way better on the training data
‫than it is on the validation data.
‫So if we come back, it's like our model has memorized the course materials rather than the problem
‫solving principles of those course materials.
‫And it's struggling to adapt to a data set it hasn't seen before, a.k.a a practice exam or our validation
‫set.
‫So right now our model has a poor ability to generalize.
‫Who?
‫So that was I think that's enough for one video.
‫That's a lot of excitement.
‫What I want you to do a little bit of homework is before we go and check the Tennessee board logs in
‫the next video because we've used our censor board callback before.
‫I want you to check out this a question.
‫Is this a really important one after we've tried to model?
‫I want you to go.
‫Question.
‫It looks like our model is overfitting because it's performing.
‫Far better on the training data set than the validation data set.
‫What are some ways to prevent model overfitting in deep learning neural networks?
‫So we'll turn that into markdown.
‫That's a question I want you to look up before we get into the next video.
‫What are some ways to prevent a model overfitting in deep neural networks?
‫So check that out.
‫Even if you're not sure of them, I just want you to start getting curious about what's going on, and
‫I want you to start picking up on the clues as to when your model is overfitting, such as performing
‫far better on the training data set than the validation data set.
‫And a little note here, note overfitting to begin with is a good thing.
‫It means our model is learning.
‫So that I'm phenomenally excited that we have just trained our first deep learning neural network together
‫using transfer learning.
‫Dog vision is coming to life.
‫So we finished training a model, at least a first subset of a model.
‫This is not on the full data, this is only on 1000 images.
‫So our next step is to if we come in to our keynote, what are we up to?
‫So we fit the model to the data.
‫We haven't made a prediction yet.
‫So what we might do is evaluate the model using our tensor board callback so you can see that and then
‫we'll figure out how to make a prediction with our trained model.
‫Sound like a plan?
‫So check out this question and I'll see you in the next video.