‫Welcome back.
‫It's in the last few videos, we've been spending a fair bit of effort getting our our data into an
‫accessible format.
‫Now we've got that.
‫So we've got let's just quickly have a look.
‫We've got bullying labels, which is our labels in numerical format.
‫We'll have a look at the first two so it doesn't take up too much room.
‫There we go.
‫That's true and false.
‫We can access those as zeros and ones really easily.
‫And the same goes for our file paths.
‫So these are our training images here.
‫And we left off discussing how Kaggle doesn't provide us a validation set.
‫So creating our own validation set.
‫And so since the data set from Kaggle doesn't come.
‫With a validation set, we're going to create our own.
‫And remember, from way back in Machine Learning 101, the most important concept in machine learning
‫is the three sets.
‫Okay, the training set, the validation set and the test set.
‫So right now we've got the training set and a test set, but we'd like this validation set which is
‫a.k.a the practice exam.
‫So to reiterate, we'll train a machine learning model on the training set, we'll evaluate it on a
‫validation set to see if our experiments that we've been doing on the training set are going alright.
‫And then finally the final exam is our test set.
‫So we'd like this little intermediate set to validate our initial experiment.
‫So that's why we're going to make a validation set.
‫And now to do so, what can we use?
‫What have we used in the past?
‫I want you to think about this.
‫It was in the psychic loan section.
‫If you said train test split, you'd be correct.
‫Now it can be a bit confusing because it says train test split, but really the functionality of the
‫function is just splitting data into two different sets, one set of certain size and another set of
‫another size.
‫Let's set that up.
‫But first, what we might do is set up X and Y variables.
‫Now, the reason we do this is because.
‫Having to remember bullying labels and file names is a bit tedious.
‫So what we might just do is create X, which is our standard variable for your data, and Y, which
‫is usually the standard variable for your label.
‫So we're just going to make copies of those.
‫Beautiful.
‫And now since we're working with 10,000 plus images, 10,222, to be exact.
‫It's a good idea to start working with a portion of them first to make sure things are working before
‫we go and train a machine learning model on all of them.
‫This is a thing to note for your own projects is that when you're doing experiments, remember what's
‫our goal?
‫It's to minimize the time between experiments so we can figure out what works and what doesn't.
‫And if we're using 10,000 examples every single time, that's going to take a while.
‫And if you've never trained a machine learning model or a deep learning model on 10,000 plus images,
‫so you don't have a perspective of how long that takes, you can take my word for it that it would take
‫a long time.
‫So we'll probably reduce it down and start with about 1000 images to begin with and then increase it
‫as we need.
‫So let's write a little note for ourselves.
‫We're going to start off experimenting 1000 images and increase as needed.
‫Now, this goes for not just images, but for all kind of machine learning projects.
‫And we've seen this in the past using some samples of our data.
‫If you are working on a text classification and you had 100,000 emails and you wanted to classify whether
‫they were spam or not spam, you might only start with 5000 and then build up from there.
‫So let's set a little parameter.
‫So set number of images to use for experimenting.
‫Now I want to show you something really cool within CoLab, so we'll create a little parameter here
‫and now it's in capitals because convention you might see here and there with with deep learning projects
‫is that hyper parameters.
‫So things that you can set as a user are often set in capitals.
‫That's where you might see these variables that are in full, full capitals.
‫So let's go 1000 and then in CoLab it has this little.
‫Little cool to that.
‫You can.
‫It's like a magic function at prem.
‫And now I'm still working these out, so maybe you can leave a little suggestion on where I can find
‫out how to make more of these if you figure it out, because I only know a couple.
‫Now, watch this.
‫So what we've done is we've just gone, hey, this is a parameter that we can set.
‫Create a type slider.
‫The minimum is 1000, max is 10,000, and the step is 1000.
‫Now images has now just appeared.
‫Look, if we go like this.
‫Oh, yeah.
‫5000.
‫8000.
‫Well, I could play around with that all day.
‫That's something that CoLab has over Jupiter, aside from being backed by a GPU periphery.
‫All right, so now we've got a little parameter.
‫We've got X and Y.
‫Let's use psychic learns, train test split.
‫So let's split our data into train and validation from model selection input train test split.
‫Now again, this is called train test split, but the basic functionality of this function is to split.
‫You could do this manually by using indexing, but we'll just use a function to help us out.
‫Is just a split data into two different sets into training and validation of total size.
‫Now images.
‫Which is this variable we've set up here.
‫Run that cell x, train x val this time, not x test y, train y val equals train test split x up to
‫num images.
‫So it's only going to take the first thousand file names because remember X is just our file names.
‫We haven't imported our images just yet because it's a lot faster to work with just file names.
‫And why is our labels in boolean form?
‫And test size.
‫Again, this is just validation.
‫Size will be 20% and then the random state is kind of like setting random seed.
‫Well, it is like setting random seed to 42.
‫So random state equals 42 is the same as going up random seed 42.
‫And if you're not sure what random seed does, there's a video on that.
‫Otherwise you might want to just quickly Google.
‫What does random seed do?
‫So if we go.
‫Len X train just to check all the the lengths of our data are correct because you wouldn't believe the
‫amount of times I've been burned because my data was the wrong shape.
‫You will spend hours battling to get your data into the right shape.
‫Trust me.
‫I've been there.
‫All right, wonderful.
‫So we've got 1000 samples and we've got a 20% test size.
‫So that means there's going to be 800 samples in the training split and 200 samples in the validation
‫split.
‫So let's just have a quick look.
‫Let's have a gaze at the training data.
‫So X train just to verify that we're still on this earth.
‫We're still working with the right stuff.
‫Actually, that's a bit too many.
‫I've got to remember that these are full blown bullying labels, so there's a lot there.
‫So train, we've got file paths, correct?
‫That is what we want.
‫And training for labels we have.
‫Bullying labels.
‫That is what we are after.
‫All right.
‫So now that we have our data in to a training and a validation set, a small subset, mind you, because
‫remember, our goal to begin with is to minimize time between experiments.
‫What we can probably finally do is turn out images or at least create some functionality to turn our
‫images and labels into tenses.
‫Let's do that in the next video.