‫Alrighty.
‫So now we've got a function to pre process our images aka turn them into tenses.
‫We can now start to figure out what we need to do next.
‫So we've got get data ready, turn it into tenses.
‫Pick a model.
‫Hmm.
‫Well, we're not quite up to this.
‫So if we come back, we're going to read a little bit more into learning images in TensorFlow.
‫We'd figure out that TensorFlow likes to see things in the form of a data batch, a.k.a a tensor containing
‫an image example or a piece of data example.
‫Remember, this is data agnostic.
‫We're just using images as our practical example.
‫So it likes it in the form of give me a piece of data and it's label as a combined tensor so I can process
‫them really fast.
‫And if you're wondering what a batch is, well, it's also called a mini batch, which is a small portion
‫of your data.
‫Now, generally, if you're wondering what a good batch size is or how much is a small portion, 32
‫is generally the default batch size with most deep learning models.
‫And I'll show you this is how I figure out a lot of things.
‫So we go Yan Liqun batch size and you might be wondering who the hell is Yan Liqun?
‫What is batch size?
‫All these things are new to me.
‫Well, I want to show you a little trick when you find or when you hear of someone who is, let's say,
‫a figure in the field of machine learning or deep learning.
‫AI trick that I use to figure out what they think of a certain topic is I Google their name and I Google
‫the concept I'm looking for.
‫Another one is Jeremy Howard.
‫So John Larkin and Jeremy Howard, you could go Jeremy Howard batch size Google that I'm just telling
‫you these two people off the top of my head, Josh McCown I'll let you research him, but he is one
‫of the founders of convolutional neural networks, which is kind of what we're going to use to process
‫our images.
‫But getting sidetracked here if we Google Yan Liqun batch size.
‫And then we come to to Twitter to what is actually an amazing resource to figure things out.
‫Look, this training with large mini batches is bad for your health.
‫More importantly, it's bad for your test error.
‫Friends don't let friends use mini batches larger than 32.
‫Then we've got a paper here linked on different batch sizes.
‫So I'll let you look into that.
‫I'll link this tweet in the resources section so you can check out that paper.
‫But this is what we're going to work towards.
‫We're going to transform our data into batches of size 32.
‫So they process really efficiently in TensorFlow.
‫So let's make a little heading, turning out data into batches.
‫And these little tidbits, the one I just showed you, like searching someone's name, these kind of
‫things I think are important, right?
‫How would you know that if I didn't sort of tell you or you sort of didn't stumble across it?
‫The only way I found that was by stumbling across, Hey, there's all this great information that people
‫are sharing.
‫How do I find that every so often and again, my way is not perfect, but it's helpful.
‫I Google someone's name and a certain concept I want to find out about more.
‫This is why it's important to share your work if you find something valuable.
‫Okay.
‫Getting sidetracked once again.
‫But why would we turn now something into batches?
‫Well, let's say.
‫Why turn our data into batches?
‫And if you're still confused on batches, we're just taking a small portion, 32 images and calculating
‫on those 32 images at one time.
‫And then the next loop, the next training session we calculate on the next 32.
‫Because if we tried to calculate patterns in all 10,000 images.
‫So here, let's say you're trying to process 10,000 plus images in one go.
‫They all might not fit into memory.
‫So what I mean by that is that a CPU has, although it's really fast to computing, it still has a limited
‫amount of memory.
‫And so my computer has 16 gigabytes of RAM the GPU backing, this might have eight gigabytes of RAM.
‫So if we have 10,000 plus images and we're trying to process the patterns on that in one go, that may
‫be more than eight gigabytes, which is going to slow down our training process.
‫It's going to break everything.
‫So that's why we do 32 at a time.
‫So that's why we do about 32 images at a time.
‫You can 32 images, batch size.
‫That's right here.
‫This is batch size.
‫You can manually adjust the batch size if need be.
‫All right, so we spent enough time talking.
‫Let's start writing some code.
‫What do we need?
‫So write it here.
‫In order to use TensorFlow effectively, we need our data in the form of tensor tuples, which look
‫like this.
‫So image in the form of a tensor and label.
‫So as pairs.
‫So then we'd pass 32 of these at a time to our machine learning model, and it would figure out the
‫patterns in this image that relate to this label, and then it would end up doing that in a sort of
‫loop fashion until it gets through all 10,000 plus images.
‫All right.
‫Let's build some functionality.
‫But first we need a way to get tuples.
‫So we need before we can turn them into batches of 32, we need to get a tuple that looks like this.
‫So let's write a little function.
‫So create a simple function to return a tuple, a tuple of tenses that rolls off the tongue nicely.
‫So we want to go def get image label.
‫Because we've got a function here that's going to return our image.
‫So all we want to do is just basically get the label associated to whatever path we pass this function.
‫So we go here image path.
‫And now really you're going to laugh at me for how simple this function is, because all we're doing
‫is passing at the image path.
‫The actual label and the image preprocessing function we just wrote.
‫So process image, image path and then we want to go return image label.
‫Look at that.
‫How basic is that?
‫So takes an image file path name and the so she added label processes the image and returns a table
‫of image label.
‫The docstring is longer than the function.
‫Beautiful.
‫We didn't even need that line.
‫We could have just done it all.
‫Return here.
‫Now, let's just give a quick example there.
‫Let's just see exactly what this would look like.
‫Process, image.
‫We want file names.
‫Let's get 40 to our favorite number.
‫And what is it?
‫What's it called?
‫Bull Lane.
‫Labels.
‫We could just use X and Y, couldn't we?
‫That's probably a bit better.
‫X.
‫Why?
‫42.
‫Demo of the above.
‫Beautiful.
‫So that's in a tuple.
‫Our image is in a tensor because it we pass it to process image, but our label is not yet in a tensor.
‫But that's all right.
‫We could turn it into a tensor by just going TF Constant.
‫There we go.
‫So now we've got a couple of an image and label pair.
‫In the form of tenses.
‫Now we're making some good progress here in getting our data ready.
‫Now, remember how I said a lot of machine learning is just preprocessing your data into a format that
‫fits into a machine learning model?
‫That's what we're doing here.
‫And now, again, this process was sort of taking getting our data into the format of data label in
‫a couple of tenses is agnostic to whatever data you're using.
‫So if you're not using images, you're using something like text, this would be a very similar process,
‫except the way you pre process.
‫The text will be different to the image.
‫You still have to get it into tenses.
‫All right.
‫So we've got a little function to get image and label pairs in the form of tuples.
‫We'll write a function in the next video to turn our image and label pairs into batches.
‫And there's a few little tidbits that we need to pay attention to depending on if our images are in
‫a training batch, a validation batch and a test batch.
‫But we'll see that in the next video.