Hello and welcome back to the course on deep learning in the press the Tauriel we found out what convolutional
neural networks are all about.
And today we're going to dive into Step 1 convolution.
So this is the convolution function and we try to stay away from mathematics and keep things intuitive.
But I couldn't help but share this formula for you because it is so simple a convolution is basically
a combined integration of the two functions and it shows you how one function modifies the other or
modifies the shape of the other and if you've done any signal processing or electrical engineering or
a profession where signal processing is required you would have inevitably come across a conclusion
function.
It is quite popular now.
Once again we're going to keep the mathematics lights or keep them separate.
And if you'd like to get into the math behind the convolutional neural networks a great additional read
is Introduction to convolutional neural networks by Jensine Wu who is a professor at Nanjing University
in China.
This paper was published literally days ago like five or six days ago and it is oriented specifically
at people who are starting out at beginners who are getting to know convolutional neural networks so
the mathematics there should be accessible actually e-mailed Professor Johnson.
And yeah he said his whole goal is to make or break the complex things down so that people who are new
to this field can understand.
And also he mentioned that he's got some materials available on his home page so if you yourself if
you just remove the last two parts and you just go to like Slash W.J. X to that part that's his home
page and you'll be able to find more additional tutorials and materials which haven't been published
as papers but but he uses them in his tutorials so you might find those useful so browse around there
if you'd like to get an introduction into the mathematics behind coalitional neural networks and kind
of build a solid base around that area.
But we're going to move on and we're going to talk about the convolution.
So what is a good solution in intuitive terms here on the left.
We've got an input image as we discussed that's how we're going to look at images just ones and zeros
to simplify things.
And you can see the smiley face there.
Then we've got a feature detector so feature detectors a three by three Matrix.
Does it have to be three by three.
No it doesn't.
Alex net I think uses seven by seven.
And then some other one of those other famous ones uses like five by five feature detectors.
They can be different but usually you'll see that they are three by three and they are you know reasons
to make them three by three so we're going to stick to the conventional way.
Having a three by three feature detector also the feature detectors called these are important terms
because you might come across them.
There are many different terms for the feature detector but the most common ones are feature detector
or Eik might hear it being called kernel or you might hear it being called Filter.
So in this course we're going to be using either filter or a feature detector interchangeably but just
bear in mind that it has those names and a coalition operation is signified by an X in a circle.
Just as you saw in the formulas before and here what happens is on an intuitive level or just think
of it in terms of what is actually happening in the background rather than the mathematics.
Well you take this feature detector or filter and you put it on your image like you see on the left.
So you cover the for instance in this case the top left corner the nine pixels in the top left corner
and you basically multiply each a valuable value so respect to value so the top 0 by the top left value
by the top left value then basically is in position of a 1 one by position about 1 1 position number
or 0 1 0 1 0 2 by 0 2 and so on.
So it's element wise multiplication on these matrices.
And then you add up the result.
So in this case nothing matches up so it's always either 0 by 0 0 or by 1.
So the result is zero.
And here you can see that one of them matched up one on the left matched up.
And therefore we've got a 1 here.
Nothing matched up nothing matched up nothing matched up.
Then we move on to the next throw so and step at which we're moving this whole filter is called the
stride.
So here we have a stride of one pixel.
Here you can see again something matched up the bottom right corner matched up against stride but a
bottom one in the middle matched up here top right hand match up the nothing measure.
The stride is one.
You can change the stride.
You can make it one two.
You're going to get three whatever you like.
The Eventually the one that works well is usually or two.
So that's what people stick to.
And we'll we'll talk about what the stride is towards the end of this tutorial.
So here we've got so we're matching absolutely when I hear you can see we've got two because two of
them matched up and so on and so on.
So on there we go there's another one that matched up there we go and we're done.
So what's what have we created.
Right.
Couple of important things here.
The image on the right is called a feature map also has several terms it also can be called sometimes
it can Vold feature.
So in your blog and volution operation operator to something it doesn't become convoluted it becomes
convolved and it has sometimes like I think to myself in the wrong way but it is the correct term is
convolved is a kind of old feature or it can also be called the activation map.
But we're going to be calling it a feature map in this course so it can be called any one of those things
and what have we done here.
Well as you can see we've reduced the size of the image.
That's number one and that's the important thing I wanted to mention about your input image and the
feature text and the stride.
If you have a stride of one you can see the image reduced a bit but if you have a right to the image
is going to produce more so the feature is going to be even smaller.
And that's a very important function of the feature detector of this whole convolution step is to make
the image smaller because that'll be it'll be easier to process it and it'll be just faster.
It will and you'll be just foster because imagine like here we've got a what a seven by seven image
but imagine if you have a proper photo right.
Or if you have like a 256th on 56 pixel image that's it's a huge number of pixels I CHONE if it is x
squared or like let's say you have a 300 but 300 pixels.
So we don't get confused with the R.G. B 256 has to say like we have a 300 by 300 image in terms of
size and pixels.
Then you have 300 square number of pixels that's a huge number and therefore feature detectors will
reduce the size of the image and therefore stride of two is actually beneficial.
But then the question is do we lose information.
Are we losing information when we're applying the feature detector.
Well some information we are losing of course because we have less values and of resulting matrix.
But at the same time the purpose of the feature detector is to detect certain features certain parts
of the image that are integral.
And so for instance if you think about it this way like the feature detector has a certain pattern on
it.
The highest number in your feature map is when that pattern matches up.
In fact the highest number you can get is in an all simplified example is when the feature is that it
matches exactly and you can see that number four we have in our feature map that's exactly.
So if you look at it here that's exactly where this feature detector because there's only four ones
and it matched perfectly so you can see this this part over here.
So the feature was detected here.
And as we discussed at the very start of this section that it features is how we see things is how we
recognize it.
We don't look at every single pixel so to speak in what we see on an image or in real life.
We don't look at every single picture we look at features we look at the nose the hats the the feather
the eyes under the little black marks under the cheetah's eyes to distinguish between a cheetah and
a leopard or the shape of the train.
We don't to distinguish between a bullet train and normal train and so on so we don't look at everything
we look at features and that's what we're preserving and that's what the feature map helps us preserve.
Actually that's what it allows us to bring forward and get rid of all of the unnecessary things that
even as humans we don't process so much information going into your eyes that at any given time like
gigabytes of information if you look at every single dot if not terabytes of information going into
your eyes per second and still we're able to proceed because we get rid of what is unnecessary only
focus on the important features features that are important to us and that is exactly what the feature
does.
So now moving on this is our input image and you we create a feature map so the front one let's say
the front one is the one we just created but then how come there's many of them.
But we create multiple feature maps because we use different filters.
Right.
And that's another way that we preserve lots of the information so we don't just have one feature map
we look for certain features and then or basically the network decides through its training and this
is something we'll discuss towards the end of the section through his training it decides which features
are important for certain types or certain categories and it looks for them and therefore will have
different filters and we'll talk about filters just now.
But basically Ill apply these filters so to get this feature map it applied a filter like the one we
saw but then to get this feature Mabbett apply a different filter to get this feature up apply a different
filter and so on.
And so basically it just creates these feature maps.
And actually that's why personally I think the term feature detector is better than filters.
Remember we're here we have this filter which we also can call a feature detector Well actually the
word feature detector I think is better suited.
And the reason for that is that's what the purpose is right.
We don't want to just we don't want to just filter out our image.
But even though that's a whole that's the same same just a question of terminology.
But basically we want to detect features.
All right.
In this in this lair we're going to our own this feature map we've detected where certain features are
in the image and this feature map we've detected where certain other features are where a certain specific
feature is located and this feature map will be detected where a certain other feature is located on
the image.
So that's that's what we're doing.
And listen we've got a couple of examples So here we're using and this is from Gip dot org.
Their documentation it's a free like kind of tool like paint and you can use it to adjust your images
or work with your images.
But basically they have some valuable examples in their documentation and here they have a picture of
the Taj Mahal and you can choose which filter you want to apply.
So if you download this program and you upload a photo into it and then you can actually start a conversion
matrix and apply filters and you will see that these things these English matrices actually applied
in image processing and design and so on.
So let's have a look at what we get what we get so.
So if we apply this filter five in the middle minus one one is one one is one minus one.
You can see that it sharpens the image.
And so this is it's quite intuitive if you think of it.
So 5 is the pixel of the main pixel like in the middle of the of the filter or the feature detector
and then minus one minus one one just one just kind of like reduces the pixels around the a in an intuitive
sense.
Then blur.
So basically takes equal significant gives equal significance to all of the pixels are all the one in
the center and therefore it combines them together and you get a blur edge enhance.
So here you can see that's minus one and one and then you get zeros right.
So you did delete to remove the pixels around the main one in the middle and you only keep this one
at a minus one and it gives you an edge and this was a bit harder to understand how it works.
Like probably harder just to think of it intuitively edge detect.
Right so this one probably makes more sense.
Right you take them a middle one.
You reduce the middle one.
The Probably like the strength of the middle pixel and then you look for the ones you look for.
These ones you see increase the strength of the ones around them.
So you have the ones there.
Yes that's.
That gives you like an edge takes and you can see which you get there and boss another one.
So the the key here is that it's symmetrical and you can see the image becomes asymmetrical as well
so you got like that kind of feeling that it's standing out towards you.
And that's what you get when you have like minuses here and plus here again this is very this is getting
a bit technical now but at least we can get some kind of intuitive and Lissa's go quickly through them
again.
So there's sharpen there's blur there's edgin hands there's an edge detect there's and boss as so as
you can see these are great examples of the same image but we're getting feature maps.
So we use different feature detectors to get different feature maps of the same image and therefore
now we have lots of the last of this versions of this image where in each one we've tried to detect
certain things in these terms they're not applicable to us.
Their second boss is probably not applicable to us in terms of convolutional neural networks but age
detect that's important.
We want to detect the edges edge enhance probably not blur sharpen so certain things like edgy text.
Probably the most important one for our type of work.
And in terms of understanding computers they will decide for themselves or neural networks will decide
for itself what's important what's not and it probably won't be even recognizable to the human eye.
You won't be able to understand what those features mean.
But the computer will decide and that's the beauty of neural networks that they can process so many
different things and understand without even having that intuition or without having that explanation
why they will understand which features are important to them whether we have a name for them or not
that that's a whole that's an irrelevant question for the artificial neural network.
And my favorite one.
Here's a image of Geoffrey Hinton from Geoffrey Hinton passed through one of these filters.
All right so that brings us to the end of Teresa Tauriel.
I hope you enjoyed learning about convolution.
The key takeaway is that convolution the primary purpose of we can evolution is to find features in
your image using the feature detector put them into a feature map and by having them in a future map
it still preserves the spatial relationships between pixels which is very important for us to you know
because if they are completely jumbled up then we've we've lost the pattern.
And at the same time it's important to understand that most of the time the features a neural network
will detect and use to recognize certain images and Klaas's will mean nothing to humans but nevertheless
they work.
And that's what convolution is.
And I look forward to seeing you next Tauriel.
Until then enjoy learning.