‫Welcome back.
‫In the last video, we looked at some aggregation functions with using numpy, such as getting the sum
‫of all the elements in array.
‫We saw how much faster numpy code is to pure python code when working on numpy data types.
‫So always use numpy aggregation functions on numpy arrays.
‫And then we left off with a brief introduction to standard deviation and variance.
‫Now the main thing to remember about these two metrics is, in a word, they're just the measures of
‫the spread of data.
‫But let's not talk about it.
‫Let's always focus on running code and seeing practical examples.
‫So that's what we're going to do on this video.
‫So I go demo of and var standard deviation STD is short for standard deviation var assure for variance
‫so we'll create high var array equals MP array not one 100 200 304,000.
‫5000.
‫Wonderful low var array equals MP array.
‫Maybe we'll make this one nice and simple.
‫Two, four, six, eight, ten.
‫There we go.
‫Now, before we even run any code.
‫Have a look at the numbers in these arrays.
‫Imagine these are actual data.
‫What would you infer from just looking at these numbers?
‫So this one starts from one goes to 100, 203 hundred, then jumps to 4000, finishes on 5000.
‫And this one kind of just goes to four, six, eight, ten.
‫So first of all, you might look at the difference between the two maximums.
‫That's what I kind of looked at.
‫So this one goes from 1 to 5000 and this one goes from 2 to 10.
‫So if we looked at this, we might say that these numbers to me, these are more spread out, whereas
‫these are pretty close together and the maximum distance is only eight and they have the same step in
‫between each one.
‫Whereas this kind of varies, it goes up by 100 a time and then up by a few thousand and then up by
‫a thousand to finish off.
‫But let's put it in a practice.
‫Let's run the code, let's see what's going on.
‫So shift into let's find the variance of both of them high bar and we can compress tab for auto complete
‫MP bar.
‫We want to have auto complete.
‫Oh no, we need to actually type the start of a variable.
‫That would be helpful.
‫Beautiful.
‫We'll hit shift and enter r.
‫Now, this is an absolutely massive number here and this is not too big.
‫Hmm.
‫Well, we're not going to look at the mathematics behind these two.
‫What's more important is the concept here and that variance, if we go up here, remember, variance
‫equals a measure of the average degree to which each number is different to the mean.
‫Higher variance equals wider range of numbers.
‫Yep, lower variance equals lower range of numbers.
‫And that makes sense with these two arrays.
‫This array here high var array has very high variance, whereas low array because the numbers are close
‫together, it has a lower variance.
‫So let's have a look.
‫Let's do standard deviation STD.
‫We want high var array and we also want the low var array.
‫We want standard deviation before we do this.
‫What do you think the standard deviation is here?
‫Is it going to come out similar to what this looks like?
‫Let's have a look.
‫There we go.
‫And that makes sense.
‫The standard deviation is higher for the high variance array than for the low variance array.
‫And what is standard deviation?
‫A measure of how spread out a group of numbers is from the mean.
‫Okay.
‫They've both got mean in their definition.
‫So let's have a look.
‫Let's find the mean.
‫Dot.
‫Hi.
‫Var array and we want np main lover array.
‫However, it's not defined.
‫This is why we should use tab autocomplete.
‫So what this is saying is the standard deviation.
‫This number here is the average distance.
‫A number is away from the mean.
‫So if the mean is 1600, that means on average any other number in this array in high var array is 2072
‫away from the next number.
‫The same goes for the low var array.
‫Any number in low is on average 2.8 away from the mean.
‫So that means ten this number is although it's four away from six because the mean is six, eight is
‫closer to six.
‫So that's what brings the average down.
‫All right.
‫Now let's check out this spread.
‫Remember, that's the main crux of standard deviation and variance is just the spread of numbers, how
‫they appear.
‫Let's plot them so we can do that by importing map plot lib.
‫We're going to have a look at map plot lib in a future section.
‫So don't worry too much if you're not sure what's going on here.
‫Map, map, lib pie, plot as plot.
‫That's what we want.
‫Yeah.
‫Plot dot hist.
‫So this is a histogram.
‫Hi.
‫We just want that tab order complete if in doubt.
‫Tab order complete.
‫See what comes up shift into.
‫No.
‫Module name at plot.
‫Oh, I got that wrong.
‫That plot loop.
‫There we go.
‫Okay, so there we got the high var array.
‫Now let's do the same for the low.
‫Var array.
‫Hist low bar array.
‫And we won't plot.
‫Don't show.
‫Wonderful.
‫So now we can see visually, if you couldn't see it from these numbers in the number line here.
‫Now, visually, we can kind of see the spread of the numbers of high vara is a lot bigger.
‫So there's a lot more white space here.
‫A lot of the numbers appear towards zero and only two of the samples appear up here, whereas in our
‫low var array, the spread is kind of similar across the whole thing because there's gaps of two between
‫each data point and the maximum distance here is only eight.
‫All right, so that's standard deviation and variance in a conceptual level.
‫Now, remember, if you do want the mathematics behind this, how these two values here, the standard
‫deviation and the variance are calculated.
‫I'll leave some extra resources.
‫But just remember, the main takeaway from this is that standard deviation and variance are measures
‫of spread of data.
‫Now with that being said, we finished up some of the most common aggregation functions you'll see in
‫NumPy.
‫Let's get into the next section where we can check out some more ways of manipulating arrays.