‫Welcome back.
‫And in the last video, we learned a few little arithmetic tricks.
‫And we learned that arithmetic is a fancy word for mathematical operations that we can do on numpy arrays.
‫And so now we're going to learn a bit more about manipulating NumPy arrays through aggregation.
‫Now, aggregation is another one of those fancy words.
‫Let's Google it.
‫Define aggregation.
‫The formation of a number of things into a cluster.
‫Yeah, that's kind of what we're going to do.
‫So aggregation, performing the same operation on a number of things.
‫And in our case, the number of things is our numpy array.
‫So let's have a look.
‫You might be familiar with the sum method.
‫So some or python some.
‫So let's make a list.
‫List we list equals one, two, three.
‫Nice and simple and we'll go type list.
‫List.
‫Listed list is not defined.
‫Of course, we're making typos.
‫There we go.
‫Type Python list.
‫So if we wanted to get the total of listed list, we could type in this some listed list.
‫Now, that's a python function right there.
‫Some.
‫But NumPy has its own version of these.
‫So let's see, let's go back to our a one array, which is a lot more trusty.
‫Well, not a lot more, but it's just a different type to list the list, even though it contains the
‫same information.
‫It's in a type numpy and D array.
‫So let's have a look.
‫What can we do with a one?
‫We can call some on it and we'll get six.
‫We can also call NP some on a one.
‫We get six as well.
‫Okay.
‫Now, why would they be different functions here and what is the main difference between these two?
‫It can get confusing having two different ways to perform the same aggregation on a set of data and
‫remember aggregation just performing the same operation on a number of things.
‫Some is just an aggregation to add up all the elements of a certain array or list.
‫Now the little tidbit you can use here is of which sum you should use, whether it be Python some or
‫NP some is use Python's methods on Python data types and use non PI's methods on numpy arrays.
‫So this is some.
‫Actually, we might do this one in markdown, so that's a bit easier to read.
‫We don't want heading one.
‫We'll put this in code.
‫So this is our tidbit.
‫And pay some.
‫There we go.
‫Now let's see this in action.
‫So we're going to create a massive numpy array.
‫A massive array.
‫This will really demonstrate the power of how fast numpy can be.
‫Remember how we talked about right at the start.
‫We'll create a massive array with 100,000 different elements.
‫This won't be too uncommon for machine learning problems.
‫So there we go.
‫Size 100,000.
‫Let's just view just so we can see what it looks like.
‫Massive array.
‫Lets view the first 100 elements.
‫There we go.
‫All random numbers.
‫Great.
‫We might reduce that to ten, so we have a bit more space.
‫Wonderful.
‫Now we're going to use one of Jupyter notebooks or Python's Magic Functions with remember, a magic
‫function has a little percentage sign at the start of it.
‫We got time it and this is going to time.
‫How long a particular line of code takes to run?
‫We'll try python some so some across massive array.
‫We'll just put a little comment here.
‫So Python some and then time it.
‫We want num pi's some massive array.
‫And we'll put a little comment here.
‫Now paes some.
‫So let's run this and see what happens.
‫Now, it might take a little while because it's adding up 100,000 different numbers.
‫So what have we got here?
‫Well, the first line is going to be dedicated towards this line here, and the second line is going
‫to be dedicated towards non pi some.
‫So the first line to sum over using Python, some across 100,000 different numbers took 17.9 milliseconds
‫per loop.
‫Plus or -2.94 milliseconds, ten loops.
‫But now implies some impact.
‫Some put that amp on, some took 34 micro.
‫This little symbol here is mu four microseconds.
‫So let's convert that.
‫We want microseconds to milliseconds.
‫Equals one microsecond equals 0.001 milliseconds.
‫So let's convert.
‫This is merely seconds.
‫We want to convert that to microseconds.
‫7.9 milliseconds, actually two microseconds.
‫There we go.
‫So 17.9 milliseconds equals 17,900 microseconds.
‫My goodness.
‫So 17,900 divided by 34.
‫Equals 526.
‫So that means num pies, npc's sum is 526 times faster than Python's Sum.
‫So going forward, remember the tidbit here, if you're working with NumPy data, use numpy methods.
‫But if you're working with Python data types, use python methods.
‫Because going back to the start, numpy is being optimized to perform numerical calculations.
‫So whenever you want to perform numerical calculations, always remember to use the numpy version of
‫an aggregation function.
‫And speaking of aggregation functions, let's have a look at a few more we'll go to we'll have a look
‫at our second array that we created.
‫There's our trusty two dimensional array from before.
‫Let's find the main of this.
‫So we've got NP main A2 3.633.
‫Beautiful.
‫Now the main would be if we added these up one plus two plus 3.3 plus four plus five plus 6.5, then
‫divided by the total number of items.
‫So six, we won't do that, but that's how you find the mean.
‫If we wanted the maximum, you can use MP max.
‫If we wanted the minimum, we can do MP min.
‫If we wanted the standard deviation, we can do STD, the standard deviation.
‫Remember, we can always press shift tab, compute the standard deviation along the specified axis.
‫If we pass no access, it's going to do it over the entire array, which is what we want.
‫Wonderful.
‫And if we want the variance, we can do np over a two wonderful shift tab, compute the variance along
‫the specified axis.
‫And now you might be saying, Danielle, what the hell is the standard deviation and what the hell is
‫the variance?
‫Well variance, let's put a little point here.
‫What you need to remember is the variance equals the measure of the average degree to which each number
‫is different to the mean.
‫So we want higher variance equals wider range of numbers.
‫Lower variance equals lower range of numbers.
‫Beautiful.
‫And now the standard deviation.
‫The formal definition is standard deviation equals.
‫Standard deviation is a measure of how spread out a group of numbers is from the main.
‫The standard deviation is actually just the square root of the variance.
‫So let's try this out.
‫So we got MMP.
‫There's another aggregation function square root of NP over a two should be equal to the standard deviation.
‫So let's write this down.
‫Standard deviation equals square root of variance.
‫Shift and enter.
‫There we go.
‫These two numbers are the same.
‫Now, if you'd like to look into a bit more of the math behind the standard deviation and the variance,
‫I'll leave some resources in the resources section for you.
‫Otherwise, in the next video we'll see a bit more of an example of these two.
‫In practice, they're an important concept to know going forward in any type of data science or machine
‫learning or statistical work.
‫So let's take a quick break.
‫Play around with some aggregation functions that you've seen on some of the arrays we've made or create
‫your own.
‫And I'll see you in a second.