‫Welcome back.
‫How did you go?
‫Did you try and use a different model to improve upon our score?
‫That's that's one of the key things to learn and take away from this course, actually, is just experiment,
‫be willing to try different things, even if you're not sure what's going on.
‫Because put it this way, you're far smarter doing than you are thinking about it.
‫So write more code, see what works, see what doesn't.
‫And that way you'll start to build up your intuition on what's good and what's not good in the world
‫of machine learning.
‫So what we're going to do here is we're going to try and improve on here on this score.
‫What if Reg didn't work or the score didn't fit our needs and then we can go?
‫Well, we could always try a different model.
‫So now let's refer back to the machine learning map again.
‫We'll follow it through.
‫We ended up here, we're not quite sure what few features should be important.
‫So we might have tried.
‫You might have tried lasso, you might have tried elastic net, you might have tried SVR.
‫But let's just say that reg regression was not working at all.
‫So let's dive up into here Ensemble Regresses and SVR with a different kernel.
‫So the only difference between these two is different kernels.
‫But let's jump into ensemble regresses because ensembles are an important concept in machine learning,
‫so let's read about them.
‫The goal of ensemble methods is to combine the predictions of several base estimators built with a given
‫learning algorithm in order to improve generalizability.
‫So generalizability is the ability for a machine learning model to predict accurately on unseen data
‫and robustness over a single estimator.
‫So the key takeaway from an ensemble model compared to other models is that it combines the predictions
‫of several models.
‫So rather than just using one type of model, it's like if you were making a decision, maybe ten doctors
‫are smarter at making a decision on a diagnosis versus just one doctor.
‫So that's the whole goal of ensemble methods.
‫But if you want to read more, you can dive through the documentation.
‫But for now, as always, let's practice with some code.
‫I'm going to copy this.
‫We could always try a different model.
‫How about we try an ensemble model and we're right down here?
‫An ensemble is a combination of smaller models to try and make better predictions than just a single
‫model.
‫And then we're going to say, learn ensemble models can be found here.
‫And then if we were to go through that documentation, there's a very popular model in here called the
‫random forest algorithm, random forest forests of randomized trees.
‫So I'm just going to tell you, this is one of those things where it's like the random forest classifier
‫is very powerful.
‫And then same with the random forest regression.
‫Now, if you'd like a more deep dive on random forests, they're based on decision tree.
‫So let's look up this decision trees.
‫I'm going to throw a lot of information at you, but I trust that you'll be able to look this up and
‫and understand it yourself.
‫We're going to focus on writing the code.
‫This is what?
‫The Random Forest.
‫Oh, Statcast.
‫That's a great video.
‫So decision trees are like, you start with something and then you go, yes, no, yes, no, yes,
‫no.
‫Jump through it.
‫A random forest is a combination of lots of lots of lots of different decision trees.
‫So random forest, decision tree.
‫There we go.
‫What's this look like?
‫Yeah.
‫So if we leverage lots of different decision trees, what do we come out with?
‫And if you'd like to see what the random forest is, so I would go Random Forest.
‫This is how I understand different machine learning algorithms.
‫Random Forest explained understanding random forest.
‫A complete guide to the random forest algorithm.
‫Random Forest explained Random Forest.
‫Here we go.
‫Here's one of my favorites.
‫Random Forest, simple explanation.
‫From Will Coercion.
‫Coercion.
‫He's got a great blog on Medium with all different data science topics, so make sure you go and check
‫out his blog, Random Forest Simplified.
‫So if we start with an instance which is one single sample and then we make a decision and we want to
‫classify it into something or class A, class B, class something, and then it's got a final class.
‫So it combines the decisions of one tree to tree, tree up to N.
‫I mean, you can have hundreds or thousands of different trees and then it combines them into majority
‫voting and then it picks the final class.
‫Now this is more for classification.
‫However, you can do the same thing with regression and psychic.
‫Learn has a wonderful module.
‫If we have a look, let's go.
‫Random forest regress our skyline.
‫Let's see how we can use this.
‫So there was a lot of information there.
‫Let me just do a quick recap.
‫We're going to use this model here, which is an ensemble algorithm scale.
‫Learn ensemble and ensemble is a combination of different models to make one prediction, a very common
‫or very powerful algorithm, machine learning algorithm that leverages the ensemble technique is a random
‫forest.
‫And a random forest is a combination of lots of different decision trees.
‫In our case, the default value is that it's going to use 100 different decision trees to make a decision.
‫So our by default, if we use this model, it's actually 100 different models predicting on what we
‫want it to break down.
‫So let's see it in action if you want a background of it, how it works, there's plenty of articles
‫for that.
‫We're going to write the code for it, so let's go import the random forest regression model class from
‫the ensemble module.
‫You might have noticed that this one was from the linear model.
‫So the difference here is that this is an ensemble.
‫So it's from the ensemble module ensemble import, random forest regression.
‫Now we'll also find out later on that there's a random forest classifier we may have already seen that
‫may not have so random forest.
‫That's how powerful of an algorithm it is.
‫It can be used for both classification and regression.
‫So we're going to set up the random seed, we're going to create the data.
‫We're repeating a lot of the code here.
‫So this is the beauty of Cyclone is that many of its APIs and modules and classes and functions reuse
‫similar code if they have similar functions.
‫So our random forest model, we want to train it on some data and score it.
‫That's a similar function to what we did with the Ridge model.
‫It's just that the underlying algorithm under the hood for ridge and random forests is different.
‫So the way it comes to make predictions under the hood is different.
‫That's what we're trying to use to improve upon this.
‫So create the data, the X, we don't really have to recreate X because it's above, but we're going
‫to anyway.
‫For completeness, we'll drop the target axis equals one, y equals housing.
‫DF These are our labels.
‫So we're trying to predict the median house price based on the X features, and then we're going to
‫split into train and test sets.
‫X train x test y, train y test equals train test split.
‫Then we're going to pass it in X and Y.
‫We'll set the test size equal to 0.2 and then we're going to create random forest model.
‫It's going to say this as model equals random forest regression and then model fit on the training data
‫because we want our model to learn from the training data and predict on the test data.
‫And then we're going to check the score of the model on the test set model the score.
‫How do you think this will go?
‫Intuition based.
‫You could have used never used a random forest in your life.
‫I just want you to have a think how this might go.
‫Let's remind ourselves of the docstring here and estimate is we have 100 smaller models, 100 decision
‫trees predicting on our data versus the ridge.
‫It's not going to tell us too much, but yeah, so it doesn't tell us too much.
‫Linearly squares with l two regularization.
‫You can look into what that means, but let's just have a think.
‫Is this going to perform better or worse than our Ridge model?
‫Let's find out together.
‫You ready?
‫Three, two, one.
‫Go.
‫It's got 20,000 samples to go through.
‫So it has to make lots of little decisions on all of those samples.
‫Would you look at that?
‫Now, that is the beauty of machine learning.
‫We did one quick experiment.
‫We even wrote more code than we had to write.
‫And we got a boost of what's that, almost 20 to 22 score points.
‫Remember the default score for regression model in CI kit learn is the default evaluation metric is
‫the coefficient of determination, which is the R-squared value.
‫We're going to look at more predictive metrics later on, but if you'd like to look into that or you
‫could just go regression prediction metrics.
‫There we go.
‫Regression metrics for machine learning three best metrics to evaluate regression model, beautiful.
‫We've got heaps of resources on these things.
‫Again, we're focused on just doing experiments and writing code.
‫If you want to research more, please, this is how I learn more things.
‫Just go search for it.
‫Follow your curiosity.
‫Back up.
‫Those things that go.
‫Hmm.
‫Then you didn't really cover that.
‫So then come in here, type some stuff, and fill in the gaps.
‫All right.
‫But now, now that we've covered choosing a regression algorithm, how about we do the same?
‫But for a classification algorithm?
‫So we've gone through these steps and had a look at a few of those.
‫Let's now try with classification.
‫So I'm going to write a little heading here and that way when we come back, we know what we're working
‫on.
‫2.2 can be picking a machine learning model for a classification problem.
‫Now, if you'd like to challenge yourself, I would definitely encourage you to do this.
‫Is that have a look go line classification data set.
‫Find something that toy data sets.
‫Lode.
‫Iris.
‫Iris is a classification data set.
‫You can check that out.
‫Or you could go into the toy data sets, find something to do with classification in here.
‫And then once you've got a data set instantiated.
‫Go through this little checklist and try to build one of these models on that data set that will really
‫it's going to be very similar to what we did for regression, but that will really help you learn if
‫you give these things a go yourself.
‫Otherwise, we're going to do that together in the next video.
‫I'll see you there.