‫Since we've built a model which is able to make predictions, we've done some on the test data set,
‫we've exported them.
‫Now the people you share these predictions with or in fact yourself, I know in my case I'm very, very
‫interested, might be curious of which parts of the data led to these predictions.
‫So this is where feature importance comes in.
‫Feature importance.
‫In other words, it seeks to figure out which different attributes of the data were most important when
‫it comes to predicting the target variable, or in our case, the sale.
‫Let's write that down here.
‫Feature Importance seeks to figure out which different attributes of the data were most important when
‫it comes to predicting.
‫Let's put this in bold target variable.
‫So in our case, it's the sale price.
‫All right, so how might we do this?
‫We know we're using a random forest aggressor.
‫We've seen in a previous video in the classification project, as well as the Psychic Loan Project,
‫some psychic loan models have the attribute feature and performances.
‫But if we didn't know if we're using a model that we weren't familiar with, maybe we could go something
‫like this.
‫A random forest regression feature importance.
‫Beautiful.
‫So maybe this will give us some good results.
‫If we were to go and explore these, we got this like get loan documentation.
‫That's always helpful.
‫So we're looking in here.
‫Maybe we might search for.
‫Well, it knew what we want.
‫Feature importance is return.
‫The feature importance is the higher, the more important the feature.
‫Let's see what that actually does.
‫So let's take our model.
‫So find feature importance of our best model.
‫So we'll take our ideal model and we'll find feature importance.
‫Wow.
‫Now that returns a fairly large array.
‫Whole bunch of different values here.
‫Some of them are zero.
‫Some of them are pretty low.
‫Ten to the negative six.
‫They're.
‫But what does this have in common?
‫Let's check out the length of this.
‫What does this have in common with our training data set?
‫102, 102.
‫Ah, what we might infer is that we've got 102 columns here and we've got 102 values here.
‫So we're getting a value for every feature.
‫So sales ID would map to this.
‫And machine ID would map to this.
‫Now we could make a dictionary by going x train columns and intertwine these values.
‫But I don't know about you.
‫I prefer to see things visually.
‫So let's make a why don't we make a helper function which helps us do that?
‫So we might make a helper function, helper function for plotting feature importance so we can see it
‫visually.
‫So go here def plot features and we might take a list of columns and a list of important pieces and
‫we're going to set end to 20.
‫Now this will make sense in a second because we have 102 values here.
‫But realistically, if we want to look at a plot, maybe we only want the top 20 values.
‫And I mean, that's why we set it so we can adjust it with this little function here.
‫A helper function is a great I should have started using functions earlier.
‫Writing functions saves a lot of time.
‫Put it that way.
‫I used to just write the same line of code over and over and over again in different cells.
‫But I've started to get into the habit of writing functions for different uses.
‫So what we might do is use a little pandas trick here called chaining.
‫And what that means is just simply putting a number of different pandas functions in brackets.
‫And we'll see that in a second because we're going to make a dataframe called DF and it'll have two
‫columns.
‫Features can be columns and feature importance is can be important is right.
‫So we're just creating a data frame here and then we're going to call sort value.
‫So see here, these are still within the brackets, these two here.
‫So what this means is it's going to do PDO, DataFrame and dot sort values in one hit and we want to
‫sort it by feature importance and we want ascending equal to false.
‫So we want it to go from highest to lowest.
‫Yeah, that makes sense.
‫And then we want to reset the index drop equals true.
‫Right.
‫Actually this bracket should be down here.
‫So because we've got a bracket here and here and two dots, that means it's just going to do these three
‫pan, two steps in one hit.
‫And now we need to plot.
‫So plot.
‫The DataFrame we've created.
‫So what we might do is go fig ax, instantiate a plot, plot subplots, and then we might do a horizontal
‫bar.
‫Because in my experience plotting features, it looks really good on a horizontal bar.
‫And now we only want up to N, that's why we have in there.
‫So that's going to be the first 20 examples.
‫And the same with feature importance is we only want up to 20.
‫So the first 20 then we're going to set the Y label.
‫So ax dot set y label to features because it's a horizontal bar, X and Y are rearranged and we go ax
‫set X label, add some communication to it, we'll put feature importance down the bottom.
‫Let's see what this looks like actually.
‫So we've got a helper function, we've just run it there.
‫All we have to do to call this is go plot features and then we'll pass it column.
‫So we just want the columns from X train because that's a variable there.
‫And then the feature and Potencies are just the ideal model dot feature importance is let's see what
‫this looks like.
‫Okay.
‫So we've got the features on the left here.
‫Their value.
‫Here, I want the top one.
‫The most valuable feature to be at the top.
‫So what we might do is ax dot.
‫How do we do it?
‫Invert.
‫Can we invert?
‫I think it's asked.
‫Invert y axis.
‫Maybe that's not a function.
‫If in doubt, run the code.
‫Invert y axis.
‫That didn't work.
‫Maybe it is a function.
‫Bomb.
‫If in doubt, run the code.
‫Trust your instincts.
‫Right.
‫That's how you learn things.
‫You practice by running it without looking up.
‫If in doubt, you can always look it up, right?
‫You can always ask how to invert a horizontal bar plot and map plot lib.
‫Google should be able to help.
‫All right.
‫So what can we infer from this?
‫So there's a fair bit going on.
‫We've got about 20 different features here.
‫So it's time that you made.
‫Let's have a look at our next train.
‫Don't head.
‫Let's get that in here.
‫So it's saying that the year that the bulldozer was made is the most important feature based on the
‫ideal model and then the product size.
‫What is that?
‫Let's look at our data dictionary.
‫Product size.
‫Don't know what this is.
‫Okay, that's great.
‫Um.
‫The size grouping for a product group.
‫Subsets with product group.
‫This is an example of where you maybe need to do some more research and figure out what features are.
‫What if you've got something like this in your data dictionary?
‫Might have to reach out to like a subject matter expert.
‫Or we could just go x train.
‫Product size.
‫Maybe we check the value can't.
‫Okay.
‫So there's one, two, three, four, five, six different sizes.
‫Now zero is going to be missing.
‫So there's a lot of missing values there.
‫All right.
‫But maybe this doesn't really tell us much, so maybe we could do it on our original dataframe.
‫There we go.
‫All right.
‫So the product size, well, that kind of makes sense that the product size is influencing the sale
‫price sale year and closure.
‫Also doing a fair bit of damage there and closure value counts.
‫Okay.
‫So that's hmm.
‫That doesn't really make much sense to me.
‫So and then we go back to the data dictionary now we find and closure machine configuration.
‫Does machine have an enclosed cab or not?
‫All right.
‫So maybe we need a need to figure out what these different values might mean.
‫But this is the kind of exploration you probably do towards the end of after you've built a model.
‫Right.
‫Is bringing this to someone, after you've made some predictions, you might take this in a sort of
‫a presentation.
‫So maybe you're meeting with a client or meeting with a another team member or something and going,
‫Hey, this is what the most important features our model has found.
‫Does this agree with your sort of intuition, with what someone already knows about the data?
‫Does this make sense if a model is using these features to derive its predictions, does this make sense?
‫Or this information here might influence how you go about collecting data in the future on your bulldozer
‫sale.
‫So you might put a bit more effort into the values here that are contributing most to predicting the
‫sale price of a bulldozer in the future.
‫The last probably question that I'll leave you with is we've kind of just glazed over it a little bit,
‫but I want you to do a little bit of research is why might knowing the feature importance of a train
‫model be helpful?
‫That's a finishing question.
‫So question to finish.
‫Is it going to involve some research?
‫Why might, knowing the feature importance is of a trained machine learning model be helpful?
‫So that's going to finish off this project.
‫To finish off with a little question here, what you might want to try and do is if you've followed
‫along with all of these steps here, you might want to see how far you can you can go with hyper parameter
‫tuning.
‫So something like this, maybe you could leave this run for a while on your computer, see what it finds
‫out, see if you can improve the the valid arm sleep score and see how far you'd get up on the leaderboard.
‫Now, if you do reach a point where you're not really improving with a random forest model, what a
‫final extension may also be is.
‫Final challenge.
‫Is what other machine learning models could you try?
‫On AL data set.
‫And what that might involve is something like searching for the psychic learn machine learning map.
‫So what we've done is we've gone through this and we've gone to regression problem.
‫We found ensemble regresses and we've used a random forest.
‫Maybe you want to try another one of these regression models using the formatted data that we've worked
‫through in this notebook.
‫So I'll leave a little hint here.
‫So hint.
‫Check out the regression section of this map or try to look at something like cat boost dot A.I. or
‫x g boost A.I. So these are two extra resources.
‫So extra curriculum, of course, they're optional at this point because once we've started to work
‫through these sort of projects, now we've gone end to end on a full machine learning project.
‫Your next steps are kind of trying to figure things out on your own, taking what you've learnt from
‫here, and then expanding on that.
‫That's the kind of information, that's the kind of knowledge that that is really going to help out,
‫right?
‫So rather than sort of always going through projects like this, it's doing some research and trying
‫out new things.
‫Remember step six of our little framework here, experimentation.
‫That's the challenge I leave to you.
‫But that being said, if you have made it this far, if you have gone through the entire project, congratulations.
‫We've just gone end to end on a regression problem using machine learning.
‫How cool is that?
‫So have a look at the notebook.
‫See, what I'm proving is you can make if you have any questions whatsoever, feel free to ask them.
‫Leave them in the Discord Chat or in the Udemy interface somewhere where you can leave a question and
‫we'll see what we can do from there.
‫But congratulations working through a first end to end regression project.
‫How exciting.
‫All the best.