‫Well, we've completed all the metrics that Boss requested.
‫Hmm.
‫We should be able to put together a pretty good report containing our confusion matrix, a handful of
‫cross validated metrics such as Precision Recall and F1, as well as which features contribute most
‫to the model making a decision.
‫You might be wondering why haven't we got a heading for experimentation?
‫So if we come back here, why haven't we gone through this?
‫Right.
‫Well, the secret here is this whole thing, especially from the modeling section onwards, has been
‫experimentation.
‫We've been trying different things.
‫We've been testing out different methodologies.
‫We've been exploring our data.
‫We've been experimenting the whole time.
‫So actually we've been fulfilling this framework.
‫We've defined our problem, we've looked at the data, we've defined an evaluation metric, we've checked
‫features, we've done some modelling.
‫And the truth is we could keep going with our modelling, trying to improve it or trying to improve
‫our logistic regression model that we've left off with, or trying to find another classification model
‫that does better.
‫But of course things can't go on forever in practice in the biz.
‫This is kind of what we'd call a proof of concept, right?
‫We'd go try a few things.
‫We'd look at different data once we've done it in a segregated timeline, in a defined timeline, like
‫maybe this whole process took two weeks or something like that.
‫We've actually done this in about 2 hours here.
‫So really, don't worry.
‫If you're not picking up on things, these processes take a little bit of time.
‫So have we hit?
‫This is where we'd probably revisit our evaluation metric to kind of decide whether we're going to push
‫forward or do we have enough time.
‫We might talk to the client, we might talk to the business we're partnering with.
‫We might talk to some of our colleagues and go, Hey, we've tried a few things, but we still haven't
‫reached our evaluation metric of at least 95% accuracy of predicting whether or not a patient has heart
‫disease during the proof of concept.
‫This is kind of where you get to in machine learning, right?
‫As you work through these these processes and there's really no stop to it.
‫And that's what makes it also hard.
‫Well, in our case, we didn't get to 95%, but we did get pretty close.
‫Right.
‫We did get to to 90% or so accuracy with our logistic regression model, wherever it is, somewhere
‫up here, just over there.
‫So it's accuracy.
‫That was about 88% when we cross validated.
‫So what could you try next?
‫You might be thinking once you get to this point, you're like, Oh, well, I didn't fulfill the evaluation
‫metric.
‫Is everything we've done wasted?
‫No.
‫Well, what we found out in this case, we've shown what doesn't work, at least with the current data
‫that we have, at least with the models that we've had, we've figured out what doesn't work, which
‫is as just as as valuable as figuring out what does.
‫Because now we know, okay, we can try something new.
‫All right.
‫Well, this is really where step six comes into its own.
‫So this is where you'd go starting to do your own research, signing the search up different ways of
‫doing, doing different classification problems, building out improved models, different ways of looking
‫at data.
‫What you might ask yourself is.
‫Here if you haven't hit your valuation metric yet.
‫Ask yourself.
‫Some things like could you collect more data?
‫Because remember, machine learning models loved to learn on more data.
‫So could you collect more samples?
‫Right now we've got about 303 or so of patient samples.
‫Could you double that?
‫Potentially.
‫Maybe you have to talk to a subject matter expert.
‫Maybe you have to talk to a doctor or go back to the client you're working with or whatever project
‫you're on and go, Hey, is there any way we could get more data to enrich this data set?
‫Or could you try a better model?
‫You might be asking.
‫Well, we've used a logistic regression model and I've looked at here, where is it, the cyclone map.
‫There aren't too many that we could try again.
‫Hmm.
‫That's where you might have to start doing some research.
‫You might have to look into something like cat boots.
‫We won't cover for now.
‫I'll just plant the seed.
‫I'll write it here.
‫Like cat boots.
‫That's some extra curriculum for you.
‫Like Cat Boost or.
‫Or X g boost.
‫These are things you might want to look up, right?
‫So you might want to search those two things.
‫What else is there?
‫Well, okay, if we have tried a logistic regression model and we could we improve it.
‫So could we improve the current models?
‫In brackets beyond what we've done so far.
‫So maybe we've done a little bit of hyper parameter tuning using grid search CV or randomize search
‫CV.
‫So maybe we want to try a little bit more, see if we can squeeze a couple of percentage points of accuracy
‫out.
‫Again, you might be thinking, well, this doesn't seem very structured.
‫And the truth is that it's not.
‫It's all about minimizing your time between experiments and machine learning.
‫So let's say we did meet our valuation metric.
‫If we did so, if our model is good enough, how could you export it and share it with others?
‫So maybe we go here so I could learn how to export a model.
‫And then we'd go here.
‫Model persistence.
‫So say we did want to send a model to a colleague or a client or something like that.
‫You might want to look up here how to send a model.
‫We've done this in the psychic loan section, so if we go here, if your model is good enough.
‫You have hit your valuation metric.
‫How would you export it?
‫And share it with others.
‫Right.
‫So.
‫The real takeaway point here, the key to remember is we've gone through this entire framework now.
‫We've got some data.
‫We've defined a problem.
‫We've explored that data.
‫We've defined an evaluation metric.
‫We've looked at the features of our data.
‫We've done a bunch of machine learning modeling.
‫And really this whole process has been an experiment.
‫And we've used the tools that we've learned so far all within one beautiful Jupyter Notebook.
‫And now the final takeaway is that there is no real answer to what you could do next.
‫It's up to you to ask questions and to start figuring these things out, to start searching, to not
‫be afraid, to be wrong.
‫That is the biggest thing that will hold you back in machine learning if you don't ask questions, if
‫you don't go through things like this, or if you don't reach out to other people, if you don't ask
‫the community, if you don't look on StackOverflow, if you don't find examples, you're not going to
‫improve.
‫Finishing off this project, what you could probably do as an extension is to ask these questions here.
‫Maybe we can't collect more data because the data set is fixed, but we could potentially try a better
‫model like cat booster boost.
‫That's maybe something you could try.
‫And you could also try to do some research on how.
‫We can improve the current models using hyper parameter tuning.
‫But congratulations, you've just finished.
‫If you're following along, you've just gone.
‫End to end on a complete machine learning classification problem.
‫Isn't that amazing?
‫Predicting heart disease, using machine learning.
‫I hope you're starting to see.
‫Actually, this is probably the more important point rather than going on to what's next.
‫If you've followed through with this and you're still wondering, you're going, Holy goodness, Daniel,
‫there is so much here.
‫I don't know how I could remember all of this.
‫Do not worry, because I'm still in that position, right?
‫Even working as a machine learning engineer, I still run through that every single time.
‫Like, Oh, what can I do?
‫I always end up on rabbit hole searches like this and then bring it back into a notebook.
‫And trust me, the first time you build a notebook like something like this for a machine learning problem,
‫it probably won't look like what we've got.
‫Ours is pretty neat, right?
‫So it's pretty structured.
‫It goes through the framework step by step.
‫Your first time will probably have things all over the place.
‫And that's good.
‫That's good because it means you're trying new things.
‫What can you do from here?
‫Check out this code.
‫A It'll all be in the resources section.
‫It'll be in an attachment somewhere that you can access, go through it.
‫If you can find another classification data set somewhere on the Internet, you can find those pretty
‫easy.
‫So classification.
‫Machine learning data set.
‫You can find it find a whole bunch.
‫So data sets UCI machine learning repository that's what we found the heart disease data set right back
‫in the beginning.
‫That's where we found this.
‫Maybe you could find some more where you can find someone, Kaggle or KDE nuggets.
‫There's a whole bunch of different resources, so maybe you could try find another data set and then
‫go back through this pipeline that we've created for a classification problem just now with a different
‫data set.
‫And you could have an end to end insert your data set here classification problem, but that's going
‫to wrap up project one end to end classification.
‫If you have any questions, feel free to leave them.
‫Otherwise, I'll see you in the next set of videos.