‫Welcome back.
‫So what is data?
‫I mean, we've talked about it already and we should have an idea of what data is.
‫And we also know why business care about data, right?
‫We live in a world now where data is valuable and some of the most valuable companies are valuable because
‫they collect data.
‫Companies like Facebook and Google sell data to make money.
‫They provide products for customers by using the data to improve their products.
‫Data is the new gold.
‫So businesses care because by understanding their data it could be part of the product, such as creating
‫a YouTube recommendation engine.
‫We can ask if the company is doing okay to monitor the company sales statistics, and you can also use
‫data to improve the company and ask, can we do better?
‫What can we improve on?
‫How can we make our apps faster?
‫So businesses care a lot about data, but I want to take a bit of time to really understand what data
‫engineers do by manipulating or collecting this data.
‫So let's talk about the four different types of data.
‫We learned that there's things called structured data.
‫Now, this is data that is actually organized well for us to understand and read because data gets collected
‫in different ways from different sources.
‫Right?
‫But once on a computer, there's a standard way usually in a table or what we call a matrix, a table
‫that makes it easy to read.
‫Attributes are usually columns and rows are usually instances, and sometimes we have the outputs in
‫the rightmost column, like we saw with machine learning where we predict an output based on the inputs
‫and structure.
‫Data usually comes from things such as relational databases like MySQL post graphs and other relational
‫databases that allow us to do something called SQL queries.
‫That is, to perform actions on the structure data to collect information.
‫Then there's semi-structured data.
‫There's still structured data, but often in something like an XML CSV, which we've seen or JSON form.
‫And these are just different extensions and different ways to store data in different files.
‫For example, a really popular websites to grab data that's freely open online and find data sets,
‫maybe even do competitions, or see other people's notebooks and talk about machine learning topics.
‫Well, Kaggle allows us to actually look at data sets that are available for us to use and play with.
‫And in here if I type something like XML.
‫It'll give me all of the information or all the data.
‫Let's make this bigger that are in XML format.
‫So, for example, if I click on blood cell images.
‫And I.
‫Take a look over here.
‫I see that the data sources, there's some CSV files, but there's also some XML files.
‫And if I scroll down, there's also some images as well.
‫So different formats of data that we can use.
‫By the way, if you click on filter here, you'll actually see some of the file types available for
‫data.
‫Now we know about CSPs.
‫There's also JSON where JSON, if we take a look and let's go to this beer styles one.
‫If I scroll down, we see that it's in JSON format and JSON format.
‫You can see has this tree like structure to explore data.
‫So just a little bit different than CSV, just another way to organize data.
‫Like I've mentioned.
‫On Kaggle, you'll find all these types of data sets using different things.
‫SQL Light, for example, is a type of database and using SQL, so structured data.
‫So if I do SQL Lite over here and search for something using relational databases.
‫You'll see that this uses SQL.
‫And it tells us what kind of comms it has.
‫So lots of ways to explore Kaggle, but these two types of data are usually well organised and easy
‫for us to manipulate with tools such as pandas for CSPs.
‫And you can actually do SQL as well with pandas.
‫The third kind is unstructured data.
‫And Danielle has already mentioned this, right?
‫This idea of an unstructured data are well, they're not usually in simple formats that we can really
‫manipulate and analyze easily.
‫These are often things such as emails or PDFs or any sort of documents where perhaps actually understanding
‫them is a little bit more difficult.
‫It's unstructured.
‫Finally, we have binary data.
‫These are things such as audio files, image files, video files.
‫They're in binary.
‫That is ones and zeros that computers can understand.
‫But for us to categorize them, that's really, really difficult.
‫As you can see, it goes from top to bottom.
‫From a really organized, well labeled data to something that is a little bit harder to organize, especially
‫for machines.
‫So why do we have to talk about these four types of data?
‫Well, because most businesses.
‫The bigger they get aren't only going to have structured data, they're going to have all sorts of data,
‫all sorts of inputs that needs to somehow all work together.
‫So one of the jobs of a data engineer is to essentially use the fact that there's all these types of
‫data and somehow combine them or organize them in a way that is useful to the business.
‫Let's take a break here and explore some more in the next video.