Types of SL models
There are, in general, two types of SL models: regression and classification models. The difference between the two is quite simple and lies in the nature of the response variable.
Regression models attempt to predict a continuous response. This means that the response can take on a range of infinite values. Consider the following examples:
- House pricing: Where the value to be predicted is the cost of a house based on features such as square footage, number of bedrooms, and location
- Temperature forecasting: Where the model predicts the temperature for future days or hours based on historical weather data
- Stock market price prediction: Where the continuous response could be the future price of a stock based on its historical performance and other economic indicators
Classification models, on the other hand, predict categorical responses. These responses are discrete and have a finite number of values, often referred to as classes or categories. Here are some examples:
- Email spam detection: The model classifies emails as either “spam” or “not spam” based on content, sender information, and other attributes
- Medical diagnosis: A model might classify patient outcomes based on test results, predicting categories such as “disease” or “no disease”
- Image recognition: Classifying images into predefined categories such as “cat,” “dog,” “car,” and so on, based on pixel data and patterns
Both regression and classification tasks use a similar process of learning from historical data, but their applications and evaluation metrics differ due to the nature of their output. Our earlier heart attack example is classification because the question was: Will this person have a heart attack within a year? This has only two possible answers: Yes or No. We will see a full example of regression later in this chapter and full examples of both classification and regression in the next chapter.
Deciding between classification and regression
Sometimes, it can be tricky to decide whether or not you should use classification or regression. Consider that we are interested in the weather outside. We could ask the question, How hot is it outside? In this case, your answer is on a continuous scale, and some possible answers are 60.7 degrees or 98 degrees. However, as an exercise, go and ask 10 people what the temperature is outside. I guarantee you that someone (if not most people) will not answer in some exact degrees but will bucket their answer and say something like It’s in the 60s.
We might wish to consider this problem as a classification problem, where the response variable is no longer in exact degrees but is in a bucket. There would only be a finite number of buckets in theory, making the model perhaps learn the differences between 60s and 70s a bit better.
What if we aren’t predicting anything, but instead we wanted to use ML to simply better understand and interpret our data? That’s where UL comes in very handy.