Classification and regression are two major types of supervised learning used in statistics and machine learning. Both start with input data and a known target, then learn a rule for making predictions on new cases. The key difference is the kind of answer they predict: classification predicts a category, while regression predicts a number.
This distinction matters because it determines the model, loss function, evaluation metric, and how the result should be interpreted.
In classification, the model learns boundaries that separate classes, such as spam versus not spam or disease present versus absent. In regression, the model learns a numerical relationship, such as predicting temperature, price, or time. Some algorithms, such as decision trees, random forests, and neural networks, can be adapted for both tasks, but they use different output layers or training goals.
Choosing the correct task type helps avoid misleading predictions and incorrect measures of performance.
Understanding Statistics: Classification vs Regression
The target variable decides more than the format of the answer. It changes what counts as an error. For a school attendance model, predicting absent when a student is present is one kind of mistake.
Predicting present when a student is absent may be more serious if the result triggers a welfare check. A classification model can give a probability before a final label. The school must then choose a cutoff.
A lower cutoff catches more possible absences, but it creates more false alarms. This tradeoff is why accuracy alone can hide important problems.
Class balance needs careful attention. Imagine one hundred emails where ninety five are ordinary and five are spam. A model that calls every email ordinary gets ninety five percent accuracy, yet it never finds spam.
Precision tells you how often a positive prediction is right. Recall tells you how many real positive cases the model finds. A confusion matrix lists correct and incorrect results for each class.
It makes these patterns visible. In medical screening, high recall is often important because missing a real illness can be harmful. In a system that blocks accounts, higher precision may matter because wrongly blocking innocent people causes harm.
Regression errors have size and direction. If a weather model predicts twenty degrees when the true temperature is twenty one degrees, it is off by one degree. If it predicts thirty degrees, it is off by ten degrees.
Mean absolute error treats the ten degree error as ten times as large. Mean squared error gives large errors extra weight, so a few extreme misses can strongly affect the score. This can be useful when big mistakes are especially costly, such as estimating flood levels.
It can be misleading when unusual data points come from faulty measurements. Students should check the units of a regression score. An error measured in degrees, kilograms, or pounds has a practical meaning that a bare score may not show.
Real data rarely fits neatly into either task. A shop might predict the exact number of items it will sell, which is regression. It might instead predict whether stock will run out, which is classification.
The second target can be created from the first by choosing a boundary, but information is lost when exact counts become labels. Data collection matters just as much as model choice. Inputs must be available at the time of prediction.
A model should not use tomorrow's information to predict today. Keep separate training, validation, and test data so that the final score reflects new cases.
Look for missing values, biased samples, and labels made by people who may disagree. Good predictions require a clear target, sensible evaluation, and honest checking of errors.
Key Facts
- Classification predicts a discrete label, such as cat, dog, or bird.
- Regression predicts a continuous value, such as house price, height, or speed.
- A simple regression model can be written as y = mx + b.
- Classification often estimates class probabilities, such as P(class = spam) = 0.93.
- Common classification metrics include accuracy = correct predictions / total predictions, precision, recall, and F1 score.
- Common regression metrics include MAE = mean(|y - y_hat|), MSE = mean((y - y_hat)^2), and RMSE = sqrt(MSE).
Vocabulary
- Classification
- Classification is a supervised learning task where the model predicts which category or class an input belongs to.
- Regression
- Regression is a supervised learning task where the model predicts a numerical value from input data.
- Target variable
- The target variable is the output the model is trained to predict, such as a class label or a number.
- Decision boundary
- A decision boundary is the line, curve, or surface that separates predicted classes in a classification model.
- Loss function
- A loss function measures how wrong a model's predictions are during training.
Common Mistakes to Avoid
- Treating any number as a regression target is wrong because some numbers are actually category codes, such as 0 for no and 1 for yes.
- Using accuracy to judge a regression model is wrong because regression predictions are usually not exactly equal to the true value.
- Using linear regression for a category prediction without adjustment is wrong because its outputs are not guaranteed to be valid class probabilities.
- Ignoring class imbalance in classification is wrong because a model can look accurate by mostly predicting the majority class.
Practice Questions
- 1 A model predicts whether 200 emails are spam or not spam. It correctly classifies 174 emails. What is its accuracy?
- 2 A regression model predicts house prices with errors of 25,000, 30,000. What is the mean absolute error?
- 3 A hospital wants to predict whether a patient has flu, COVID, or neither based on symptoms. Is this a classification or regression problem, and why?