Machine learning is a way for computers to find patterns in data and use those patterns to make predictions. Two of the most common prediction tasks are classification and regression. Classification sorts an input into a category, such as spam or not spam.
Regression predicts a number, such as a temperature, price, or test score.
Understanding AI & Machine Learning: Classification vs Regression
The main difference affects the shape of the answer a model produces. A classification model usually calculates a score for each possible group. For an email filter, it may give a high score for spam and a lower score for safe mail.
A rule called a threshold turns that score into a final decision. Changing the threshold changes the result. A strict spam filter catches more unwanted messages, but it may wrongly hide useful mail.
A regression model does not use category boundaries. It learns how the output tends to change as the input values change. Its prediction can fall anywhere within a useful numerical range.
Training is a cycle of prediction, comparison, and adjustment. Each training example contains input features plus a known result. The model makes a prediction, measures how far it was from the known result, then changes its internal settings slightly.
This process repeats across many examples. Good training data must represent the situation where the model will be used. A model trained only on sunny-day road images may perform badly in rain, snow, or darkness.
Students should know that a model can memorize training examples without learning a general pattern. This is called overfitting. Separate validation and test data help reveal it.
The way performance is measured depends on the task and its consequences. Accuracy is useful when each class matters equally. It can be misleading when one class is rare.
For example, if only one in one hundred transactions is fraudulent, a model that labels every transaction as normal can appear highly accurate. Precision tells how often positive alerts are correct. Recall tells how many real positive cases the model finds.
For regression, mean absolute error gives the average size of the mistakes. It is important to inspect individual errors too. A prediction that is wrong by a small amount is not equal in impact to one that is wrong by a large amount.
These tasks appear in ordinary tools. Classification helps phone cameras recognize faces, banks flag suspicious payments, and schools sort survey responses. Regression can estimate travel time, electricity use, crop yield, or the amount of stock needed by a shop.
In many real systems, the prediction is only one part of a larger decision. A doctor may use a risk score as evidence, not as a final diagnosis.
A delivery company may use a time estimate to plan routes, while allowing for traffic and weather. Predictions always contain uncertainty, even when the model gives a single answer.
When learning this topic, pay close attention to the target being predicted and the features available before that target is known. Data from the future must not slip into the inputs, because this creates leakage and gives unrealistically strong results. Correlation is not proof that one feature causes an outcome.
A model may learn unfair patterns if past data reflects unequal treatment of groups. Start with a simple baseline, such as guessing the most common class or using the average value. A more complex model is useful only when it improves on that baseline with new data.
Key Facts
- Classification predicts a class label, such as cat, dog, pass, or fail.
- Regression predicts a numerical value, such as y = 2x + 5.
- A model is trained by comparing predictions to known answers in a training data set.
- Classification accuracy = number of correct predictions / total number of predictions.
- Regression error can be measured with mean absolute error: MAE = sum of |actual - predicted| / n.
- The input features are the measurable clues the model uses, such as age, height, pixels, or study time.
Vocabulary
- Classification
- Classification is a machine learning task that predicts which category or group an input belongs to.
- Regression
- Regression is a machine learning task that predicts a continuous numerical value.
- Feature
- A feature is an input measurement or property that a model uses to make a prediction.
- Training Data
- Training data is a collection of examples with known answers used to teach a machine learning model.
- Model
- A model is the learned rule or pattern that connects input features to predicted outputs.
Common Mistakes to Avoid
- Calling every prediction classification is wrong because some predictions are numbers, not categories. If the answer can vary along a number line, it is usually regression.
- Treating labels like red, blue, and green as numbers is wrong when the numbers do not have real size or order. Category names should be handled as classes, not measurements.
- Judging a regression model only by whether it is exactly correct is wrong because numerical predictions are often close but not perfect. Use error measures like MAE to measure how far off the predictions are.
- Training and testing on the exact same examples is wrong because it can make the model look better than it really is. A separate test set checks whether the model works on new data.
Practice Questions
- 1 A classifier labels 50 emails as spam or not spam. It gets 42 correct. What is its accuracy as a fraction and as a percent?
- 2 A regression model predicts house prices of 210,000 dollars, 260,000 dollars, and 300,000 dollars. The actual prices are 200,000 dollars, 250,000 dollars, and 330,000 dollars. What is the mean absolute error?
- 3 Decide whether each task is classification or regression and explain why: predicting whether a plant is healthy or unhealthy, predicting tomorrow's high temperature, and predicting the type of animal in a photo.