Machine learning models often make yes or no predictions, such as spam or not spam, sick or healthy, and pass or fail. Accuracy alone can hide important errors, especially when one class is much more common than the other. Precision, recall, and F1 score help us understand what kinds of mistakes a model is making.
These measures are especially important when false alarms or missed cases have real consequences.
A confusion matrix organizes predictions into four groups: true positives, false positives, true negatives, and false negatives. Precision asks how many predicted positives were actually correct, while recall asks how many real positives the model found. The F1 score combines precision and recall into one balanced score using the harmonic mean.
Together, these tools help students compare models and choose the best one for a specific goal.
Key Facts
- True positive, TP: the model predicts positive and the actual label is positive.
- False positive, FP: the model predicts positive but the actual label is negative.
- False negative, FN: the model predicts negative but the actual label is positive.
- Precision = TP / (TP + FP)
- Recall = TP / (TP + FN)
- F1 score = 2 × (precision × recall) / (precision + recall)
Vocabulary
- Confusion matrix
- A table that compares a model's predicted labels with the actual labels.
- Precision
- Precision is the fraction of positive predictions that were actually correct.
- Recall
- Recall is the fraction of actual positive cases that the model correctly found.
- False positive
- A false positive happens when a model incorrectly predicts that a negative case is positive.
- F1 score
- The F1 score is a single measure that balances precision and recall.
Common Mistakes to Avoid
- Using accuracy when the classes are imbalanced. This is wrong because a model can look accurate by mostly predicting the common class while missing the important rare cases.
- Mixing up precision and recall. Precision focuses on how trustworthy the positive predictions are, while recall focuses on how many actual positives were found.
- Forgetting false negatives in recall. This is wrong because recall measures missed positive cases, so FN must be included in the denominator.
- Assuming a higher F1 score is always the best choice. This can be wrong because some problems care more about precision, such as avoiding false accusations, while others care more about recall, such as finding disease cases.
Practice Questions
- 1 A model finds spam emails. It has TP = 40, FP = 10, TN = 90, and FN = 20. Calculate the precision, recall, and F1 score.
- 2 A medical screening model has TP = 72, FP = 18, and FN = 8. Calculate its precision and recall. Which score is higher?
- 3 Two models detect dangerous machine failures. Model A has high precision but low recall. Model B has lower precision but high recall. Explain which model might be safer to use and why.