Precision, recall, and F1 score are statistics used to judge how well a classification model finds the cases we care about. They are especially important when the classes are imbalanced, such as rare diseases, fraud, or equipment failures. In these situations, accuracy can look high even when the model misses most of the important positive cases.
These metrics focus attention on different kinds of mistakes and their real costs.
A confusion matrix sorts predictions into true positives, false positives, true negatives, and false negatives. Precision asks how many predicted positives were actually positive, while recall asks how many actual positives were found. The F1 score combines precision and recall into one value using the harmonic mean, which strongly penalizes a model if either one is low.
Changing the decision threshold usually creates a tradeoff: raising recall often lowers precision, and raising precision often lowers recall.
Key Facts
- Precision = TP / (TP + FP)
- Recall = TP / (TP + FN)
- F1 score = 2PR / (P + R), where P is precision and R is recall
- Accuracy = (TP + TN) / (TP + FP + TN + FN)
- False positive means the model predicted positive, but the true class was negative.
- False negative means the model predicted negative, but the true class was positive.
Vocabulary
- Confusion matrix
- A table that counts correct and incorrect predictions for each actual class.
- Precision
- The fraction of positive predictions that are actually correct.
- Recall
- The fraction of actual positive cases that the model correctly finds.
- F1 score
- A single score that combines precision and recall using the harmonic mean.
- Decision threshold
- The cutoff score a model uses to decide whether to label a case as positive.
Common Mistakes to Avoid
- Using accuracy alone on imbalanced data is misleading because a model can get a high accuracy by mostly predicting the majority class.
- Confusing precision with recall is wrong because precision measures correctness among predicted positives, while recall measures coverage of actual positives.
- Ignoring false negatives is dangerous when missing a positive case has a high cost, such as failing to detect a disease.
- Averaging precision and recall with the ordinary mean is not the F1 score because F1 uses the harmonic mean, which penalizes an uneven balance.
Practice Questions
- 1 A model has TP = 40, FP = 10, TN = 900, and FN = 50. Calculate precision, recall, F1 score, and accuracy.
- 2 A spam filter makes 120 positive predictions, and 96 of them are truly spam. There are 150 spam emails total. Find the precision and recall.
- 3 For a medical screening test, explain whether you would prefer higher recall or higher precision, and describe the type of mistake you are trying hardest to avoid.