Machine learning models often make yes or no predictions, such as spam or not spam, sick or healthy, and pass or fail. Accuracy alone can hide important errors, especially when one class is much more common than the other. Precision, recall, and F1 score help us understand what kinds of mistakes a model is making.
These measures are especially important when false alarms or missed cases have real consequences.
A confusion matrix organizes predictions into four groups: true positives, false positives, true negatives, and false negatives. Precision asks how many predicted positives were actually correct, while recall asks how many real positives the model found. The F1 score combines precision and recall into one balanced score using the harmonic mean.
Together, these tools help students compare models and choose the best one for a specific goal.
Understanding AI & Machine Learning: Precision, Recall, and F1 Score
A model usually produces a score before it produces a final label. For a medical image, the score might represent how likely the image shows a disease. A decision threshold turns that score into a yes or no result.
If the threshold is low, the model marks more cases as positive. This can catch more real cases, but it creates more false alarms. If the threshold is high, the model gives fewer positive results.
Its positive predictions may be more trustworthy, but it can miss people who need attention. Precision and recall change when the threshold changes.
The better choice depends on the job. In airport security, missing a dangerous item can be far worse than checking an innocent bag. High recall is important because staff want to find as many real threats as possible.
In an email inbox, too many spam warnings can hide useful messages. Higher precision may matter more there. A hospital screening test may accept many false positives at first.
Patients with a positive result can receive a slower, more accurate follow-up test. The first model is used to find possible cases, not to make the final diagnosis.
The F1 score is useful when both kinds of performance matter and the positive cases are uncommon. It uses a harmonic mean, which gives a low result when either precision or recall is low. A model cannot get a strong F1 score by doing well on only one of them.
For example, a model that finds nearly every fraud case but wrongly flags thousands of normal purchases may have poor precision. Another model may flag only a few purchases and be correct each time, yet miss most fraud. Looking at the F1 score helps reveal that neither model is balanced enough for many tasks.
Students should inspect the actual error counts, not only the final scores. A precision value can look good when the model made very few positive predictions. Recall can look good when the test set contains only a small number of real positive examples.
The data used for testing must be separate from the data used for training. It should resemble the real situation where the model will be used. Labels need checking too.
If people disagree about whether an image contains a disease or whether a message is spam, the scores have limits. Comparing several thresholds and reading examples of mistakes gives a clearer picture than trusting one number.
Key Facts
- True positive, TP: the model predicts positive and the actual label is positive.
- False positive, FP: the model predicts positive but the actual label is negative.
- False negative, FN: the model predicts negative but the actual label is positive.
- Precision = TP / (TP + FP)
- Recall = TP / (TP + FN)
- F1 score = 2 × (precision × recall) / (precision + recall)
Vocabulary
- Confusion matrix
- A table that compares a model's predicted labels with the actual labels.
- Precision
- Precision is the fraction of positive predictions that were actually correct.
- Recall
- Recall is the fraction of actual positive cases that the model correctly found.
- False positive
- A false positive happens when a model incorrectly predicts that a negative case is positive.
- F1 score
- The F1 score is a single measure that balances precision and recall.
Common Mistakes to Avoid
- Using accuracy when the classes are imbalanced. This is wrong because a model can look accurate by mostly predicting the common class while missing the important rare cases.
- Mixing up precision and recall. Precision focuses on how trustworthy the positive predictions are, while recall focuses on how many actual positives were found.
- Forgetting false negatives in recall. This is wrong because recall measures missed positive cases, so FN must be included in the denominator.
- Assuming a higher F1 score is always the best choice. This can be wrong because some problems care more about precision, such as avoiding false accusations, while others care more about recall, such as finding disease cases.
Practice Questions
- 1 A model finds spam emails. It has TP = 40, FP = 10, TN = 90, and FN = 20. Calculate the precision, recall, and F1 score.
- 2 A medical screening model has TP = 72, FP = 18, and FN = 8. Calculate its precision and recall. Which score is higher?
- 3 Two models detect dangerous machine failures. Model A has high precision but low recall. Model B has lower precision but high recall. Explain which model might be safer to use and why.