A confusion matrix is a compact table that shows how a classification model's predictions compare with the true labels. It is especially useful when a model must sort cases into two groups, such as positive and negative, pass and fail, or disease and no disease. Instead of giving only one accuracy score, the matrix shows the different ways a model can be right or wrong.
This matters because different mistakes can have very different costs in medicine, security, finance, and science.
Understanding Statistics: The Confusion Matrix
To build the table, each case needs two labels. One label comes from reality, such as whether a message really is spam. The other comes from the model.
Every case is placed in one of four boxes. A true positive is a correctly detected target case. A true negative is a correctly rejected non-target case.
A false positive is a false alarm. A false negative is a missed target case. The words positive and negative do not mean good or bad.
They simply name the class the model is trying to find. Different books may put actual labels in rows or columns, so always read the headings before interpreting a cell.
Many classification models first produce a score rather than a final yes or no decision. A spam filter might give each email a score between low risk and high risk. A chosen cutoff turns that score into a predicted class.
Lowering the cutoff catches more real spam, though it can send more normal emails to the spam folder. Raising the cutoff reduces false alarms, though more spam may get through. This is a tradeoff, not a mistake in the mathematics.
The best cutoff depends on the situation and on the harm caused by each type of error. A cancer screening test often aims to miss as few cases as possible. A system that blocks bank accounts needs to avoid wrongly blocking legitimate customers.
Accuracy can hide a serious problem when one class is rare. Imagine one thousand products, with only ten defective ones. A model that labels every product as not defective gets nine hundred ninety predictions right, yet it finds no defects.
This is why students need to examine the separate boxes, not just one overall score. Precision focuses on whether flagged cases really belong to the target group. Recall focuses on whether the real target cases were found.
Specificity focuses on correctly leaving non-target cases unflagged. The F1 score combines precision with recall, but it does not show every concern. For example, it gives less direct attention to true negatives, which may matter greatly in some tasks.
A useful confusion matrix comes from data the model did not use for learning. Testing on training data can make a model look much better than it will perform on new cases. The test set should resemble the real population as closely as possible.
Small test sets can give unstable results, especially when positive cases are uncommon. Looking at individual false positives and false negatives is often more informative than only comparing scores.
Errors may reveal unclear labels, missing information, biased examples, or a cutoff that does not fit the real goal. When studying these tables, state which class counts as positive, check the row and column directions, and connect each error type to a real consequence.
Key Facts
- Total cases = TP + TN + FP + FN
- Accuracy = (TP + TN) / (TP + TN + FP + FN)
- Precision = TP / (TP + FP)
- Recall = Sensitivity = TP / (TP + FN)
- Specificity = TN / (TN + FP)
- F1 score = 2TP / (2TP + FP + FN)
Vocabulary
- Confusion matrix
- A table that counts correct and incorrect predictions for each actual class in a classification problem.
- True positive
- A case that is actually positive and is predicted positive by the model.
- False positive
- A case that is actually negative but is incorrectly predicted positive by the model.
- False negative
- A case that is actually positive but is incorrectly predicted negative by the model.
- Precision
- The fraction of predicted positive cases that are actually positive.
Common Mistakes to Avoid
- Swapping false positives and false negatives: a false positive is predicted positive when the actual class is negative, while a false negative is predicted negative when the actual class is positive.
- Reading the axes backward: always check which direction shows actual class and which direction shows predicted class before identifying TP, TN, FP, and FN.
- Using accuracy alone on imbalanced data: high accuracy can hide poor performance on the smaller or more important class.
- Treating precision and recall as the same measure: precision asks how many predicted positives were correct, while recall asks how many actual positives were found.
Practice Questions
- 1 A test has TP = 40, TN = 50, FP = 10, and FN = 20. Find the total number of cases, accuracy, precision, and recall.
- 2 A classifier checks 200 emails for spam. It has 70 true positives, 100 true negatives, 20 false positives, and 10 false negatives. Calculate specificity and F1 score.
- 3 In a disease screening test, explain whether a false positive or a false negative is usually more dangerous, and justify your answer using the meaning of each error.