A receiver operating characteristic curve, or ROC curve, shows how a binary classifier behaves as you change its decision threshold. It plots the True Positive Rate on the vertical axis against the False Positive Rate on the horizontal axis. This matters because many real classifiers output scores or probabilities, not just yes or no answers.
The ROC curve helps compare models across all possible threshold choices instead of judging them at only one cutoff.
As the threshold decreases, the classifier usually labels more cases as positive, which can increase both true positives and false positives. The Area Under the Curve, or AUC, condenses the ROC curve into one number between 0 and 1. A larger AUC means the classifier is generally better at ranking positive examples above negative examples.
In applications such as medical testing, fraud detection, and signal detection, ROC and AUC help balance sensitivity against false alarms.
Key Facts
- True Positive Rate = TP / (TP + FN)
- False Positive Rate = FP / (FP + TN)
- ROC curve plots True Positive Rate on the y-axis and False Positive Rate on the x-axis.
- AUC = area under the ROC curve, with 1.0 meaning perfect separation and 0.5 meaning random ranking.
- Changing the decision threshold moves the classifier to different points on the ROC curve.
- A classifier with a ROC curve closer to the top-left corner usually has better performance.
Vocabulary
- True Positive Rate
- The fraction of actual positive cases that the classifier correctly labels as positive.
- False Positive Rate
- The fraction of actual negative cases that the classifier incorrectly labels as positive.
- Decision Threshold
- The cutoff score used to decide whether a classifier output should be labeled positive or negative.
- ROC Curve
- A graph of True Positive Rate versus False Positive Rate over many decision thresholds.
- AUC
- The area under the ROC curve, used as a single-number summary of how well a classifier ranks positives above negatives.
Common Mistakes to Avoid
- Confusing False Positive Rate with false positives is wrong because the rate divides false positives by all actual negatives, FP / (FP + TN).
- Reading one point on the ROC curve as the whole model performance is wrong because each point represents only one threshold.
- Assuming higher AUC always means the best real-world model is wrong because costs, class imbalance, and required operating threshold can change the best choice.
- Interpreting AUC as accuracy is wrong because AUC measures ranking quality across thresholds, while accuracy measures correct classifications at one threshold.
Practice Questions
- 1 A classifier gives TP = 80, FN = 20, FP = 30, and TN = 70 at one threshold. Calculate the True Positive Rate and False Positive Rate.
- 2 At three thresholds, a model has ROC points (0, 0), (0.2, 0.7), and (1, 1). Estimate the AUC using trapezoids between the points.
- 3 Two classifiers have AUC values of 0.92 and 0.78, but the second classifier has a lower False Positive Rate at the operating threshold required by a hospital. Explain why the second classifier might still be chosen.