← Back

ROC & Precision-Recall for Busy People

·Bryan Lai
TechnicalMLEvals

ROC & Precision-Recall for Busy People

Want to judge your classifier?

Use ROC for balanced data. Use Precision-Recall for imbalanced data.

ROC

ROC tells you how well the model separates positives from negatives.

The two numbers:

  1. TPR / Recall: Out of the real positives, how many did you catch?
  2. FPR: Out of the real negatives, how many did you falsely flag?
Good ROC Curve hugs top left, Bad ROC Curve is diagonal

The ROC curve plots TPR against FPR.

A good curve hugs the top left.

AUC turns the curve into one number:

  1. 1.0: perfect.
  2. 0.5: coin flip.
  3. Below 0.5: worse than guessing.

If AUC is 0.8, the model has an 80% chance of ranking a random positive above a random negative.

ROC is useful when the classes are fairly balanced.

Precision-Recall

Precision-Recall focuses on the positive class.

Use it when positives are rare.

The two numbers:

  1. Precision: Out of the things you flagged positive, how many were really positive?
  2. Recall: Out of the real positives, how many did you catch?
Precision Recall Curve

The Precision-Recall curve plots precision against recall.

A good curve hugs the top right.

AUC-PR turns the curve into one number.

Higher is better, but there is no universal 0.5 baseline. The baseline depends on how rare the positive class is.

Cheat Sheet

FeatureROC/AUCPrecision-Recall/AUC-PR
BEST FOR:Balanced Data. General Model Check.Imbalanced Data. Positive Class Performance Focus.
MAIN FOCUS:TPR vs. FPR trade-off (all settings).Precision vs. Recall trade-off.
KEY SCORE:AUC-ROCAUC-PR
"RANDOM" SCORE:AUC-ROC = 0.5AUC-PR: Baseline depends on positive class rarity.
THINK ABOUT:Overall class separation.Performance on the positive class, especially if rare.

Spam Analogy

ROC asks whether the spam filter separates spam from real email.

Precision-Recall asks whether the filter catches spam without eating good email.

Balance still matters. Do not over-optimize precision or recall alone.