Precision · Recall · F1
Press Next → or use ← → arrow keys
Why Accuracy Alone Is Dangerous
A trivial model that predicts "healthy" for everyone hits 95% accuracy — and misses every single TB case. Accuracy treats all correct answers equally, ignoring which class was right. On imbalanced data, it is a mirror that tells only pleasant lies.
A false alarm means one extra test and some anxiety. A missed case means untreated disease and possibly contagion. Any single number that treats them equally is unfit for the decision. We need Precision and Recall.
The Confusion Matrix — Foundation of Everything
In disease detection, fraud, and safety systems, a missed threat is far worse than a false alarm. Every metric that follows — precision, recall, F1, F-beta — is just a different way of asking: "Which type of error do I punish more?"
Precision — Of My Alerts, How Many Were Real?
Recall — Of Actual Cases, How Many Did I Catch?
The Precision–Recall Trade-off
Lower the threshold → catch more true positives (recall ↑) but flag more false alarms (precision ↓). Raise it → cleaner alerts (precision ↑) but miss more real cases (recall ↓). F1 peaks somewhere in the middle — but where is a domain decision, not a maths one.
F1 — Why The Harmonic Mean Beats The Arithmetic
F-Beta — Tilt The Balance To Your Domain
"Missing a TB case is four times as costly as a false alarm." That domain judgement translates directly into β = 2, and her primary metric becomes F2 = 0.612 — properly rewarding her model's 90% recall while still accounting for the flood of false alarms.
Dr. Meera's Model — Every Metric At A Glance
| Metric | Formula | Value | Reading |
|---|---|---|---|
| Accuracy | (TP+TN)/N | 89.5% | Misleading on imbalance |
| Precision | TP/(TP+FP) | 31.0% | 7 of 10 alerts are false |
| Recall | TP/(TP+FN) | 90.0% | Catches 9 of 10 TB cases |
| Specificity | TN/(TN+FP) | 89.5% | Clears 9 of 10 healthy |
| F1 | 2PR/(P+R) | 46.1% | Pulled down by precision |
| F2 | 5PR/(4P+R) | 61.2% | Rewards high recall |
| Balanced Acc | (Recall+Spec)/2 | 89.75% | Class-balanced view |
| MCC | see slide 10 | 0.491 | Most robust single score |
No single metric captures classifier quality. Report a slate: accuracy for orientation, precision + recall for the trade-off, F-beta for a balanced summary aligned to cost, and MCC for the imbalance-robust single number.
Advanced Metrics — MCC, Kappa & Balanced Accuracy
Precision/Recall/F1 each ignore the TN cell. MCC uses all four — that's why it's the most trustworthy summary when classes are skewed.
Multi-Class — Macro vs Micro vs Weighted
| Class | Support | Precision | Recall | F1 |
|---|---|---|---|---|
| Benign | 600 | 0.94 | 0.96 | 0.95 |
| Type-A | 300 | 0.82 | 0.78 | 0.80 |
| Type-B | 100 | 0.61 | 0.55 | 0.58 |
Divergence between Macro (0.78) and Weighted (0.87) is a red flag: a minority class is failing silently. Always report the per-class table alongside averages — it's the only view that catches this.
Implementation — sklearn One-Liner To Full Report
from sklearn.metrics import ( classification_report, precision_score, recall_score, f1_score, fbeta_score, matthews_corrcoef, cohen_kappa_score, balanced_accuracy_score, precision_recall_fscore_support, ) # ── One line, full slate ───────────────────────── print(classification_report(y_test, y_pred, target_names=['Healthy', 'TB'])) # ── Individual metrics ─────────────────────────── p = precision_score(y_test, y_pred) r = recall_score(y_test, y_pred) f1 = f1_score(y_test, y_pred) f2 = fbeta_score(y_test, y_pred, beta=2) # recall-heavy mcc = matthews_corrcoef(y_test, y_pred) # best single number # ── Threshold tuning — trade P for R ───────────── y_proba = model.predict_proba(X_test)[:, 1] y_pred_low = (y_proba >= 0.3).astype(int) # lower τ → higher recall # ── Multi-class averaging strategies ───────────── for avg in ['macro', 'weighted', 'micro']: p, r, f, _ = precision_recall_fscore_support(y_test, y_pred, average=avg) print(f"{avg}: P={p:.3f} R={r:.3f} F1={f:.3f}")
The default 0.5 threshold rarely maximises what you actually care about. Sweep τ on a validation split to find the best operating point, then evaluate exactly once on the pristine test set.
When To Use Which Metric
Data science can plot every curve. But whether it is better to miss 5 TB patients or send 100 healthy people for extra tests is a medical ethics decision. Get the domain expert's judgement first — then pick the metric that encodes it.
Common Pitfalls — What Silently Breaks Reports
Golden Rules — Six Habits That Save Reports
Precision · Recall · F1 — Honest Numbers For Real Decisions
You now understand the confusion matrix, the trade-off, F1's honest harmonic mean, F-beta's tunable bias, and the imbalance-robust MCC. Every classifier evaluation you do — from spam to sepsis — rests on the ideas you just learned.
Study the Precision–Recall curve & PR-AUC, then ROC-AUC, then cost-sensitive learning. Practice on Kaggle's Credit Card Fraud or Pima Indians Diabetes — both properly imbalanced, both perfect for exercising every metric here.
🎯 End of tutorial · Press ← to review, or click Restart