পাঠ ৪৩ · ৪৫-এর মধ্যে · মডিউল ৫

Home / AI Courses / Machine Learning / Imbalanced

Imbalanced classes — SMOTE

Imbalanced data & resampling techniques

৬ মিনিট পড়া মাঝারি · Intermediate imbalanced-learn

এই পাঠে যা শিখবেন

Imbalanced data কেন কঠিন — accuracy ফাঁদ
Class weighting — algorithm-level fix
Random undersample/oversample — সরল কিন্তু lossy/leak-prone
SMOTE — synthetic minority oversampling
Pipeline-এ proper application — leak-প্রতিরোধী

১ · Imbalanced বাস্তবতা

Real-world classification-এ class balanced rare। উদাহরণ:

Credit card fraud — ০.১% transaction fraudulent।
Disease screening — ১% population affected।
Loan default — ৩-৫% borrower।
Spam — ১০-২০% (relatively balanced)।
Click-through — ১-৫%।

The accuracy trap: ০.১% fraud-এ "always non-fraud predict" — accuracy ৯৯.৯%। কিন্তু একটিও fraud catch হয়নি। Useless model।

Imbalance handling-এর তিন approach

১) Algorithm-level: class_weight, focal loss — model-কে minority importance বেশি।
২) Data-level: resampling — undersample majority, oversample minority।
৩) Hybrid: ensemble (BalancedRandomForest), SMOTE + ENN।
সাথে — proper metric (PR-AUC, F1, recall@FPR)।

২ · Class weighting

Loss function-এ minority class-এর weight বাড়ান। sklearn-এ class_weight="balanced":

$$w_c = \frac{n}{K \cdot n_c}$$

যেখানে $n$ = total sample, $K$ = number of class, $n_c$ = class $c$-এর sample। Inverse-frequency weighting। Logistic regression, SVM, tree — সবই support।

৩ · Random undersampling

Majority class থেকে random subset। ৯৯৯ majority → ১০০ random। Total ১০০ + ১ = ২০০।

Pros: simple, fast।
Cons: information loss — useful data discard।
Variants: Tomek links, NearMiss, ENN — informed undersample।

৪ · Random oversampling

Minority class duplicate। ১ minority → ৯৯৯ duplicate (with replacement)।

Pros: no info loss।
Cons: overfit prone — exact duplicate replicate।

৫ · SMOTE — Synthetic Minority Oversampling Technique

Chawla et al. (২০০২) — landmark paper। Idea: minority class-এর existing point-গুলোর মধ্যে interpolate করে synthetic example।

Algorithm:

Minority class-এর প্রতিটি sample $\mathbf{x}_i$।
$k$-nearest minority neighbor (default $k=5$)।
Random একটি neighbor $\mathbf{x}_j$ বাছাই।
Random $\lambda \in [0, 1]$।
Synthetic: $\mathbf{x}_{\text{new}} = \mathbf{x}_i + \lambda (\mathbf{x}_j - \mathbf{x}_i)$।
$\mathbf{x}_{\text{new}}$ minority class-এ যোগ।

Interpolation = "between two real points-এর মাঝে synthetic"। Oversampling-এর duplicate problem এড়ায়, কিন্তু realistic point।

৬ · SMOTE-এর variants

Borderline-SMOTE: শুধু borderline minority point-এ apply — confusing region।
ADASYN: harder-to-learn minority point-এ বেশি synthetic।
SMOTE-NC: categorical feature handle।
SMOTE + ENN: SMOTE তারপর Edited Nearest Neighbors clean।
SMOTE + Tomek: Tomek link clean borderline noise।

৭ · Proper metric

Imbalanced-এ accuracy meaningless। Use:

Precision: $\frac{TP}{TP + FP}$ — predicted positive কতগুলো সঠিক।
Recall (Sensitivity): $\frac{TP}{TP + FN}$ — actual positive কতগুলো ধরা।
F1: harmonic mean of precision-recall।
PR-AUC: precision-recall curve area — imbalanced-এ ROC-এর চেয়ে informative।
Recall@FPR: "5% false alarm-এ কত fraud catch" — domain-relevant।
Cost-sensitive: FN ও FP-এর actual business cost।

পাঁচ approach: original imbalance, class_weight (no data change), random under/oversample, SMOTE (synthetic interpolation)। SMOTE — duplicate-prone overfit এড়ায়।

৮ · imbalanced-learn — SMOTE pipeline

Python · imblearn

# pip install imbalanced-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline as ImbPipeline

# Heavily imbalanced 1:99
X, y = make_classification(n_samples=10000, weights=[0.99, 0.01],
                           n_features=20, random_state=0)
print(f"Class distribution: {np.bincount(y)}")

# 1) Baseline (no balancing)
base = LogisticRegression(max_iter=1000)
auc_base = cross_val_score(base, X, y, cv=5,
                           scoring="average_precision").mean()
print(f"Baseline PR-AUC: {auc_base:.3f}")

# 2) Class weight balanced
weighted = LogisticRegression(class_weight="balanced", max_iter=1000)
auc_w = cross_val_score(weighted, X, y, cv=5,
                        scoring="average_precision").mean()
print(f"Class-weighted PR-AUC: {auc_w:.3f}")

# 3) SMOTE in pipeline (proper — fit only on training fold)
smote_pipe = ImbPipeline([
    ("smote", SMOTE(random_state=0)),
    ("clf", LogisticRegression(max_iter=1000)),
])
auc_smote = cross_val_score(smote_pipe, X, y, cv=5,
                            scoring="average_precision").mean()
print(f"SMOTE pipeline PR-AUC: {auc_smote:.3f}")

Class weight ও SMOTE — দু'টিই baseline-এর চেয়ে PR-AUC বাড়ায়। imblearn-এর Pipeline (sklearn-এর extension) SMOTE only train fold-এ apply করে — leak-free।

৯ · কখন কোনটা

Class weight: first try — সবচেয়ে সরল। Tree, logistic, SVM — সবই support।
SMOTE: tabular numeric data, সামান্য imbalance (১-১০%)।
Random oversample: SMOTE possible না (text, image)।
Undersample: majority class noisy বা huge।
Ensemble: BalancedRandomForest, EasyEnsemble — robust।
Anomaly detection: extreme imbalance (<০.১%) → one-class SVM, Isolation Forest।

SMOTE high-dim ও non-Euclidean feature-এ poor। Categorical-এ SMOTE-NC, image-এ data augmentation, text-এ EDA/back-translation। SMOTE blindly apply — সবসময় সঠিক না। Class weight first try — usually 80% benefit।

ভাবনার প্রশ্ন

প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।

প্র ০১ SMOTE-এ data leak common — কেন? কীভাবে এড়ান?

ML practitioners-এর সবচেয়ে common SMOTE bug।

Bug pattern:

Full data SMOTE।
তারপর train/test split।
Test set-এ synthetic point থাকে।
Test point original-এর সাথে interpolate — info leak।

Result:

Inflated test accuracy।
Production-এ disappointing।

Correct way:

Split first।
SMOTE only on training fold।
Test untouched, original distribution।

imbalanced-learn Pipeline:

imblearn.pipeline.Pipeline — SMOTE step in CV।
Each fold-এ SMOTE auto-fit on training only।
Test prediction without SMOTE।

Why imblearn over sklearn:

sklearn Pipeline — SMOTE step apply test-এ as well (transform)।
imblearn — SMOTE skip during predict (correct behavior)।

Detection:

Train accuracy > 99%, validation 80% — suspect leak।
Performance drop in production।
Synthetic samples count check।

Other SMOTE pitfalls:

(১) Categorical:

SMOTE interpolation — categorical meaningless।
Use SMOTE-NC বা SMOTE-N।

(২) High-D:

k-NN-based — curse of dimensionality।
PCA first বা skip SMOTE।

(৩) Time series:

Synthetic time point invalid।
Skip SMOTE, use class weight।

(৪) Outlier:

Outlier interpolation — synthetic outlier amplify।
Pre-clean data।

(৫) Imbalance ratio:

1:1000 extreme — SMOTE limited।
Anomaly detection alternative।

Best practices:

imblearn Pipeline সবসময়।
Cross-validation দিয়ে compare baseline।
PR-AUC metric।
Held-out test set untouched।

Bangladesh fraud case:

0.1% fraud rate।
SMOTE pipeline + XGBoost + class_weight tuning।
PR-AUC 0.30 → 0.50 typical।
Production: precision@5% recall।

Modern alternatives:

Focal loss — neural network।
Self-paced learning।
Cost-sensitive learning।
Anomaly detection (Isolation Forest, One-Class SVM)।

মূল উপলব্ধি: SMOTE powerful কিন্তু error-prone। Pipeline + proper CV + held-out test — three-layer defense। Bangladesh fintech-এ — fraud detect মানেই SMOTE-leakage trap এড়ানোর engineering।

প্র ০২ PR-AUC vs ROC-AUC — কখন কোনটা better? Imbalanced-এ কেন PR?

চমৎকার metric question। দু'টিই common, কিন্তু imbalance-এ behavior ভিন্ন।

ROC-AUC:

Receiver Operating Characteristic curve area।
X: FPR, Y: TPR।
Baseline: 0.5 (random)।
Range 0-1।

PR-AUC (Average Precision):

Precision-Recall curve area।
X: Recall, Y: Precision।
Baseline: minority class proportion।
Range 0-1।

Imbalanced-এ ROC misleading:

FPR = FP / (FP + TN)।
TN huge (99% majority) → FPR small even with many FP।
ROC look great, precision low।

উদাহরণ:

10000 samples: 100 positive, 9900 negative।
Model: TP=80, FP=200, FN=20, TN=9700।
TPR (recall) = 80/100 = 0.80।
FPR = 200/9900 = 0.02 — very low।
ROC: high TPR, low FPR — looks good।
Precision: 80/(80+200) = 0.286 — actually poor!
PR-AUC reflect this; ROC misses।

When PR-AUC:

Imbalanced (<30% minority)।
Positive class focus (fraud, disease)।
FP cost high।
Domain — "alarm fatigue" matter।

When ROC-AUC:

Balanced classes।
Both class equally important।
Threshold-independent ranking।
Cross-method comparison common।

Other metrics:

F-beta: precision-recall tradeoff custom।
MCC (Matthews): imbalanced-friendly।
Cohen's kappa: chance-corrected accuracy।
Recall@FPR: domain-relevant operating point।
Precision@k: top-k recommendation।

Cost-sensitive:

Business cost FP vs FN ভিন্ন।
Custom loss — TP × benefit, FP × cost, FN × cost।
Production-এ গুরুত্বপূর্ণ।

Visualization:

ROC curve — overall ranking।
PR curve — recall vs precision tradeoff।
Calibration plot — probability quality।
Confusion matrix — operating point।

Bangladesh case:

Fraud — PR-AUC critical।
Disease screening — sensitivity (recall) primary, PR consider।
Spam — PR-AUC, F1।
Loan default — recall@FPR=10% (regulator interpretable)।

Modern reporting:

Both ROC ও PR present।
Multiple operating point।
Cost-sensitive analysis।
Subgroup performance — fairness।

মূল উপলব্ধি: Imbalanced-এ PR-AUC > ROC-AUC। Both ভাল reporting। Cost-sensitive analysis — production-এ paramount।

প্র ০৩ Class weight বনাম SMOTE — কোনটা ভাল empirically?

ML practitioner-এর recurring debate।

Class weight:

Algorithm-level (loss reweight)।
No data change।
Fast।
No leak risk।
Interpretable।

SMOTE:

Data-level (synthetic generation)।
Distribution change।
Interpolation creates new pattern।
Leak risk।
Categorical/text struggle।

Empirical findings:

Many studies — comparable performance।
Class weight often within 1-2% PR-AUC of SMOTE।
SMOTE marginally better with sufficient minority।
SMOTE worse with extreme imbalance (< 1%)।

When class weight wins:

High-D, sparse data।
Categorical heavy।
Small minority (< 100 sample)।
Well-defined metric।

When SMOTE wins:

Continuous numeric features।
Moderate imbalance (5-30%)।
Sufficient minority for k-NN (> 100)।
Distinct decision boundary।

Hybrid approaches:

SMOTE + class_weight after।
SMOTE + Tomek link cleanup।
Borderline-SMOTE।
Ensemble — multiple resampled training।

Decision tree-based:

XGBoost-এ scale_pos_weight।
BalancedRandomForest।
EasyEnsemble।
Often outperform SMOTE।

Modern alternatives:

Focal loss (Lin et al.) — neural network।
Self-paced learning।
Anomaly detection (extreme imbalance)।
Generative — GAN, VAE for synthetic data।

Practical workflow:

Baseline (no balancing) — establish floor।
class_weight="balanced" — quick win।
SMOTE pipeline — if data continuous।
Hyperparameter tune balancing strength।
Compare PR-AUC, recall@FPR।
Best in CV → held-out test।

Caveats:

Probability calibration — resampling distort।
Calibration curve check।
Platt scaling বা isotonic regression।

Bangladesh examples:

Credit fraud — XGBoost scale_pos_weight + SMOTE often top।
Disease screening — class weight + threshold tune।
Quality control — SMOTE rare defect।
Click prediction — sufficient minority data, balanced sampling।

Compute consideration:

SMOTE generates synthetic data — training slower।
Class weight no overhead।
Production scoring — same speed (no SMOTE at inference)।

মূল উপলব্ধি: Both works often similarly। Class weight first try; SMOTE specific advantage। Empirical comparison + domain knowledge — decisive। One-size-fits-all উত্তর নেই।

প্র ০৪ Bangladesh-এ একটি bank fraud detection-এ ০.১% fraud rate — full pipeline ডিজাইন।

Real-world critical application — Bangladesh fintech-এ।

Setup:

10M monthly transactions।
10K confirmed fraud (0.1%)।
Goal: real-time scoring।

(১) Data:

Transaction: amount, time, location, merchant।
User profile: age, history, account age।
Device: IP, fingerprint।
Velocity: transaction frequency।
Network: graph features।

(২) Feature engineering:

Time-based: hour-of-day, day-of-week।
Velocity: txn count last 1h, 24h।
Amount: deviation from user mean।
Geo: distance from home।
Merchant: risk score, category।

(৩) Imbalance handling:

XGBoost scale_pos_weight = 999।
SMOTE on numeric features (in pipeline)।
Categorical: target encoding (CV-aware)।
Tune both via Optuna।

(৪) Modeling:

XGBoost — primary।
Anomaly detection (Isolation Forest) — supplement।
Network analysis — graph fraud ring।
Ensemble।

(৫) Validation:

Time-based split (last month test)।
PR-AUC, recall@FPR=1% primary।
Cost analysis — TP benefit, FP cost (customer friction)।

(৬) Threshold tuning:

Operating point — block FPR < 0.1%।
Step-up auth — moderate threshold।
Allow — low risk।
Multi-tier action।

(৭) Production:

Real-time API < 100ms।
Feature store (Redis) — pre-computed velocity।
Streaming (Kafka) for txn ingestion।
ONNX Runtime — fast inference।

(৮) Monitoring:

Drift detection daily।
FPR drift — false alarm increase warn।
Recall lag (true label arrives later)।
Performance dashboard।

(৯) Feedback loop:

Manual review confirm fraud।
Customer dispute → label correction।
Continuous retraining।
Champion-challenger A/B।

(১০) Adversarial:

Fraudster adapt — concept drift।
Frequent retrain।
Pattern shift detection।
Rule + ML hybrid।

(১১) Compliance:

Bangladesh Bank guideline।
Customer notification।
Audit trail।
Explainability (SHAP) for blocked txn।

(১২) Customer experience:

False positive painful — wedding gift block disastrous।
Step-up auth instead of block।
SMS verification।
Dispute fast resolution।

(১৩) Stack:

Kafka → Flink (feature engineering) → ML scoring → action engine।
Postgres + Redis।
MLflow + Optuna।
Grafana monitoring।

(১৪) ROI:

Block ৭০% fraud → ৭ Cr BDT save monthly (typical Bangladesh bank)।
FP rate ১% — customer friction tolerable।
Implementation cost ১-২ Cr — payback months।

মূল উপলব্ধি: Imbalanced fraud detection — ১০% algorithm, ৯০% engineering + business + compliance। Bangladesh banking-এ এটা mature emerging area। Quality fraud team competitive moat।

অনুশীলন

হিসাব করুন: 1000 sample, 10 positive (1%)। "Always negative" predictor — accuracy, precision, recall?
- Accuracy = 990/1000 = 99%।
- Precision = 0/0 = undefined (no positive predicted)।
- Recall = 0/10 = 0%।
- F1 = 0।
- "Accuracy 99%" trap — model useless।

imblearn-এ চেষ্টা: SMOTE pipeline + cross-validation।

from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline as ImbPipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

pipe = ImbPipeline([
    ("smote", SMOTE(random_state=0)),
    ("clf", LogisticRegression(max_iter=1000)),
])
scores = cross_val_score(pipe, X, y, cv=5, scoring="average_precision")
print(f"PR-AUC: {scores.mean():.3f} ± {scores.std():.3f}")

ভাবুন: Bangladesh-এর rural disease screening (TB) — 0.5% prevalence। Imbalance handling decision।
- Cost: miss TB severe (FN) > FP (further test)।
- Metric: recall (sensitivity) primary, precision secondary।
- Class weight: miss-cost weighted।
- SMOTE: on continuous test result, not categorical symptom।
- Threshold: low — high recall।
- Validation: regional split।
- Action: positive → confirmatory test (X-ray, sputum)।
- Edge: false negative consequence — community spread।