Concept drift
এই পাঠে যা শিখবেন
- Concept drift vs covariate drift
- ৪ types — sudden, gradual, recurring, incremental
- Detection algorithms — DDM, EDDM, ADWIN
- Adversarial drift — fraud detection context
১ · Concept drift কী
Same input, different correct answer.
- Spam: "free money" 2010 = spam; "free shipping" 2025 = legitimate offer।
- Fraud: ATM withdrawal pattern that was fraud 2020 — now common COVID-pattern।
- Recommendation: "Bangla movie" 2018 vs 2025 — taste evolution।
Difference from covariate drift:
- Covariate: $P(X)$ changes; output stays correct mostly।
- Concept: $P(y|X)$ changes; same X → different y over time।
২ · ৪ Types
Sudden: COVID lockdown — abrupt regime change।
Gradual: taste slowly shifts; old + new concept coexist।
Recurring: Eid sale every year — pattern returns।
Incremental: small continuous shift; like compound interest।
৩ · Detection algorithms
- DDM (Drift Detection Method): error rate increase tracker। 2 thresholds: warning, drift। Simple।
- EDDM (Early DDM): distance between consecutive errors। Detect gradual better।
- ADWIN (Adaptive Windowing): sliding window; auto-adjusts size। Statistical guarantee।
- Page-Hinkley: CUSUM-based change point detection।
৪ · DDM Python sketch
import numpy as np
class DDM:
"""Drift Detection Method (Gama et al. 2004)."""
def __init__(self, warning_thr=2.0, drift_thr=3.0):
self.warning_thr = warning_thr
self.drift_thr = drift_thr
self.reset()
def reset(self):
self.n = 0
self.p_min = float("inf")
self.s_min = float("inf")
self.p = 0.0 # error rate
self.s = 0.0 # std
def update(self, error: int):
"""error: 0 or 1 — was prediction wrong?"""
self.n += 1
self.p = self.p + (error - self.p) / self.n
self.s = np.sqrt(self.p * (1 - self.p) / self.n)
if self.p + self.s < self.p_min + self.s_min:
self.p_min = self.p
self.s_min = self.s
if self.p + self.s > self.p_min + self.drift_thr * self.s_min:
return "drift"
elif self.p + self.s > self.p_min + self.warning_thr * self.s_min:
return "warning"
return "stable"
# Usage — stream of (label, prediction)
ddm = DDM()
for label, pred in stream:
error = int(label != pred)
state = ddm.update(error)
if state == "drift":
print(f"⚠️ Drift detected at sample {ddm.n}")
# trigger retrain workflow
ddm.reset()river library ready DDM/EDDM/ADWIN provide।
৫ · River library
Online ML library Python-এ — concept drift detector built-in।
from river import drift
import random
adwin = drift.ADWIN()
# Simulate stream
for i in range(2000):
if i < 1000:
x = random.gauss(0, 1)
else:
x = random.gauss(2, 1) # concept change!
in_drift, _ = adwin.update(x)
if in_drift:
print(f"⚠️ ADWIN detected drift at {i}")pip install river। Multiple detector available: ADWIN, DDM, EDDM, KSWIN, Page-Hinkley।
৬ · Response strategies
- Retrain on recent data: simple, common।
- Online learning: incremental update — River's models।
- Ensemble: "champion-challenger" pool, swap on drift।
- Periodic retrain: ignore detection, scheduled retraining handle drift।
৭ · Adversarial drift
Fraudster, attacker — concept drift fastest। They observe model, evolve।
- Fraud pattern: model catches → fraudster shifts → model lags।
- Recommendation gaming: SEO spam evolves।
- Phishing: language adaptation।
Strategies:
- Frequent retraining (weekly, daily)।
- Diverse signal (single feature gameable; multi-signal harder)।
- Anomaly detection layer atop classification।
- Human-in-loop labeling for rapid feedback।
৮ · COVID drift case study
২০২০-এর COVID — many ML model worldwide broke।
- E-commerce — sudden shift to online; recommendation training pre-COVID irrelevant।
- Fraud detection — work-from-home pattern; legitimate looked fraud-like।
- Demand forecasting — broken; historical pattern useless।
Lesson: scheduled retraining alone insufficient। Drift detection + emergency retrain capability essential।
ভাবনার প্রশ্ন
প্র ০১"Drift vs noise — distinguish কীভাবে?"
Random fluctuation drift নয়; persistent pattern drift।
Statistical tests:
- Window comparison statistical significance।
- Multiple consecutive windows confirm।
- Magnitude vs noise floor।
Time-based confirmation:
- Day-1 alert → wait day-2। Both drift = real।
- Single day alert + day-2 stable = noise।
Cross-validation:
- Multiple metrics agree (accuracy + drift + business KPI)।
- Single metric drift suspicious।
BD context:
- Friday spike not drift if recurring।
- Sudden Friday change after years stability — drift।
মূল উপলব্ধি: Drift = pattern; noise = random। Persistence + significance + multi-signal corroboration → confidence।
প্র ০২"Retraining cadence — drift-triggered vs scheduled?"
দু'টি approach trade-off।
Scheduled (e.g., weekly):
- Pros: predictable, planning easy।
- Cons: drift between cycles unaddressed।
Drift-triggered:
- Pros: responsive, just-in-time।
- Cons: unpredictable load, complexity।
Hybrid (most common):
- Weekly base retraining।
- Drift detect → emergency retrain trigger।
- Best of both।
Champion-challenger:
- Continuous training, periodically promote if better।
- Drift = challenger wins more often।
Cost considerations:
- Training cost — frequent → expensive।
- Spot instance + checkpoint — cheap retraining।
BD context — fraud:
- Daily retraining standard for serious fraud system।
- Adversarial pace dictate।
মূল উপলব্ধি: Hybrid practical. Scheduled foundation + drift trigger emergency। Pure scheduled = stale; pure drift-triggered = chaotic।
প্র ০৩"Champion-challenger setup কীভাবে practical?"
Champion-challenger = continuously train new model, periodically compare।
Setup:
- Champion = current production model।
- Challenger = new model trained recent data।
- Both score same data; performance compare।
- Challenger wins → promote।
Benefits:
- Always model in pipeline ready।
- Drift = challenger naturally outperforms।
- Smooth transition no sudden retraining।
Implementation:
- Daily training pipeline (challenger)।
- Shadow deployment (challenger receive request copies)।
- Weekly comparison report।
- Challenger wins by margin → A/B → promote।
Cost:
- 2× training compute (champion + challenger)।
- 2× shadow inference।
- Justified for critical models।
BD context — bKash:
- Fraud champion-challenger common pattern।
- Daily challenger trained latest week's data।
- Gradual promote — trust earned through data।
মূল উপলব্ধি: Champion-challenger = drift-resilient framework। Costs more but stability dramatic। Critical model justify।
প্র ০৪"COVID-era ML failures — কী শিখলাম?"
2020 — global ML failure event।
What broke:
- Demand forecasting — supply chain models gave nonsense।
- Fraud detection — false positive spike (work-from-home)।
- Recommendation — taste shift। sudden।
- Credit scoring — unemployment shift, "good signal" change।
- Pricing models — price-elasticity broke।
Lessons:
(১) Scheduled retraining insufficient:
- Weekly retrain — ৪ সপ্তাহ data outdated already।
- Need: drift-triggered + manual override।
(২) Multiple model layers:
- Single ML model — single point of failure।
- Ensemble + rule-based fallback resilient।
(৩) Human-in-loop:
- "Trust the model" without check broke।
- Anomaly review queue critical।
(৪) Domain-specific concept anchors:
- "What can never change" identify।
- Constraints baked into model।
(৫) Backtest with hypothetical shocks:
- "What if user behavior shifts X%" simulation।
- Robustness measure।
BD-specific:
- 2020 lockdown e-commerce surge — recommendation models retrained twice in month।
- Fintech credit scoring — temporary rule overrides।
মূল উপলব্ধি: COVID exposed ML system fragility। Routine practice insufficient — emergency capability mandatory। Hybrid (rule + ML), human review, multi-source data — resilience।
অনুশীলন
- Implement DDM: Stream of (label, prediction) generate; DDM apply।
উপরের code use। Inject concept drift midway, observe alert।
- River: ADWIN ব্যবহার করুন; sudden vs gradual data-এ behavior compare।
Sudden faster detection; gradual slower lag.
- চিন্তা: bKash adversarial fraud — concept drift response system design।
- Daily retraining pipeline।
- Champion-challenger A/B।
- Anomaly detection layer (autoencoder)।
- Manual review queue for borderline।
- Alert: drift > threshold → fraud team notified।
- Rapid feedback (1-week chargeback signal)।