পাঠ ২৬ · ৩৩-এর মধ্যে · মডিউল ৪

Home / AI Courses / MLOps / Concept drift

Concept drift

Concept drift — when relationships change

৬ মিনিট পড়া উচ্চ · Advanced Concept

এই পাঠে যা শিখবেন

Concept drift vs covariate drift
৪ types — sudden, gradual, recurring, incremental
Detection algorithms — DDM, EDDM, ADWIN
Adversarial drift — fraud detection context

১ · Concept drift কী

Same input, different correct answer.

Spam: "free money" 2010 = spam; "free shipping" 2025 = legitimate offer.
Fraud: ATM withdrawal pattern that was fraud 2020 — now common COVID-pattern.
Recommendation: "Bangla movie" 2018 vs 2025 — taste evolution.

Difference from covariate drift:

Covariate: $P(X)$ changes; output stays correct mostly.
Concept: $P(y|X)$ changes; same X → different y over time.

২ · ৪ Types

Concept drift types

Sudden: COVID lockdown — abrupt regime change.
Gradual: taste slowly shifts; old + new concept coexist.
Recurring: Eid sale every year — pattern returns.
Incremental: small continuous shift; like compound interest.

৩ · Detection algorithms

DDM (Drift Detection Method): error rate increase tracker. 2 thresholds: warning, drift. Simple.
EDDM (Early DDM): distance between consecutive errors. Detect gradual better.
ADWIN (Adaptive Windowing): sliding window; auto-adjusts size. Statistical guarantee.
Page-Hinkley: CUSUM-based change point detection.

৪ · DDM Python sketch

Python · Simple DDM

import numpy as np

class DDM:
    """Drift Detection Method (Gama et al. 2004)."""
    def __init__(self, warning_thr=2.0, drift_thr=3.0):
        self.warning_thr = warning_thr
        self.drift_thr = drift_thr
        self.reset()

    def reset(self):
        self.n = 0
        self.p_min = float("inf")
        self.s_min = float("inf")
        self.p = 0.0  # error rate
        self.s = 0.0  # std

    def update(self, error: int):
        """error: 0 or 1 — was prediction wrong?"""
        self.n += 1
        self.p = self.p + (error - self.p) / self.n
        self.s = np.sqrt(self.p * (1 - self.p) / self.n)

        if self.p + self.s < self.p_min + self.s_min:
            self.p_min = self.p
            self.s_min = self.s

        if self.p + self.s > self.p_min + self.drift_thr * self.s_min:
            return "drift"
        elif self.p + self.s > self.p_min + self.warning_thr * self.s_min:
            return "warning"
        return "stable"


# Usage — stream of (label, prediction)
ddm = DDM()
for label, pred in stream:
    error = int(label != pred)
    state = ddm.update(error)
    if state == "drift":
        print(f"⚠️ Drift detected at sample {ddm.n}")
        # trigger retrain workflow
        ddm.reset()

DDM tracks error rate; statistical thresholds-এ "warning" বা "drift" raise. Production-এ river library ready DDM/EDDM/ADWIN provide.

৫ · River library

Online ML library Python-এ — concept drift detector built-in.

Python · River drift detector

from river import drift
import random

adwin = drift.ADWIN()

# Simulate stream
for i in range(2000):
    if i < 1000:
        x = random.gauss(0, 1)
    else:
        x = random.gauss(2, 1)  # concept change!
    in_drift, _ = adwin.update(x)
    if in_drift:
        print(f"⚠️ ADWIN detected drift at {i}")

pip install river. Multiple detector available: ADWIN, DDM, EDDM, KSWIN, Page-Hinkley.

৬ · Response strategies

Retrain on recent data: simple, common.
Online learning: incremental update — River's models.
Ensemble: "champion-challenger" pool, swap on drift.
Periodic retrain: ignore detection, scheduled retraining handle drift.

৭ · Adversarial drift

Fraudster, attacker — concept drift fastest. They observe model, evolve.

Fraud pattern: model catches → fraudster shifts → model lags.
Recommendation gaming: SEO spam evolves.
Phishing: language adaptation.

Strategies:

Frequent retraining (weekly, daily).
Diverse signal (single feature gameable; multi-signal harder).
Anomaly detection layer atop classification.
Human-in-loop labeling for rapid feedback.

৮ · COVID drift case study

২০২০-এর COVID — many ML model worldwide broke.

E-commerce — sudden shift to online; recommendation training pre-COVID irrelevant.
Fraud detection — work-from-home pattern; legitimate looked fraud-like.
Demand forecasting — broken; historical pattern useless.

Lesson: scheduled retraining alone insufficient. Drift detection + emergency retrain capability essential.

Concept drift-এর ৪ pattern — different detection strategies + response.

Bangladesh fraud teams concept drift fastest experience. Adversarial → daily retrain + anomaly layer + human review combo standard.

ভাবনার প্রশ্ন

প্র ০১"Drift vs noise — distinguish কীভাবে?"

Random fluctuation drift নয়; persistent pattern drift.

Statistical tests:

Window comparison statistical significance.
Multiple consecutive windows confirm.
Magnitude vs noise floor.

Time-based confirmation:

Day-1 alert → wait day-2. Both drift = real.
Single day alert + day-2 stable = noise.

Cross-validation:

Multiple metrics agree (accuracy + drift + business KPI).
Single metric drift suspicious.

BD context:

Friday spike not drift if recurring.
Sudden Friday change after years stability — drift.

মূল উপলব্ধি: Drift = pattern; noise = random. Persistence + significance + multi-signal corroboration → confidence.

প্র ০২"Retraining cadence — drift-triggered vs scheduled?"

দু'টি approach trade-off.

Scheduled (e.g., weekly):

Pros: predictable, planning easy.
Cons: drift between cycles unaddressed.

Drift-triggered:

Pros: responsive, just-in-time.
Cons: unpredictable load, complexity.

Hybrid (most common):

Weekly base retraining.
Drift detect → emergency retrain trigger.
Best of both.

Champion-challenger:

Continuous training, periodically promote if better.
Drift = challenger wins more often.

Cost considerations:

Training cost — frequent → expensive.
Spot instance + checkpoint — cheap retraining.

BD context — fraud:

Daily retraining standard for serious fraud system.
Adversarial pace dictate.

মূল উপলব্ধি: Hybrid practical. Scheduled foundation + drift trigger emergency. Pure scheduled = stale; pure drift-triggered = chaotic.

প্র ০৩"Champion-challenger setup কীভাবে practical?"

Champion-challenger = continuously train new model, periodically compare.

Setup:

Champion = current production model.
Challenger = new model trained recent data.
Both score same data; performance compare.
Challenger wins → promote.

Benefits:

Always model in pipeline ready.
Drift = challenger naturally outperforms.
Smooth transition no sudden retraining.

Implementation:

Daily training pipeline (challenger).
Shadow deployment (challenger receive request copies).
Weekly comparison report.
Challenger wins by margin → A/B → promote.

Cost:

2× training compute (champion + challenger).
2× shadow inference.
Justified for critical models.

BD context — bKash:

Fraud champion-challenger common pattern.
Daily challenger trained latest week's data.
Gradual promote — trust earned through data.

মূল উপলব্ধি: Champion-challenger = drift-resilient framework. Costs more but stability dramatic. Critical model justify.

প্র ০৪"COVID-era ML failures — কী শিখলাম?"

2020 — global ML failure event.

What broke:

Demand forecasting — supply chain models gave nonsense.
Fraud detection — false positive spike (work-from-home).
Recommendation — taste shift. sudden.
Credit scoring — unemployment shift, "good signal" change.
Pricing models — price-elasticity broke.

Lessons:

(১) Scheduled retraining insufficient:

Weekly retrain — ৪ সপ্তাহ data outdated already.
Need: drift-triggered + manual override.

(২) Multiple model layers:

Single ML model — single point of failure.
Ensemble + rule-based fallback resilient.

(৩) Human-in-loop:

"Trust the model" without check broke.
Anomaly review queue critical.

(৪) Domain-specific concept anchors:

"What can never change" identify.
Constraints baked into model.

(৫) Backtest with hypothetical shocks:

"What if user behavior shifts X%" simulation.
Robustness measure.

BD-specific:

2020 lockdown e-commerce surge — recommendation models retrained twice in month.
Fintech credit scoring — temporary rule overrides.

মূল উপলব্ধি: COVID exposed ML system fragility. Routine practice insufficient — emergency capability mandatory. Hybrid (rule + ML), human review, multi-source data — resilience.

অনুশীলন

Implement DDM: Stream of (label, prediction) generate; DDM apply.
উপরের code use. Inject concept drift midway, observe alert.
River: ADWIN ব্যবহার করুন; sudden vs gradual data-এ behavior compare.
Sudden faster detection; gradual slower lag.
চিন্তা: bKash adversarial fraud — concept drift response system design.
- Daily retraining pipeline.
- Champion-challenger A/B.
- Anomaly detection layer (autoencoder).
- Manual review queue for borderline.
- Alert: drift > threshold → fraud team notified.
- Rapid feedback (1-week chargeback signal).