পাঠ ৪৪ · ৪৫-এর মধ্যে · মডিউল ৫

Home / AI Courses / Machine Learning / Churn Project

প্রজেক্ট: Customer churn prediction

End-to-end project — telecom churn (Bangladesh context)

১৫ মিনিট পড়া মাঝারি · Intermediate sklearn / XGBoost / FastAPI

এই পাঠে যা শিখবেন

End-to-end ML project — অনুসন্ধান থেকে deployment
EDA — feature ও churn relationship
Pipeline + SMOTE + XGBoost — modular code
Hyperparameter tuning Optuna দিয়ে
Model interpretation SHAP
Business metric — saved revenue calculation

১ · Project setup

Bangladesh telecom market — Grameenphone, Robi, Banglalink, Teletalk। Annual churn rate ~১৫-২৫%। প্রতি churn-এ revenue loss + acquisition cost (নতুন গ্রাহক জোগাড় ৫× more expensive than retain)।

Goal: যে গ্রাহক পরবর্তী ৩০ দিনে churn করতে পারেন — early warning। Retention team তাদের special offer দেবে।

Project structure

১) EDA — data understand।
২) Feature engineering।
৩) Pipeline + imbalance + tuning।
৪) Model interpretation।
৫) Deployment + monitoring।
৬) Business impact।

২ · Data generation (synthetic)

Python

import numpy as np
import pandas as pd

np.random.seed(0)
n = 5000

# Telecom churn-এর realistic synthetic data
df = pd.DataFrame({
    "customer_id": [f"BD{i:05d}" for i in range(n)],
    "tenure_months":   np.random.choice(range(1, 73), n),
    "monthly_recharge_bdt": np.random.lognormal(5.5, 0.5, n).astype(int),
    "data_gb":         np.random.exponential(3, n).round(1),
    "voice_minutes":   np.random.gamma(2, 100, n).astype(int),
    "sms_count":       np.random.poisson(20, n),
    "plan_type":       np.random.choice(["prepaid", "postpaid"], n, p=[0.85, 0.15]),
    "complaints_3m":   np.random.poisson(0.5, n),
    "payment_delay_days": np.random.exponential(2, n).round(1),
    "age":             np.random.randint(18, 65, n),
    "is_urban":        np.random.choice([0, 1], n, p=[0.4, 0.6]),
})

# Churn probability — feature-driven
churn_prob = (
    0.05
    + 0.20 * (df["tenure_months"] < 6)        # new customer high churn
    + 0.15 * (df["complaints_3m"] >= 2)
    + 0.10 * (df["payment_delay_days"] > 5)
    - 0.10 * (df["plan_type"] == "postpaid")  # postpaid sticky
    - 0.05 * (df["data_gb"] > 5)              # heavy user sticky
    + np.random.normal(0, 0.05, n)
).clip(0, 1)
df["churned"] = np.random.binomial(1, churn_prob)

print(f"Total customers: {len(df)}")
print(f"Churn rate: {df['churned'].mean():.2%}")
print(df.head(3).T)

৫০০০ গ্রাহক, ~১৫% churn rate। Tenure কম, complaints বেশি, payment delay-এ churn বেশি।

৩ · EDA — exploratory analysis

Python

# Churn rate by feature
print("=== Churn by tenure ===")
print(df.groupby(pd.cut(df["tenure_months"], [0, 6, 12, 24, 72]))["churned"].mean())

print("\n=== Churn by complaints ===")
print(df.groupby("complaints_3m")["churned"].mean().head())

print("\n=== Churn by plan ===")
print(df.groupby("plan_type")["churned"].mean())

print("\n=== Numeric feature correlation with churn ===")
num_features = ["tenure_months", "monthly_recharge_bdt", "data_gb",
                "voice_minutes", "complaints_3m", "payment_delay_days"]
print(df[num_features + ["churned"]].corr()["churned"].sort_values())

EDA-এ patterns: tenure ছোট = churn বেশি, complaints বেশি = churn বেশি, postpaid sticky। ML model এই pattern সব একসাথে exploit করবে।

৪ · Train/Test split (time-aware-like)

Python

from sklearn.model_selection import train_test_split

# Stratified to preserve class ratio
features = ["tenure_months", "monthly_recharge_bdt", "data_gb",
            "voice_minutes", "sms_count", "plan_type",
            "complaints_3m", "payment_delay_days", "age", "is_urban"]

X = df[features]
y = df["churned"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=0, stratify=y)

print(f"Train: {len(X_train)} (churn {y_train.mean():.2%})")
print(f"Test:  {len(X_test)} (churn {y_test.mean():.2%})")

৫ · Pipeline — preprocessing + SMOTE + XGBoost

Python

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline as ImbPipeline
from xgboost import XGBClassifier

numeric_features = ["tenure_months", "monthly_recharge_bdt", "data_gb",
                    "voice_minutes", "sms_count", "complaints_3m",
                    "payment_delay_days", "age"]
categorical_features = ["plan_type", "is_urban"]

preprocessor = ColumnTransformer([
    ("num", Pipeline([
        ("impute", SimpleImputer(strategy="median")),
        ("scale", StandardScaler()),
    ]), numeric_features),
    ("cat", Pipeline([
        ("impute", SimpleImputer(strategy="most_frequent")),
        ("onehot", OneHotEncoder(handle_unknown="ignore")),
    ]), categorical_features),
])

pipe = ImbPipeline([
    ("preprocess", preprocessor),
    ("smote", SMOTE(random_state=0, k_neighbors=5)),
    ("clf", XGBClassifier(
        n_estimators=200, max_depth=5, learning_rate=0.1,
        random_state=0, eval_metric="logloss",
        use_label_encoder=False)),
])

# Need sklearn pipeline import
from sklearn.pipeline import Pipeline

# Re-define preprocessor with sklearn Pipeline:
numeric_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="median")),
    ("scale", StandardScaler()),
])
categorical_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="most_frequent")),
    ("onehot", OneHotEncoder(handle_unknown="ignore")),
])
preprocessor = ColumnTransformer([
    ("num", numeric_pipe, numeric_features),
    ("cat", categorical_pipe, categorical_features),
])

pipe = ImbPipeline([
    ("preprocess", preprocessor),
    ("smote", SMOTE(random_state=0)),
    ("clf", XGBClassifier(n_estimators=200, max_depth=5,
                          learning_rate=0.1, random_state=0,
                          eval_metric="logloss")),
])

pipe.fit(X_train, y_train)
print("Pipeline trained")

৬ · Evaluation

Python

from sklearn.metrics import (classification_report, roc_auc_score,
                              average_precision_score, confusion_matrix,
                              precision_recall_curve)

y_pred = pipe.predict(X_test)
y_prob = pipe.predict_proba(X_test)[:, 1]

print("=== Classification report ===")
print(classification_report(y_test, y_pred, digits=3))

print(f"\nROC-AUC:  {roc_auc_score(y_test, y_prob):.3f}")
print(f"PR-AUC:   {average_precision_score(y_test, y_prob):.3f}")

print("\n=== Confusion matrix ===")
print(confusion_matrix(y_test, y_pred))

# Find threshold for top-10% predicted churn
threshold = np.percentile(y_prob, 90)
print(f"\n=== Top-10% targeting (threshold {threshold:.3f}) ===")
predicted = y_prob >= threshold
hits = (predicted & (y_test == 1)).sum()
total_actual = y_test.sum()
print(f"Recall: {hits}/{total_actual} = {hits/total_actual:.2%}")
print(f"Precision: {hits}/{predicted.sum()} = {hits/predicted.sum():.2%}")

Marketing team budget-এ — top-১০% high-risk customer-কে retention call। ১০% list-এ recall ~৪০-৫০% — অর্থাৎ ৫টি churn-এর ২টি catch হয়। Random call-এর তুলনায় ৩-৪× lift।

৭ · Hyperparameter tuning Optuna

Python · Optuna

import optuna
from sklearn.model_selection import cross_val_score

def objective(trial):
    params = {
        "n_estimators": trial.suggest_int("n_estimators", 100, 500),
        "max_depth":    trial.suggest_int("max_depth", 3, 10),
        "learning_rate": trial.suggest_float("learning_rate", 0.01, 0.3, log=True),
        "subsample":    trial.suggest_float("subsample", 0.5, 1.0),
        "colsample_bytree": trial.suggest_float("colsample_bytree", 0.5, 1.0),
    }

    pipe.set_params(**{f"clf__{k}": v for k, v in params.items()})
    scores = cross_val_score(pipe, X_train, y_train, cv=3,
                             scoring="average_precision", n_jobs=-1)
    return scores.mean()

study = optuna.create_study(direction="maximize",
                            sampler=optuna.samplers.TPESampler(seed=0))
study.optimize(objective, n_trials=20, show_progress_bar=False)

print(f"Best params: {study.best_params}")
print(f"Best PR-AUC: {study.best_value:.3f}")

Project workflow — data থেকে business impact পর্যন্ত। প্রতিটি step পরিষ্কার deliverable।

৮ · SHAP interpretation

Python · SHAP

# pip install shap
import shap

# Get the trained XGBoost model from pipeline
xgb_model = pipe.named_steps["clf"]
preprocessor_only = pipe.named_steps["preprocess"]
X_test_processed = preprocessor_only.transform(X_test)

explainer = shap.TreeExplainer(xgb_model)
shap_values = explainer.shap_values(X_test_processed[:100])

# Top features (mean absolute SHAP)
import numpy as np
mean_abs = np.abs(shap_values).mean(axis=0)
feature_names = (numeric_features +
                 list(preprocessor_only.named_transformers_["cat"]
                      .named_steps["onehot"].get_feature_names_out(categorical_features)))

importance = sorted(zip(feature_names, mean_abs), key=lambda x: -x[1])
print("Top-5 features:")
for name, val in importance[:5]:
    print(f"  {name}: {val:.4f}")

SHAP explain — কোন feature কেন এই customer-কে high-risk বলছে। Marketing team-কে actionable insight: "tenure ৩ মাস + ২ complaints → high churn risk → personal call"। Model trust + business value দু'টোই।

৯ · Deployment — FastAPI

Python · serve.py

import joblib
joblib.dump(pipe, "churn_model_v1.joblib")

# serve.py
from fastapi import FastAPI
from pydantic import BaseModel
import pandas as pd
import joblib

app = FastAPI(title="Telecom Churn API")
model = joblib.load("churn_model_v1.joblib")

class Customer(BaseModel):
    tenure_months: int
    monthly_recharge_bdt: int
    data_gb: float
    voice_minutes: int
    sms_count: int
    plan_type: str
    complaints_3m: int
    payment_delay_days: float
    age: int
    is_urban: int

@app.post("/predict_churn")
def predict(c: Customer):
    df = pd.DataFrame([c.dict()])
    prob = float(model.predict_proba(df)[0, 1])
    risk = "high" if prob > 0.6 else ("medium" if prob > 0.3 else "low")
    return {
        "churn_probability": round(prob, 3),
        "risk_level": risk,
        "recommended_action": {
            "high":   "personal call + special offer",
            "medium": "SMS retention campaign",
            "low":    "no action"
        }[risk]
    }

@app.post("/predict_batch")
def predict_batch(customers: list[Customer]):
    df = pd.DataFrame([c.dict() for c in customers])
    probs = model.predict_proba(df)[:, 1]
    return [{"prob": round(float(p), 3),
             "risk": "high" if p > 0.6 else "low"}
            for p in probs]

# uvicorn serve:app --host 0.0.0.0 --port 8000

১০ · Business impact analysis

Pure ML metric (PR-AUC) থেকে business value-এ translate:

Python

# Business impact estimate
n_customers       = 10_000_000  # 1 Crore subscribers
churn_rate        = 0.15
avg_arpu_bdt      = 250         # average revenue per user / month
retention_rate    = 0.30        # 30% successful save when contacted
campaign_cost     = 50          # BDT per outreach

# Random outreach (no model) — top 10%
random_targeted = n_customers * 0.10
random_actual_churners = random_targeted * churn_rate
random_saved = random_actual_churners * retention_rate
random_revenue_save = random_saved * avg_arpu_bdt * 12  # 1 year ARPU
random_cost = random_targeted * campaign_cost
random_roi = random_revenue_save - random_cost

# ML-targeted top 10% — recall let's say 0.45
ml_targeted = n_customers * 0.10
ml_recall = 0.45
total_churners = n_customers * churn_rate
ml_actual_churners = total_churners * ml_recall
ml_saved = ml_actual_churners * retention_rate
ml_revenue_save = ml_saved * avg_arpu_bdt * 12
ml_cost = ml_targeted * campaign_cost
ml_roi = ml_revenue_save - ml_cost

print(f"Random campaign: BDT {random_roi/10**7:.1f} Cr ROI")
print(f"ML-targeted:     BDT {ml_roi/10**7:.1f} Cr ROI")
print(f"Lift: {ml_roi/random_roi:.1f}×")

ML-targeted retention campaign random-এর চেয়ে ৩-৪× ROI। বছরে কোটি টাকা saved revenue। ML model business case-এ সরাসরি translate।

১১ · Production checklist

✅ Pipeline single object (joblib)।
✅ FastAPI endpoint (single + batch)।
✅ Pydantic input validation।
✅ Health check endpoint।
✅ Logging (request + prediction)।
✅ Drift detection (weekly)।
✅ Performance dashboard।
✅ Model versioning (v1, v2)।
✅ A/B test infrastructure।
✅ Documentation (README, model card)।

Production deployment-এ — true labels arrive late (১-৩ মাস delayed)। Live performance monitor proxy metric (engagement, top-list retention rate)। Quarterly retrain mandatory — telecom market dynamic।

ভাবনার প্রশ্ন

প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।

প্র ০১ Churn label "next 30 days within churn" — temporal label leak কী challenge? Production-এ feature time alignment?

Churn ML-এর সবচেয়ে subtle challenge — temporal correctness।

Common bug:

"Customer's last 7 days activity" feature।
If churn label observed at day 30, then days 23-30 leak into feature।
Cheating।

Correct timeline:

"As of day T" — feature computed using only data till day T।
Label — between day T and T+30।
No overlap।

Feature engineering rules:

"Past N days" — past relative to T।
"Cumulative since signup" — known at T।
"Plan currently active" — at T snapshot।
Forbidden: future activity, future complaint।

Backtest setup:

Multiple snapshot date — Jan 1, Feb 1, Mar 1।
Each snapshot — feature + label compute correctly।
Train on early, test on later।

Survival analysis alternative:

Cox proportional hazard।
Time-varying covariates।
"Tenure" treated explicitly।
scikit-survival library।

Production system:

Daily snapshot — feature store।
Feature freshness vs latency tradeoff।
Real-time vs batch aggregation।

Bangladesh telecom edge cases:

Number portability (MNP) — "churn" কীভাবে define?
Multi-SIM users — household account aggregate?
Dormant ≠ churn — definition critical।
Roaming — usage spike misleading।

Data engineering:

Event-driven architecture।
Kafka log → feature aggregator।
Time-window query optimized।
Feast/Tecton feature store।

মূল উপলব্ধি: Temporal correctness churn ML-এর foundation। Definition + label window + feature time alignment — engineering rigor।

প্র ০২ Telecom churn model 6 মাস পর performance drop — root cause কী হতে পারে? Mitigation?

Real production challenge। Concept drift inevitable।

Possible causes:

(১) Market change:

Competitor cheaper plan launch।
5G rollout — different usage pattern।
Economic downturn।

(২) Data drift:

Tariff change — recharge amount distribution shift।
New product (data pack) — usage feature change।
App update — collected feature different।

(৩) Concept drift:

Customer behavior shift — relationship feature → churn change।
"Heavy data user not churn" → reverse if data prices drop।

(৪) Feedback loop:

Model prediction → retention call → customer behavior change।
Treated customer no longer churn — model loses signal।

(৫) Seasonal:

Eid recharge spike — momentary drift।
Padma bridge opening — geo pattern shift।

Detection:

Performance metric monitor — AUC weekly।
Feature distribution PSI।
Prediction distribution shift।
Top features SHAP drift।

Mitigation:

(১) Retrain:

Schedule (monthly/quarterly)।
Trigger-based (drift detect)।
Sliding window — recent 6-12 months data।

(২) Online learning:

Continuous update with new label।
River library।
Streaming gradient।

(৩) Feature stability:

Robust features (relative, normalized)।
Avoid absolute amount, prefer percentile।
Domain features stable।

(৪) Ensemble:

Multiple model — old + new।
Weighted by recency।
Champion-challenger।

(৫) A/B test:

New model vs old।
Holdout group untreated।
Causal impact estimate।

(৬) Adapt label:

Incremental redefinition।
"Churn within 60 days" → "30 days" — sensitivity।

Bangladesh-specific:

BTRC tariff regulation change — major drift trigger।
SIM tax/duty — recharge pattern shift।
Internet shutdown event — anomaly।
Festival — seasonal noise।

Operational:

On-call rotation।
Incident postmortem।
Stakeholder communication।
Backfill plan (rule-based fallback)।

মূল উপলব্ধি: Drift inevitable। Detection + retraining + adaptation — full lifecycle। Production ML continuous, not "deploy and forget"।

প্র ০৩ Marketing team-কে SHAP-based explanation — কীভাবে actionable করেন?

Model interpretability + business action — তে gap পূরণ।

SHAP output:

Per-prediction feature contribution।
Customer X — tenure -2, complaints +1.5, payment_delay +1।
Numeric SHAP value।

Marketing translation:

"Tenure 3 months" → "new customer onboarding offer"।
"3 complaints" → "service quality follow-up call"।
"Payment delay 8 days" → "billing flexibility offer"।

Persona generation:

Cluster high-risk customer।
Each cluster — dominant SHAP profile।
Persona name: "Frustrated Newcomer", "Payment Stress"।
Tailored campaign per persona।

Action playbook:

SHAP top driver → specific intervention।
Decision matrix — driver × demographic।
Pre-built script library।

Empirical validation:

A/B test — SHAP-driven vs blanket offer।
Save rate comparison।
ROI per persona।

Visualization for non-tech:

Force plot — too complex for ops।
Top-3 reason text — natural language।
"Customer at risk because: 1) Recent complaints, 2) Payment delays, 3) Low data usage drop"।
Bangla rendering important।

Dashboard design:

Top-1000 high-risk daily list।
Sortable by risk score, persona।
Quick action button — call queue, SMS template।
Outcome tracking।

Caveats:

SHAP correlation, not causation — intervention may not work।
Confounders — high complaint customer also low engagement।
Simpson's paradox possible।
RCT recommended for major intervention।

Privacy:

SHAP individual prediction — personal data।
Aggregated insight share-friendly।
Customer-facing explanation — careful wording।

Bangladesh marketing context:

SMS template Bangla।
Bkash refund offer।
Free data pack incentive।
Local language call center scripting।

Continuous improvement:

Outcome-driven persona refinement।
Failed save root cause।
Successful save pattern reinforce।
Annual persona overhaul।

মূল উপলব্ধি: SHAP raw output → persona-based action playbook → Bangla communication → outcome tracking — full chain। ML team + marketing team partnership critical।

প্র ০৪ Churn model-এর fairness audit — Bangladesh context-এ কী protected attribute? Bias mitigation?

Responsible AI — under-discussed in Bangladesh ML, but essential।

Protected attributes:

Gender: female সাধারণত under-represent telecom data।
Religion: name-based proxy possible।
Age: elderly differential treatment।
Geography: rural vs urban।
Disability: usage pattern atypical।
Income tier: proxy for class।

Fairness metrics:

Demographic parity: equal positive rate across groups।
Equal opportunity: equal recall across groups।
Equalized odds: equal TPR ও FPR।
Calibration: probability accurate per group।

Audit process:

Identify protected groups।
Subgroup performance analysis।
Statistical test (chi-square)।
Threshold (10% gap concerning)।
Document findings।

Bias sources:

Data bias: historical sampling।
Label bias: "churn" defined cultural context।
Feature bias: proxy variables।
Algorithmic bias: objective function।

Mitigation strategies:

(১) Pre-processing:

Reweighting examples।
Disparate impact remover।
Group-specific resampling।

(২) In-processing:

Fairness constraint in loss।
Adversarial debiasing।
Distributionally robust optimization।

(৩) Post-processing:

Group-specific threshold।
Calibration adjust।
Reject option।

Tools:

AIF360 (IBM)।
Fairlearn (Microsoft)।
What-If Tool।
Aequitas।

Bangladesh-specific:

Female mobile usage typically lower — model may falsely flag churn।
Rural — limited service availability, churn cause structural।
Religious minority — name-based bias risk।
Hill tracts — distinct usage, generalization issue।

Trade-offs:

Accuracy vs fairness tradeoff।
Multiple fairness metrics conflict।
Choose by stakeholder consultation।

Documentation:

Model card — performance per subgroup।
Data sheet — collection bias acknowledged।
Decision rationale logged।

Stakeholder engagement:

BTRC consultation।
Customer ombudsman।
Civil society — gender, disability advocacy।
Public accountability।

Legal:

Bangladesh Data Protection Act emerging।
Telecom regulation evolving।
Discrimination law — limited but growing।

Continuous monitoring:

Fairness drift over time।
New protected attribute emerge।
Annual external audit।

মূল উপলব্ধি: Fairness — Bangladesh ML-এর next frontier। Telecom mass-impact industry — responsible AI critical। Engineering investment now, regulatory pressure soon।

অনুশীলন

Project extension: Above pipeline-এ "complaint_resolution_time" feature add করে retrain — performance change?

সাধারণত ১-৩% PR-AUC improvement। Feature engineering — ML-এর low-hanging fruit।
Production challenge: Real-time scoring API throughput < ১০ms দরকার। Optimization?
- ONNX Runtime convert।
- Quantize XGBoost (treelite)।
- Feature pre-compute (Redis cache)।
- Batch request।
- Hardware (GPU, dedicated server)।
ভাবুন: Bangladesh-এর একটি bank-এর জন্য similar churn model — telecom-এর সাথে কী ভিন্ন?
- Definition: account close vs dormant — different।
- Data: transaction history, balance, product holding।
- Tenure: longer typical, decade+।
- Action: retention call cost higher (relationship manager)।
- Compliance: Bangladesh Bank guideline।
- Cross-sell: churn prevention via more product।
- Segmentation: wealth tier different strategy।
- Time: quarterly reasonable, not 30 day।

এই পাঠে যা শিখবেন

১ · Project setup

২ · Data generation (synthetic)

৩ · EDA — exploratory analysis

৪ · Train/Test split (time-aware-like)

৫ · Pipeline — preprocessing + SMOTE + XGBoost

৬ · Evaluation

৭ · Hyperparameter tuning Optuna

৮ · SHAP interpretation

৯ · Deployment — FastAPI

১০ · Business impact analysis

১১ · Production checklist

ভাবনার প্রশ্ন

অনুশীলন

আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ