প্রজেক্ট — MNIST classifier

MNIST-এর ৯৯% misleading — production deployment-এ surprisingly poor। Important lesson।

MNIST limitations:

• Centered, normalized।
• Single digit per image।
• Clean background।
• Fixed size (২৮×২৮)।
• US-style handwriting bias।

Real world challenges:

• Skewed angle, rotation।
• Multiple digit per image।
• Variable size, position।
• Noisy background, lighting।
• Different style (Bangla numeral!)।

Distribution shift:

• Training distribution ≠ deployment।
• Out-of-distribution failure।
• Confidence high — accuracy low।
• Calibration broken।

Strategies for real-world:

(১) More diverse data:

• EMNIST — extended MNIST (handwritten letters)।
• SVHN — Street View House Numbers।
• Handwritten Bangla — ISI Kolkata, BUET।
• Real-world capture data।

(২) Data augmentation:

• RandomAffine — rotation, translation।
• ElasticDeformation — handwriting variation।
• RandomErasing — robustness।
• Color jitter (color images)।

(৩) Pre-processing:

• Adaptive thresholding।
• Bounding box detection।
• Centering, normalization।
• Multi-digit segmentation।

(৪) Architecture upgrade:

• ResNet, EfficientNet।
• Attention mechanisms।
• Transformer-based।
• Ensemble multiple model।

(৫) Domain adaptation:

• Source: MNIST, target: real।
• Adversarial training।
• Feature alignment।
• Active learning unlabeled data।

Bangla numeral case:

• "০, ১, ২, ৩..." — visually different।
• Conjunct shapes complex।
• BanglaLekha dataset (Hossain et al.)।
• Specific training required।

Production pipeline:

# Real-world digit recognition
# 1. Detection — find digit regions (YOLOv5)
# 2. Segmentation — separate digits
# 3. Classification — your CNN
# 4. Post-processing — sequence join

Confidence calibration:

• Temperature scaling (Guo et al.)।
• Reject low-confidence prediction।
• Out-of-distribution detection।
• Production safety net।

Real datasets to consider:

• MNIST-M (different background)।
• USPS — postal digits।
• QMNIST — extended।
• Custom collection।

Practical Bangladesh use case:

• Bank check digit recognition।
• Bangla numeral OCR (NID, education forms)।
• Postal address recognition।
• BanglaLekha + augmentation key।

Evaluation rigor:

• Held-out real test set।
• Multiple writer।
• Different conditions।
• Confusion matrix analysis।

মূল উপলব্ধি: MNIST academic baseline — production not directly applicable। Distribution shift critical। Real data + augmentation + robust architecture combination needed। Bangla numeral specific dataset। Production ML — beyond accuracy number।

Misclassification analysis — model debugging-এর critical skill।

Visualization technique:

import matplotlib.pyplot as plt
# Find misclassified
wrong = []
for x, y in test_loader:
    x, y = x.to(device), y.to(device)
    pred = model(x).argmax(1)
    mask = pred != y
    if mask.any():
        wrong.append((x[mask].cpu(), y[mask].cpu(),
                      pred[mask].cpu()))

# Plot grid
fig, axes = plt.subplots(2, 5, figsize=(10, 4))
for i, ax in enumerate(axes.flat):
    img, true, pred = wrong[0][0][i], wrong[0][1][i], wrong[0][2][i]
    ax.imshow(img.squeeze(), cmap='gray')
    ax.set_title(f'T:{true} P:{pred}')

Common confusion pairs:

• ৪ ↔ ৯: similar curve top।
• ৩ ↔ ৮: closed loops।
• ১ ↔ ৭: vertical stroke।
• ০ ↔ ৬: oval shape।

Why model confidently wrong:

• Softmax output peaked even for OOD।
• Features dominate similar pattern।
• No "I don't know" output।
• Calibration issue।

Examples:

• Sloppy 4 — top open, 9-like।
• European 7 — extra crossbar, 4-like।
• Squashed 8 — looks like 3।
• Edge cases inherently ambiguous।

Confidence analysis:

# Wrong predictions confidence
wrong_confs = []
for x, y in test_loader:
    x, y = x.to(device), y.to(device)
    out = model(x)
    pred = out.argmax(1)
    probs = F.softmax(out, dim=1).max(1)[0]
    mask = pred != y
    wrong_confs.extend(probs[mask].tolist())

import numpy as np
print(f"Mean wrong confidence: {np.mean(wrong_confs):.3f}")
print(f"Max wrong confidence: {np.max(wrong_confs):.3f}")
# Often >0.95 — overconfident

Calibration techniques:

• Temperature scaling — divide logits by T।
• Platt scaling — sigmoid fit।
• Isotonic regression।
• Deep ensembles — uncertainty estimate।

Out-of-distribution detection:

• Maximum softmax probability।
• Energy-based score।
• Mahalanobis distance feature space।
• Reject low confidence।

Saliency map analysis:

• Gradient w.r.t. input।
• Highlight important pixel।
• Misclassification — wrong region focus।
• Debug feature learning।

Grad-CAM:

• Class activation map।
• What region influenced decision।
• Visual debugging।
• Production explanation।

Improvement strategies:

• Hard example mining।
• Focal loss — hard sample focus।
• Curriculum learning — easy→hard।
• Data augmentation targeted।

Human comparison:

• Show wrong examples to humans।
• Many ambiguous — humans also struggle।
• Inter-annotator agreement।
• Upper bound accuracy।

Production safeguards:

• Reject threshold — confidence < 0.9।
• Human-in-loop for uncertain।
• Continuous monitoring।
• Drift detection।

মূল উপলব্ধি: Misclassification visualization — model understanding key। Confused pairs systematic, often visually similar। Confidence often miscalibrated — high even wrong। Calibration + OOD detection production essential। Saliency map debugging tool।

Overfitting — DL-এর most common issue। Diagnostic + fix systematic।

Diagnostic:

(১) Loss curve analysis:

• Training loss decrease।
• Validation loss increase or plateau।
• Gap widening — overfit।

(২) Accuracy gap:

• Train vs validation gap >2-3% concerning।
• $>5\%$ overfit clear।

(৩) Capacity check:

• Parameter / training sample ratio।
• >1 — likely overfit risk।

Fix hierarchy:

(১) More data — best fix:

• More training data ideal।
• Often unavailable।

(২) Data augmentation:

train_transform = transforms.Compose([
    transforms.RandomRotation(10),
    transforms.RandomAffine(0, translate=(0.1, 0.1)),
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,)),
])

• Effectively increase data।
• Free, easy।
• Domain-appropriate augmentation।

(৩) Dropout:

• Already 0.3 in code।
• Increase to 0.5 if persist।
• Multiple layers।

(৪) Weight decay:

optimizer = optim.Adam(model.parameters(),
                       lr=1e-3, weight_decay=1e-4)

• L2 regularization।
• Standard 1e-4 to 1e-3।

(৫) Early stopping:

• Validation loss monitor।
• No improvement 3 epoch — stop।
• Best checkpoint use।

(৬) Reduce model size:

• Fewer parameters।
• Smaller layer।
• Less overfit capacity।

(৭) Batch normalization:

• Already in code।
• Slight regularization effect।
• Stable training।

(৮) Learning rate schedule:

• Cosine annealing।
• Warmup gradual।
• Better convergence।

Implementation order:

1. Add augmentation first।
2. Increase dropout if still overfit।
3. Add weight decay।
4. Early stopping।
5. Architecture tweak।
6. More data if possible।

Validation set strategy:

• Train/val/test split।
• Tune on val, evaluate test।
• Cross-validation for small data।

Modern techniques:

• Mixup — sample interpolation।
• CutMix — patch swap।
• Label smoothing — soft targets।
• Stochastic depth।

MNIST-specific:

• RandomRotation(10) major help।
• RandomAffine translation 10%।
• Train acc dip slightly, test rise।
• ৯৯.৪%+ achievable।

Bias-variance trade-off:

• High variance = overfit।
• Regularization reduce variance।
• Slight bias increase OK।
• Total error decrease goal।

মূল উপলব্ধি: Overfitting diagnostic — loss/accuracy gap। Fix order: augmentation, dropout, weight decay, early stopping। MNIST RandomRotation impactful। Modern techniques (Mixup, label smoothing) extra boost। Systematic approach essential।

Practical Bangladesh ML deployment — bank check OCR। Production-grade engineering।

Use case requirements:

• Cheque amount recognition (digit + Bangla)।
• Account number reading।
• High accuracy — money matters।
• Real-time processing।
• Audit trail।

Pipeline architecture:

Stage 1 — Image processing:

• Cheque scan/photo।
• Geometric correction (perspective)।
• Skew correction।
• Background removal।

Stage 2 — Field detection:

• YOLOv8 — amount field, signature, date।
• Custom train Bangladesh check format।
• Templates per bank।

Stage 3 — Digit segmentation:

• Connected components।
• Touching digit separation।
• Vertical projection profile।

Stage 4 — Classification:

• Per-digit CNN।
• Latin + Bangla numeral।
• Confidence per digit।

Stage 5 — Post-processing:

• Sequence reconstruction।
• Sanity check (amount > 0)।
• Format validation।
• Cross-reference written amount।

Data collection:

• Real cheque sample collection।
• Diverse banks (Sonali, DBBL, City)।
• Privacy/legal compliance।
• Synthetic augmentation।

Bangla numeral specific:

• Use BanglaLekha-Isolated dataset।
• BUET handwritten Bangla numeral।
• Mixed Latin-Bangla detection।
• Conjunct/joint character handling।

Robustness:

• Stamp overlay handling।
• Strikethrough recognition।
• Various pen colors।
• Stain/fold tolerance।

Confidence-based routing:

# Critical decision logic
def route_decision(digit_confs, amount):
    min_conf = min(digit_confs)
    if min_conf > 0.99 and len(digit_confs) >= 3:
        return "auto_process"
    elif min_conf > 0.95:
        return "human_verify"
    else:
        return "human_review"

Cross-validation:

• Numeric amount vs written amount।
• Discrepancy → reject।
• Multiple model ensemble।
• Voting consensus।

Architecture upgrade from MNIST:

• EfficientNet-B0 for digit।
• Detection — YOLO/DETR।
• OCR — TrOCR style transformer।
• Ensemble multiple।

Training strategy:

• Synthetic data first।
• Real data fine-tune।
• Active learning ongoing।
• Continuous improvement।

Deployment infrastructure:

• On-premise (data sensitivity)।
• Docker + Kubernetes।
• GPU server bank-internal।
• Audit log every decision।

Monitoring:

• Per-day accuracy tracking।
• Confidence distribution।
• Reject rate trend।
• Human override rate।

Compliance:

• Bangladesh Bank guidelines।
• Data Protection Act।
• Customer data security।
• Audit trail mandatory।

Error handling:

• OOD detection — unusual cheque।
• Reject confidently।
• Human escalation।
• Customer protection priority।

Human-in-loop:

• Verification UI।
• Quick review workflow।
• Disagreement learning loop।
• Continuous data collection।

Cost analysis:

• Initial development: ৬-১২ মাস।
• Hardware: $50K-200K।
• Annual maintenance: $50K।
• ROI: significant via automation।

Bangladesh banking landscape:

• BACPS (Bangladesh Automated Cheque Processing System)।
• BB initiative driving digitization।
• Local bank tech adoption growing।
• Practical AI opportunity।

মূল উপলব্ধি: Bank check OCR — multi-stage pipeline। Detection → segmentation → classification → validation। MNIST CNN entry point only। Real production — robust architecture, confidence routing, human-in-loop। Bangladesh banking — mature ML opportunity। Compliance + accuracy critical।