Bayes-এর উপপাদ্য — উদাহরণসহ

এটি ৩০০-বছর পুরানো philosophical battle — এখনো জীবিত।

Frequentist worldview:

• Probability = long-run frequency.
• Parameters fixed, data random.
• "This coin's true bias is some specific number; we estimate it from samples."
• Tools: MLE, p-values, confidence intervals.

Bayesian worldview:

• Probability = degree of belief.
• Parameters random (with prior), data fixed.
• "My belief about this coin's bias is a distribution; data updates it."
• Tools: Bayes' theorem, posterior distributions, credible intervals.

Concrete example:

A coin flipped 10 times — 7 heads observed. What's the bias?

• Frequentist (MLE): $\hat{p} = 7/10 = 0.7$। 95% CI: [0.42, 0.92]। "If we repeat this experiment many times, 95% of such intervals will cover true p."
• Bayesian (uniform prior): Posterior = Beta(8, 4)। Mean ≈ 0.67. 95% credible interval: [0.40, 0.89]। "True p is in [0.40, 0.89] with 95% probability."
• Bayesian (informative prior): If you know coins typically fair, prior Beta(50, 50) — posterior Beta(57, 53)। Mean ≈ 0.52. Different answer!

কখন কোনটি ব্যবহার:

• Frequentist:
  • প্রচুর data — Bayesian এর সাথে practically same.
  • Standard scientific reporting.
  • Don't want to specify prior.
• Bayesian:
  • Sparse data — prior knowledge incorporate.
  • Uncertainty quantification critical.
  • Sequential decisions (active learning)।
  • Hierarchical structures.

Modern AI-তে:

• DL training: mostly frequentist (MLE)।
• Hyperparameter tuning: Bayesian optimization.
• Uncertainty quantification: Bayesian (MC dropout, deep ensembles)।
• RL: Bayesian flavor (Thompson sampling)।

Real-world disagreement:

• 2020 COVID drug efficacy debates — Bayesian reanalyses sometimes contradicted frequentist trial conclusions.
• A/B testing — Bayesian increasingly preferred (Optimizely, Google)।

মূল উপলব্ধি: "Right answer" depends on philosophy + data + use case. Modern practice — pragmatic mix. Mature data scientist দু'টি pole-ই understand.

এই paradox — ML-এর সবচেয়ে চমকপ্রদ phenomena-গুলোর একটি।

Independence assumption — কেন স্পষ্টতই ভুল:

• "New" + "York" — highly correlated. কখনো একসাথে।
• "Free" + "money" + "click" — spam-এ একসাথে আসে।
• Image pixels — neighbor pixels strongly correlated.
• True conditional independence rare.

তবু কেন কাজ করে:

• Classification accuracy ≠ probability calibration: Naive Bayes "Yes/No" decision সঠিক হলেই হয়। সঠিক probability estimate দরকার নেই।
• Argmax robustness: Even if probabilities miscalibrated, argmax (highest prob class) often still correct.
• Domain-redundancy: Multiple features pointing same direction → wrong probabilities cancel out.
• Domingos & Pazzani (১৯৯৭): "On the optimality of the simple Bayesian classifier under zero-one loss" — এই paradox formally explore.

Practical strengths:

• Speed: Training O(n × d) — extremely fast. Inference real-time.
• Scalable: Word counts maintain — incremental update easy.
• Small data friendly: Each class needs only feature distributions, not joint.
• Interpretable: Per-feature contribution visible.

সফলতার জায়গা:

• Spam filtering — ১৯৯০ থেকে standard.
• Document classification (sentiment, topic)।
• Medical diagnosis (older systems)।
• Search query understanding.

Limitations:

• Probability calibration: Outputs don't represent true probabilities. Use Platt scaling for calibration.
• Zero-frequency problem: A word never seen with a class → P = 0, kills product. Solution: Laplace smoothing.
• Continuous features: Need distribution assumption (Gaussian Naive Bayes)।
• Modern DL outperforms: CNN/Transformer for images/text — better contextual understanding.

Variants:

• Multinomial NB: Word counts.
• Bernoulli NB: Binary word presence.
• Gaussian NB: Continuous features.
• Complement NB: Imbalanced data.

Lesson:

• "Wrong assumption" + "right structure" = often surprisingly good.
• ML wisdom: simple methods first, complex if needed.
• Naive Bayes — interpretable baseline before DL.

মূল উপলব্ধি: Pragmatic সাফল্য > theoretical purity. Naive Bayes — এই philosophy-র poster child. AI-তে "good enough fast" বেশিরভাগ সময় "perfect slow"-কে হারায়।

এই scenario — surveillance AI-র সবচেয়ে gripping case study. Bayes মাথায় না রাখলে — civil liberty disaster.

Numbers:

• Population: ১৭ কোটি (1.7 × 10⁸)।
• Suspects: ১,০০০।
• Innocent: 169,999,000.
• Accuracy 99.9% → error rate 0.1%।
• If "accuracy" = sensitivity = specificity (অনুমান):
  • True positives: ১,০০০ × ০.৯৯৯ = ৯৯৯।
  • False positives: 169,999,000 × ০.০০১ = ১৬৯,৯৯৯।

Bayesian interpretation:

• Total alarms: 169,999 + 999 ≈ 170,998.
• P(actual suspect | alarm) = 999 / 170,998 ≈ 0.58%।
• আপনি AI-চিহ্নিত হলে — actual suspect মাত্র ০.৫৮%। মানে — ১৭০ জন alarm-এর মধ্যে মাত্র ১ জন আসল।

কেন বিপজ্জনক:

• Massive false-positive load: ১,৭০,০০০ innocent মানুষের harassment. Most don't know why.
• Demographic bias amplification: ফেসিয়াল রিকগনিশন brown skin-এ less accurate. Innocent ethnic minorities disproportionately affected.
• Burden of proof reversal: "Prove you're not the suspect" — alarming.
• Chilling effects: Surveillance-এর সচেতনতা political/religious expression কমায়।
• Mission creep: ১,০০০ "suspects" থেকে — ১০,০০০ জন নজরদারিতে। ১০ লক্ষ। Police state.

Real-world examples:

• 2017 South Wales Police: Facial recognition deployed. Identified 2,470 suspects, 92% (~2,300) false positives. Many wrongful detentions.
• Robert Williams (২০২০, Detroit): Black man wrongly arrested due to facial recognition error. 30 hours in jail. Innocent.
• UK Met Police trials: 81% false positive rate in some deployments.
• China's Social Credit System: Massive surveillance, accuracy claims often unverified.

Mathematical solutions — কী করা যায়:

• Higher specificity: 99.9999% accuracy needed. Currently impossible practically.
• Multi-stage verification: AI flags → human review → second biometric (fingerprint, ID) → action. Compound accuracy higher.
• Restricted use: Only investigations where prior probability is high (e.g., crime scene location, time)।
• Error budgets: "How many false positives are we willing to accept?" Public debate.

Policy implications:

• EU AI Act (২০২৪) — public-space facial recognition heavily restricted.
• San Francisco, Boston (US) — banned.
• Bangladesh-এ — National Telecommunication Monitoring Centre debate.
• Civil society pushback worldwide.

মূল উপলব্ধি: "৯৯.৯% accurate" — sounds great, statistically dangerous in rare-event search. Bayes' theorem — mathematical truth এবং civil rights protection. AI engineers যাঁরা societal impact বুঝেন — তাঁরা Bayes-এর implications প্রথমে চিন্তা করেন।

Hyperparameter tuning — DL-এর "secret sauce"। Bayesian Optimization এই tuning-কে science বানিয়েছে।

সমস্যা:

• একটি model train করতে হয়তো ৩-৬ ঘণ্টা।
• Hyperparameters (learning rate, depth, batch size, dropout, ...) — প্রতিটি combination test.
• Grid search 5 hyperparams × 5 values = 3,125 trials = মাস।
• Random search better, but still wasteful.

Bayesian Optimization-এর approach:

• Surrogate model: "Hyperparameter → performance" function-এর Gaussian Process model.
• Acquisition function: "পরবর্তী কোন hyperparameter try করব?" — exploration vs exploitation balance.
• Expected Improvement (EI): "এই point try করলে best-এর চেয়ে কত improvement প্রত্যাশা?"
• Iteration: Try → observe performance → update surrogate → choose next.

Step by step example:

• Iter 1: Random hyperparam, train, observe accuracy 80%।
• Iter 2: GP-এ — "near similar config" predicted 78%। Try different config — accuracy 75%।
• Iter 3: GP updated — surface model. Best EI point pick → accuracy 85%।
• Iter 10: 92%, GP-এ confidence high.
• Iter 20: 94% — converged.

কেন Bayes-এর সাথে সম্পর্ক:

• Prior: "Performance smooth function of hyperparameters" — Gaussian Process prior.
• Likelihood: Observed accuracy values.
• Posterior: Updated belief about performance landscape.
• Decision: Posterior থেকে next best point.

Grid vs Random vs Bayesian:

• Grid: Exponential cost, no learning. 5 dims × 10 values = 100,000 trials.
• Random: Better empirically (Bergstra & Bengio, ২০১২) — most dims unimportant.
• Bayesian: Learns from past trials. 50-100 trials reach grid-quality.

Practical tools:

• Optuna: Most popular Python library.
• Hyperopt: Older, classic.
• Google Vizier: Internal tool.
• Microsoft NNI: Distributed BayesOpt.
• Weights & Biases Sweeps: Cloud-based.

Beyond hyperparameters:

• Drug design — molecular configurations.
• Chemistry — reaction conditions.
• Hardware design — chip layout.
• A/B testing — variant selection.
• RL — sample-efficient policy search.

Limitations:

• GP scales poorly with high dimensions (>20)।
• Categorical hyperparameters tricky.
• Noisy observations (training stochasticity)।
• Modern alternatives: BOHB, ASHA — early stopping + Bayesian.

Lesson:

• "Sequential decisions under uncertainty" — Bayesian framework natural.
• $5 GPU hour saved × 100 trials = significant cost reduction.
• Modern AI-র cost optimization-এ Bayesian thinking essential.

মূল উপলব্ধি: Bayes-এর শক্তি — শুধু theory নয়, expensive optimization problem-এ practical efficiency. Production AI-তে Bayesian Optimization use না হলে — হাজার-হাজার ডলার GPU waste.