পাঠ ৩৬ · ৪৫-এর মধ্যে · মডিউল ৫
Home / AI Courses / Machine Learning / Bayesian Networks

Bayesian Networks

Bayesian networks — DAG probabilistic models
৮ মিনিট পড়া উচ্চ · Advanced pgmpy

এই পাঠে যা শিখবেন

  • Probabilistic graphical model কী — Bayes-এর গভীর প্রয়োগ
  • DAG ও conditional probability table (CPD)
  • Chain rule decomposition — কেন compact
  • d-separation — independence reasoning
  • pgmpy দিয়ে Bangladesh medical diagnosis (TB diagnosis network)

১ · কেন Bayesian Network?

১০টি binary variable-এর joint distribution — $2^{10} = 1024$ probability। ৫০টি variable — $2^{50} \approx 10^{15}$। কোনোভাবেই storage-এ আঁটে না। কিন্তু — বাস্তবে variable-গুলো সব pairwise dependent না। "Smoking → cancer → cough" — এই structure থাকলে শুধু conditional probability $P(\text{cough} \mid \text{cancer})$ store করলেই হলো।

Judea Pearl (১৯৮৮) — Bayesian Network formalize। ২০১১-তে Turing award। Causal AI-র গোড়াপত্তন। Bayesian Network মূল idea — independence structure exploit করে exponential → polynomial।

Bayesian Network = DAG + CPDs

Node = variable। Edge $X \to Y$ মানে $X$ "causes" বা directly influences $Y$। Cycle না (acyclic)। প্রতিটি node-এ একটি conditional probability distribution $P(X_i \mid \text{Pa}(X_i))$ — parent given তার distribution।

২ · Joint distribution decomposition

Chain rule of probability:

$$P(X_1, \ldots, X_n) = \prod_{i=1}^n P(X_i \mid X_1, \ldots, X_{i-1})$$

Bayesian Network-এর মূল সরলীকরণ — $X_i$ শুধু তার parents-এর উপর conditional, অন্য ancestor-এর সাথে independent (given parents):

$$P(X_1, \ldots, X_n) = \prod_{i=1}^n P(X_i \mid \text{Pa}(X_i))$$

৫টি binary variable, যদি প্রতি node-এর সর্বোচ্চ ২ parent — store মাত্র $5 \times 2^3 = 40$ probability (full joint $2^5 = 32$, তুলনীয়)। ১০ node, max ৩ parent → $10 \times 2^4 = 160$ vs $2^{10} = 1024$ — ৬× সাশ্রয়। ৫০ node-এ পার্থক্য astronomic।

৩ · উদাহরণ — Sprinkler network

চার variable:

  • Cloudy (C)
  • Sprinkler (S) — depends on Cloudy (cloudy হলে sprinkler off)।
  • Rain (R) — depends on Cloudy।
  • WetGrass (W) — depends on Sprinkler ও Rain।

Joint:

$$P(C, S, R, W) = P(C) \cdot P(S \mid C) \cdot P(R \mid C) \cdot P(W \mid S, R)$$

Full joint হত $2^4 = 16$ entry। CPD-এ — $P(C)$ ১, $P(S|C)$ ২, $P(R|C)$ ২, $P(W|S,R)$ ৪ → মোট ৯ entry।

৪ · Conditional independence ও d-separation

Bayesian Network-এ structural information directly conditional independence indicate করে। দু'টি node X ও Y — given Z — independent কি? এই উত্তর "d-separation" rule দিয়ে।

তিন basic structure:

  • Chain $X \to Z \to Y$: Z observed হলে X ⊥ Y। (Smoking → cancer → cough; cancer জানলে smoking ও cough independent।)
  • Fork $X \leftarrow Z \to Y$: Z observed হলে X ⊥ Y। (Common cause — সূর্য ☀️ — উভয় ice cream sale ও sunburn।)
  • V-structure (collider) $X \to Z \leftarrow Y$: Z observed হলে X ও Y dependent! (Burglary, earthquake → alarm; alarm shunlei দু'টোই possible cause।)

এই counter-intuitive collider behavior — Pearl-এর breakthrough। "Explain away" reasoning — alarm বাজলে যদি earthquake confirm — burglary-র probability কমে।

৫ · Inference — query answer

"Wet grass হয়েছে — cloudy ছিল কি?" — এটা $P(C \mid W)$ compute। Methods:

  • Exact inference:
    • Variable elimination — sum out non-query variables।
    • Junction tree algorithm — DAG → tree → message passing।
    • Polynomial in tree-width, exponential worst-case।
  • Approximate inference:
    • Sampling — rejection, importance, Gibbs (L40)।
    • Variational methods — distribution approximation।
    • Loopy belief propagation।

৬ · Learning — structure ও parameters

Parameter learning: structure given হলে — CPT-গুলো MLE বা Bayesian estimation থেকে।

Structure learning: সবচেয়ে কঠিন। DAG-এর exponential search space।

  • Score-based — BIC, BDe — search best DAG।
  • Constraint-based — conditional independence tests।
  • Hybrid — structure prior + data।

Practice-এ — domain expert structure দেন, parameter data থেকে learn।

Bayesian Network — Sprinkler example Joint = ∏ P(node | parents) Cloudy P(C) Sprinkler P(S|C) Rain P(R|C) WetGrass P(W|S,R) P(C) T 0.5 / F 0.5 P(S|C) C=T → S=T 0.1 C=F → S=T 0.5 P(R|C) C=T → R=T 0.8 C=F → R=T 0.2 P(W|S,R) — 4 entries
Sprinkler Bayesian Network — Cloudy → {Sprinkler, Rain} → WetGrass। Joint distribution ৪টি local CPD থেকে। Inference — query "WetGrass given, Cloudy probability?"।

৭ · pgmpy দিয়ে Bangladesh TB diagnosis

Smoking → TB → Cough; Pollution (Dhaka air) → TB-এর independent contributor। Goal — cough দেখে TB-এর probability।

Python · pgmpy
# pip install pgmpy
from pgmpy.models import DiscreteBayesianNetwork
from pgmpy.factors.discrete import TabularCPD
from pgmpy.inference import VariableElimination

model = DiscreteBayesianNetwork([
    ("Smoking", "TB"),
    ("Pollution", "TB"),
    ("TB", "Cough"),
    ("TB", "XrayPositive"),
])

cpd_smoke = TabularCPD("Smoking", 2, [[0.7], [0.3]])  # 30% smoke
cpd_poll  = TabularCPD("Pollution", 2, [[0.4], [0.6]])  # 60% Dhaka heavy
cpd_tb    = TabularCPD("TB", 2,
    # P(TB | Smoking, Pollution) — order: (S=0,P=0), (S=0,P=1), (S=1,P=0), (S=1,P=1)
    values=[[0.99, 0.95, 0.90, 0.70],   # TB = no
            [0.01, 0.05, 0.10, 0.30]],  # TB = yes
    evidence=["Smoking", "Pollution"], evidence_card=[2, 2])
cpd_cough = TabularCPD("Cough", 2,
    values=[[0.80, 0.20],
            [0.20, 0.80]],
    evidence=["TB"], evidence_card=[2])
cpd_xray  = TabularCPD("XrayPositive", 2,
    values=[[0.95, 0.10],
            [0.05, 0.90]],
    evidence=["TB"], evidence_card=[2])

model.add_cpds(cpd_smoke, cpd_poll, cpd_tb, cpd_cough, cpd_xray)
print("Model valid:", model.check_model())

# Inference
infer = VariableElimination(model)

# Q1: P(TB | Cough = yes)
q1 = infer.query(["TB"], evidence={"Cough": 1})
print("\nP(TB | Cough=yes):", q1.values)

# Q2: P(TB | Cough = yes, XrayPositive = yes)
q2 = infer.query(["TB"], evidence={"Cough": 1, "XrayPositive": 1})
print("P(TB | Cough=yes, Xray+):", q2.values)

    
Cough alone — TB-এর probability marginally বাড়ে (~১৫%)। কিন্তু X-ray positive যোগ করলে — probability dramatically rise (~৭৫%)। Bayesian network multi-evidence smoothly integrate — doctor-এর reasoning-এর mathematical model।

৮ · Bayesian Network বনাম Naive Bayes

Naive Bayes (L18) একটি বিশেষ Bayesian Network — class node অন্য সব feature-এর parent, feature-গুলো একে অপরের সাথে independent (given class)।

  • Naive Bayes: simple, fast, high-bias। Class → all features (star structure)।
  • Tree-Augmented Naive Bayes (TAN): features-এর মধ্যে limited dependency।
  • General Bayesian Network: arbitrary DAG, any structure।

৯ · কখন Bayesian Network

  • Domain knowledge structured: medical, fault diagnosis।
  • Causal reasoning দরকার: "what if" interventions।
  • Missing data robust: evidence partial — inference কাজ করে।
  • Interpretable: graph + CPD — human readable।
  • Avoid: high-dim feature-এ overkill, deep learning ভাল।
Bayesian Network "correlation ≠ causation" সমস্যা সমাধান করে না — শুধু conditional independence encode। প্রকৃত causal inference-এ Pearl-এর do-calculus, structural causal models লাগে। কিন্তু Bayesian Network — সেই পথের প্রথম স্তর।

ভাবনার প্রশ্ন

প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।

প্র ০১ "Explain away" — V-structure-এর counter-intuitive behavior কী? Real-world implication?

চমৎকার Bayesian Network-এর সবচেয়ে gripping concept।

Setup — Burglary alarm:

  • Burglary (B) — independent।
  • Earthquake (E) — independent।
  • Alarm (A) — depends on B এবং E।
  • B ও E marginally independent (no edge)।

Marginal:

  • P(B) = 0.001।
  • P(E) = 0.002।
  • P(B, E) = P(B) × P(E) = 0.000002 (independent)।

Alarm given:

  • Alarm বাজছে — B অথবা E (অথবা দু'টি)।
  • P(B | A) — B-এর probability বাড়ে (এটা causal contributor)।
  • P(B | A, E) — E দেখানে যাচ্ছে cause। B-এর probability কমে — "explained away"।
  • Counter-intuitive: B ও E marginally independent ছিল, কিন্তু A দেখে — দু'টি negatively correlated।

Mathematical:

  • P(B | A, E) < P(B | A) — explaining away।
  • একটি cause confirm — অন্যটির need কমে।

Real-world implications:

(১) Medical diagnosis:

  • Patient cough — TB or pneumonia?
  • Pneumonia confirmed → TB-এর posterior কমে।
  • একটি সঠিক diagnosis অন্যকে rule out।

(২) Fault detection:

  • Server down — disk failure or network issue?
  • Network confirmed bad — disk-এর probability decline।

(৩) Causal misinterpretation:

  • "Smart students attend top universities" — talent ও hard work both contribute।
  • Top university seen — talent confirmed → hard work-এর likelihood কমে?
  • দু'টি skill-ই দেখা যায় না collider-এর through।

Berkson's paradox:

  • Hospital data — sick + injured patients overrepresented।
  • Hospitalization status (collider) selecting on।
  • Sickness ও injury seem negatively correlated in hospital, but not in population।

Causal AI critical:

  • Conditioning on collider creates spurious correlation।
  • "Controlling for everything" can introduce bias, not remove।
  • Pearl's "Book of Why" — central theme।

Practical:

  • Statistical model — be careful selecting variables।
  • Causal DAG draw — collider identify।
  • Don't condition on collider.

Bangladesh case:

  • Loan approval data — only approved loans visible।
  • Approval (collider) → income ও credit score correlation distorted।
  • Counter-intuitive — high income borrowers default more in dataset (selection bias)।

মূল উপলব্ধি: Marginal independence ≠ conditional independence। Bayesian Network — এই subtle relationship visualize ও exploit। Real causal reasoning-এর foundation।

প্র ০২ Bayesian Network বনাম neural network — কোথায় কোনটা? Hybrid possible?

ভিন্ন paradigm — শক্তি ও দুর্বলতা complementary।

Bayesian Network strength:

  • Interpretable — graph human-readable।
  • Causal reasoning — interventions।
  • Missing data graceful — partial evidence inference।
  • Uncertainty native — probability output।
  • Domain knowledge integrate — expert prior।
  • Small data effective।

Bayesian Network weakness:

  • Discrete/few-state continuous — high-D continuous struggle।
  • Structure learning intractable।
  • Inference exponential worst-case।
  • Scaling — million-row hard।

Neural network strength:

  • Big data shine।
  • High-D, continuous — image, text।
  • Feature learning automatic।
  • End-to-end optimization।
  • State-of-art performance।

Neural network weakness:

  • Black box — interpretability difficult।
  • Causal — correlation only।
  • Missing data — imputation needed।
  • Uncertainty — calibration extra effort।
  • Domain knowledge integrate hard।
  • Small data — overfit risk।

কখন Bayesian Network:

  • Medical diagnosis — interpretability critical।
  • Risk assessment — uncertainty quantified।
  • Fault diagnosis — causal chain।
  • Decision support — domain expert involve।
  • Small structured data।

কখন neural network:

  • Image classification — pixels থেকে category।
  • Speech recognition।
  • NLP — translation, summarization।
  • Recommendation system — million users।
  • End-to-end learning।

Hybrid approaches:

(১) Bayesian deep learning:

  • Neural network weights — distribution, not point।
  • Variational inference, MCMC।
  • Uncertainty in deep models।
  • BNN, Bayes by Backprop।

(২) Probabilistic programming:

  • PyMC, Pyro, Stan।
  • Bayesian model — neural network components allowed।
  • Best of both।

(৩) Deep generative model:

  • VAE — neural encoder + probabilistic latent।
  • Normalizing flows।
  • Diffusion models।

(৪) Causal neural network:

  • Causal discovery via NN (NOTEARS algorithm)।
  • Counterfactual reasoning ML।
  • Pearl's framework + deep learning।

(৫) Neuro-symbolic AI:

  • Symbolic structure (Bayesian Network) + neural feature।
  • DeepProbLog।
  • Promising frontier।

Bangladesh applications:

  • Loan default — Bayesian Network for feature relationship + neural for transaction sequence।
  • Disease diagnosis — knowledge graph + image CNN।
  • Crop yield — weather BN + drone image NN।

মূল উপলব্ধি: Bayesian Network knowledge engineering। Neural network end-to-end learning। Real-world AI — both, not either-or।

প্র ০৩ Structure learning কেন NP-hard? Practical heuristics কী?

DAG-এর search space combinatorial — তাই NP-hardness।

Search space size:

  • $n$ node-এর জন্য DAG count super-exponential।
  • $n=5$ → 29281, $n=10$ → 4.18 × 10¹⁸, $n=20$ → 2.34 × 10⁷²।
  • Robinson formula।

NP-hardness proof:

  • Chickering (১৯৯৬) — minimum description length DAG NP-hard।
  • Bayesian score optimal DAG NP-complete।

Approach categories:

(১) Score-based:

  • BIC, AIC, BDe — DAG-এর "fit" score।
  • Greedy hill climbing — random DAG → local edge addition/removal।
  • Tabu search, simulated annealing।
  • Local optima trap risk।

(২) Constraint-based:

  • PC algorithm (Spirtes-Glymour-Scheines)।
  • Conditional independence tests (chi-square, mutual info)।
  • Independence pattern → DAG skeleton।
  • V-structure orientation rules।

(৩) Hybrid:

  • MMHC — Max-Min Hill Climbing।
  • Constraint-based skeleton + score-based orientation।

(৪) Continuous optimization:

  • NOTEARS (২০১৮) — DAG constraint as smooth penalty।
  • Gradient-based optimization।
  • Modern, scalable।

(৫) Bayesian:

  • Posterior over DAGs।
  • MCMC sampling structures।
  • Order MCMC — efficient।

Practical tricks:

  • Domain knowledge: structural prior — known edges।
  • Variable ordering: if known, search reduces।
  • Sparsity prior: few-edge DAG preferred।
  • Bootstrap: resampling stable edges।

Software:

  • bnlearn (R) — comprehensive।
  • pgmpy — Python।
  • causal-learn — modern।
  • NOTEARS — neural causal discovery।

Validation:

  • Held-out likelihood।
  • Domain expert review।
  • Causal interpretation sanity check।

Open problems:

  • Latent confounder — fundamental ambiguity।
  • Causal direction from observational data — sometimes impossible।
  • Equivalence class — multiple DAGs same independencies।

মূল উপলব্ধি: Structure learning hard, but practical heuristics work decently। Domain expert সবচেয়ে valuable input। Pure data-driven discovery এখনও open research।

প্র ০৪ Bangladesh-এর rural health centers-এ Bayesian Network deploy — TB, dengue, typhoid diagnosis support। Pipeline ও challenges?

Real Bangladesh public health impact — clinical decision support system।

Use case:

  • Rural community health worker — patient evaluate।
  • Symptom + basic test → likely diagnosis suggestion।
  • Doctor unavailable area — first-line guidance।

(১) Diseases ও symptoms:

  • TB: cough, fever, weight loss, night sweat, X-ray।
  • Dengue: high fever, headache, rash, platelet drop।
  • Typhoid: prolonged fever, abdominal pain, Widal test।
  • Common cold, malaria — confounders।

(২) Bayesian Network structure:

  • Risk factors (smoking, mosquito exposure, season) → diseases।
  • Disease → symptoms।
  • Disease → test results।
  • Tree-like structure mostly।

(৩) CPT acquisition:

  • Medical literature — base rates।
  • Hospital data — Bangladesh prevalence।
  • Expert elicitation — doctor input।
  • Local prior — district-specific (dengue urban, malaria hill area)।

(৪) Inference:

  • Variable elimination — exact, suitable for ৫-১০ disease network।
  • Real-time on tablet feasible।

(৫) UI/UX:

  • Bangla-only interface।
  • Symptom checkbox।
  • Test result entry।
  • Top-3 diagnosis with probability।
  • Explanation — "fever + low platelet → dengue likely"।

(৬) Validation:

  • Retrospective hospital records — accuracy benchmark।
  • Doctor agreement study।
  • Pilot at health center — feedback।

(৭) Challenges:

  • Data quality: rural health record incomplete।
  • Local prevalence: haor area different from char area।
  • Atypical presentation: subset symptom only।
  • Co-infection: TB + diabetes common।
  • Cultural: patient symptom report bias।
  • Connectivity: offline-first requirement।
  • Trust: health worker AI skepticism।
  • Liability: wrong suggestion legal/medical।

(৮) Ethical:

  • Decision support, not replacement।
  • Probabilistic uncertainty visible।
  • "When in doubt, refer to doctor" prominent।
  • Health worker training mandatory।

(৯) Continuous improvement:

  • Regular CPT update — new prevalence data।
  • Feedback loop — confirmed diagnosis।
  • Drift detection — symptom pattern shift।

(১০) Beyond Bayesian Network:

  • X-ray AI (CNN) — TB detection।
  • Smartphone fever camera — temperature।
  • Hybrid — Bayesian aggregation, neural feature।

Stakeholders:

  • icddr,b — medical research।
  • BRAC — health worker network।
  • DGHS — government endorsement।
  • WHO Bangladesh — international support।

মূল উপলব্ধি: Bayesian Network — Bangladesh rural health-এ tangible AI। Interpretability + uncertainty + small data — neural network-এর তুলনায় ideal। Implementation engineering + clinical partnership — সমান গুরুত্বপূর্ণ।

অনুশীলন

  1. হিসাব করুন: $A \to B \to C$ chain। P(A=T)=0.5, P(B=T|A=T)=0.8, P(B=T|A=F)=0.2, P(C=T|B=T)=0.9, P(C=T|B=F)=0.1। P(C=T) কত?
    • P(B=T) = P(B=T|A=T)P(A=T) + P(B=T|A=F)P(A=F) = 0.8×0.5 + 0.2×0.5 = 0.5।
    • P(C=T) = P(C=T|B=T)P(B=T) + P(C=T|B=F)P(B=F) = 0.9×0.5 + 0.1×0.5 = 0.5।
  2. pgmpy-এ চেষ্টা: উপরের chain network কোড করুন।
    from pgmpy.models import DiscreteBayesianNetwork
    from pgmpy.factors.discrete import TabularCPD
    from pgmpy.inference import VariableElimination
    
    m = DiscreteBayesianNetwork([("A", "B"), ("B", "C")])
    m.add_cpds(
        TabularCPD("A", 2, [[0.5], [0.5]]),
        TabularCPD("B", 2, [[0.8, 0.2], [0.2, 0.8]],
                   evidence=["A"], evidence_card=[2]),
        TabularCPD("C", 2, [[0.1, 0.9], [0.9, 0.1]],
                   evidence=["B"], evidence_card=[2]),
    )
    infer = VariableElimination(m)
    print(infer.query(["C"]).values)
  3. ভাবুন: Bangladesh নৌকা-দুর্ঘটনা risk model — weather, overcrowding, pilot experience, equipment quality। Bayesian Network DAG আঁকুন।
    • Weather (storm) → Wave height।
    • Pilot experience → Pilot decisions।
    • Equipment quality → Mechanical failure risk।
    • Overcrowding → Stability।
    • {Wave height, Pilot decisions, Mechanical failure, Stability} → Accident।
    • Risk factors independent (mostly), accident V-structure collider।
    • Use: probability of accident inference, intervention prioritize।
    • Action: equipment improvement vs pilot training cost-benefit।

আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

কোড রানার কাজ না করলে? Google Colab ব্যবহার করুন।
পূর্ববর্তী পাঠ
পাঠ ৩৫ · t-SNE ও UMAP