পাঠ ৩৯ · ৪৫-এর মধ্যে · মডিউল ৫
Home / AI Courses / Machine Learning / EM

EM অ্যালগরিদম

Expectation-Maximization — latent variable MLE
৮ মিনিট পড়া উচ্চ · Advanced NumPy / sklearn

এই পাঠে যা শিখবেন

  • Latent variable model — কেন direct MLE কঠিন
  • EM-এর E ও M ধাপ — সাধারণ template
  • Gaussian Mixture Model (GMM) — EM-এর canonical example
  • Convergence proof — Jensen's inequality ও ELBO
  • K-Means EM-এর hard limit; modern variational EM

১ · Latent variable problem

Standard MLE — observed data $\mathbf{X}$, parameter $\boldsymbol{\theta}$ — $\theta^* = \arg\max_\theta \log P(\mathbf{X} \mid \theta)$। Closed-form (linear regression) বা gradient descent।

কিন্তু অনেক model-এ latent variable $\mathbf{Z}$ — directly observed না। Marginal log-likelihood:

$$\log P(\mathbf{X} \mid \theta) = \log \sum_{\mathbf{Z}} P(\mathbf{X}, \mathbf{Z} \mid \theta)$$

Sum (or integral) inside log — gradient hard, no closed-form। উদাহরণ:

  • GMM: প্রতিটি point কোন Gaussian থেকে এসেছে — latent।
  • HMM: hidden state sequence latent (L37-৩৮)।
  • Missing data: survey-এ unanswered field।
  • Topic model (LDA): document-এর topic mixture latent।
EM-এর core idea

Latent $\mathbf{Z}$ থাকলে — yes "observe" করো, পদে বুঝো (E-step: posterior)। তারপর সেই assumed $\mathbf{Z}$ ধরে normal MLE (M-step)। বুঝে যাচ্ছো assumed $\mathbf{Z}$ পরিবর্তিত হচ্ছে — তাই iterate।

২ · EM algorithm — সাধারণ form

Initialize: $\theta^{(0)}$ random।

Repeat until convergence:

E-step: compute posterior over latents:

$$q^{(t)}(\mathbf{Z}) = P(\mathbf{Z} \mid \mathbf{X}, \theta^{(t)})$$

M-step: maximize expected complete-data log-likelihood:

$$\theta^{(t+1)} = \arg\max_\theta \mathbb{E}_{q^{(t)}} \left[ \log P(\mathbf{X}, \mathbf{Z} \mid \theta) \right]$$

এক formulation-এ — "Q function":

$$Q(\theta \mid \theta^{(t)}) = \sum_{\mathbf{Z}} P(\mathbf{Z} \mid \mathbf{X}, \theta^{(t)}) \log P(\mathbf{X}, \mathbf{Z} \mid \theta)$$

৩ · GMM — canonical EM example

$K$ Gaussian-এর mixture: $P(\mathbf{x}) = \sum_{k=1}^K \pi_k \mathcal{N}(\mathbf{x} \mid \boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k)$।

প্রতিটি $\mathbf{x}_i$-এর latent: $z_i \in \{1, \ldots, K\}$ — কোন Gaussian থেকে এসেছে।

E-step (responsibility):

$$\gamma_{ik} = P(z_i = k \mid \mathbf{x}_i, \theta) = \frac{\pi_k \mathcal{N}(\mathbf{x}_i \mid \boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k)}{\sum_j \pi_j \mathcal{N}(\mathbf{x}_i \mid \boldsymbol{\mu}_j, \boldsymbol{\Sigma}_j)}$$

M-step (re-estimate):

$$N_k = \sum_i \gamma_{ik}, \quad \pi_k = \frac{N_k}{N}$$

$$\boldsymbol{\mu}_k = \frac{1}{N_k} \sum_i \gamma_{ik} \mathbf{x}_i$$

$$\boldsymbol{\Sigma}_k = \frac{1}{N_k} \sum_i \gamma_{ik} (\mathbf{x}_i - \boldsymbol{\mu}_k)(\mathbf{x}_i - \boldsymbol{\mu}_k)^\top$$

এটি weighted-mean ও weighted-covariance। K-Means-এর soft version — point-গুলো partial membership সব cluster-এ।

৪ · Convergence — Jensen's inequality

EM monotonic — log-likelihood প্রতি iteration-এ বাড়ে বা একই থাকে। Proof outline:

For any distribution $q$:

$$\log P(\mathbf{X} \mid \theta) = \log \sum_{\mathbf{Z}} q(\mathbf{Z}) \frac{P(\mathbf{X}, \mathbf{Z} \mid \theta)}{q(\mathbf{Z})} \geq \sum_{\mathbf{Z}} q(\mathbf{Z}) \log \frac{P(\mathbf{X}, \mathbf{Z} \mid \theta)}{q(\mathbf{Z})}$$

Right-hand side — ELBO (Evidence Lower BOund)। Equality when $q(\mathbf{Z}) = P(\mathbf{Z} \mid \mathbf{X}, \theta)$ — exactly E-step পদক্ষেপ।

E-step: $q$ চয়ন → ELBO touch log-likelihood।
M-step: $\theta$ change → ELBO maximize → log-likelihood (ELBO-এর উপরে) automatically increase।

৫ · K-Means as hard EM

K-Means (L31) আসলে GMM-এর hard EM:

  • Equal $\pi_k$, equal isotropic $\boldsymbol{\Sigma}_k = \sigma^2 \mathbf{I}$।
  • $\sigma \to 0$ — responsibility hard {0, 1}।
  • প্রতিটি point একটি cluster-এ deterministic।

GMM = soft K-Means; K-Means = degenerate GMM। Soft probability → hard membership।

EM iteration — log L monotonic increase E: q(Z); M: arg max θ E_q[log P(X,Z|θ)] Initialize θ⁽⁰⁾ random E-step q(Z) = P(Z | X, θ) latent posterior M-step θ = argmax E_q[log P(X,Z|θ)] parameter update repeat log L monotonic increase iteration → log L converge to local max
EM — alternating E (latent posterior) ও M (parameter MLE)। প্রতি iteration log-likelihood monotonic — local maximum-এ converge।

৬ · sklearn-এ GMM

Python · sklearn 
import numpy as np
from sklearn.mixture import GaussianMixture

# Synthetic 2-cluster Gaussian
np.random.seed(0)
X = np.concatenate([
    np.random.normal([0, 0],  [1, 1], (300, 2)),
    np.random.normal([5, 5],  [1, 1], (300, 2)),
    np.random.normal([0, 6],  [0.5, 1], (200, 2)),
])

gmm = GaussianMixture(n_components=3, covariance_type="full",
                      max_iter=100, random_state=0)
gmm.fit(X)

print(f"Means:\n{gmm.means_.round(2)}")
print(f"Weights π: {gmm.weights_.round(3)}")
print(f"Converged: {gmm.converged_}")
print(f"# iterations: {gmm.n_iter_}")
print(f"Log-likelihood: {gmm.score(X) * len(X):.2f}")

# soft assignment (responsibility)
gamma = gmm.predict_proba(X[:5])
print(f"\nFirst 5 sample's responsibility:\n{gamma.round(3)}")

    


    

      Means প্রায় (0,0), (5,5), (0,6) — true centers recover। predict_proba — প্রতি point-এর soft membership। K-Means-এর হার্ড {0, 1}-এর বদলে।
    


    
৭ · NumPy scratch — GMM EM


    

      

        
        Python · NumPy
      

      
import numpy as np
from scipy.stats import multivariate_normal

def gmm_em(X, K, n_iter=50):
    n, d = X.shape
    # init
    rng = np.random.default_rng(0)
    pi = np.ones(K) / K
    mu = X[rng.choice(n, K, replace=False)]
    Sigma = np.array([np.eye(d)] * K)

    log_likelihoods = []
    for it in range(n_iter):
        # E-step: responsibility
        resp = np.zeros((n, K))
        for k in range(K):
            resp[:, k] = pi[k] * multivariate_normal.pdf(X, mu[k], Sigma[k])
        log_lik = np.log(resp.sum(axis=1) + 1e-300).sum()
        log_likelihoods.append(log_lik)
        resp /= resp.sum(axis=1, keepdims=True)

        # M-step
        Nk = resp.sum(axis=0)
        pi = Nk / n
        mu = (resp.T @ X) / Nk[:, None]
        for k in range(K):
            diff = X - mu[k]
            Sigma[k] = (resp[:, k:k+1] * diff).T @ diff / Nk[k]

    return pi, mu, Sigma, log_likelihoods

# 1D synthetic
X = np.concatenate([
    np.random.normal(0, 1, 200),
    np.random.normal(5, 1, 200),
]).reshape(-1, 1)

pi, mu, S, ll = gmm_em(X, K=2, n_iter=30)
print(f"π: {pi.round(3)}")
print(f"μ: {mu.ravel().round(3)}")
print(f"Σ: {S.ravel().round(3)}")
print(f"Final log L: {ll[-1]:.2f}")
print(f"Monotonic? {all(ll[i] <= ll[i+1] + 1e-6 for i in range(len(ll)-1))}")

      

        
        
      

      

    


    

      Output-এ μ ≈ ০ ও ৫ — true cluster recover। Log-likelihood প্রতি iteration বাড়ে — monotonic property verify।
    


    
৮ · EM-এর প্রয়োগ

    
    
      
  • GMM: soft clustering, density estimation।
    • 
      
  • HMM (Baum-Welch): sequence model — L38।
    • 
      
  • Missing data: Rubin's MI, EM imputation।
    • 
      
  • Factor analysis ও PCA-এর probabilistic version।
    • 
      
  • LDA topic model: document-topic distribution latent।
    • 
      
  • Item Response Theory: education assessment।
    • 
      
  • Mixture of experts: neural model gating।
    • 
    


    
৯ · Modern variants

    
    
      
  • Variational EM: intractable posterior — variational approximation।
    • 
      
  • Stochastic EM: minibatch — large data।
    • 
      
  • Generalized EM: M-step partial maximize।
    • 
      
  • Online EM: streaming data।
    • 
      
  • VAE (Variational Autoencoder): EM + neural network — modern probabilistic deep learning।
    • 
    


    

      EM local maximum — global guarantee নেই। Multiple init essential। GMM-এ degenerate solution-ও possible (single point cluster, $\Sigma \to 0$, likelihood $\to \infty$)। Regularization বা minimum eigenvalue prior অপরিহার্য।
    


  


  

    
    
  


  
  

    
 ভাবনার প্রশ্ন

    
প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।


    

      
        প্র ০১
        EM monotonic increase — তবু local maximum। Gradient ascent vs EM — কখন কোনটা?
      
      

        

          চমৎকার comparison। দু'টি local maximization technique।
        

        

          EM strength:
        

        
    
          
  • Closed-form M-step (often) — fast convergence।
    • 
          
  • Monotonic guarantee — no learning rate tuning।
    • 
          
  • Numerical stability।
    • 
          
  • Latent-variable model natural।
    • 
          
  • Complete-data likelihood often easier than marginal।
    • 
        

        

          EM weakness:
        

        
    
          
  • M-step closed-form not always — generalized EM, slower।
    • 
          
  • Local optima ।
    • 
          
  • E-step intractable for complex model — variational।
    • 
          
  • Slow asymptotic — linear convergence (gradient quadratic for Newton)।
    • 
        

        

          Gradient ascent strength:
        

        
    
          
  • Universal — যেকোনো differentiable likelihood।
    • 
          
  • Stochastic variant (SGD) — large data scaling।
    • 
          
  • Modern automatic differentiation (PyTorch, JAX)।
    • 
          
  • Quadratic convergence with Newton/quasi-Newton।
    • 
        

        

          Gradient weakness:
        

        
    
          
  • Learning rate tuning।
    • 
          
  • Numerical issues — log-sum-exp, NaN।
    • 
          
  • Constraint handling — projection, transforms।
    • 
          
  • Latent variable indirect — requires derivative through latent।
    • 
        

        

          কখন EM:
        

        
    
          
  • GMM, HMM, mixture model — closed-form M-step।
    • 
          
  • Discrete latent — sum tractable।
    • 
          
  • Small to medium data।
    • 
          
  • Probabilistic interpretation important।
    • 
        

        

          কখন gradient:
        

        
    
          
  • Neural network — non-conjugate distribution।
    • 
          
  • Continuous high-D latent — variational।
    • 
          
  • Big data — SGD parallelizable।
    • 
          
  • Custom loss function।
    • 
        

        

          Hybrid:
        

        
    
          
  • Variational EM — gradient inside E-step।
    • 
          
  • Amortized inference — neural network for E-step।
    • 
          
  • VAE = neural variational EM।
    • 
        

        

          Empirical:
        

        
    
          
  • GMM — EM standard, gradient possible but rarely used।
    • 
          
  • Topic model — variational EM standard, MCMC alternative।
    • 
          
  • HMM — Baum-Welch (EM) classical; gradient via backprop modern।
    • 
          
  • Deep generative — gradient (VAE, GAN, diffusion)।
    • 
        

        

          Convergence speed:
        

        
    
          
  • EM — linear (slow asymptotically)।
    • 
          
  • Newton — quadratic (fast)।
    • 
          
  • SGD — sub-linear (slow but cheap per step)।
    • 
        

        

          মূল উপলব্ধি: EM elegant for structured probabilistic model। Gradient ascent universal for arbitrary loss। Modern ML — combo: variational EM, neural latent। Choose tool by problem structure।
        

      

    


    

      
        প্র ০২
        ELBO কী? VAE-তে এর ভূমিকা?
      
      

        

          ELBO — modern Bayesian deep learning-এর central concept।
        

        

          Definition:
        

        
    
          
  • Evidence Lower BOund।
    • 
          
  • Marginal log-likelihood-এর lower bound।
    • 
          
  • $\log P(\mathbf{X}) \geq \mathcal{L}(q, \theta) = \mathbb{E}_q[\log P(\mathbf{X}, \mathbf{Z}|\theta)] - \mathbb{E}_q[\log q(\mathbf{Z})]$।
    • 
          
  • Equivalent: $\mathcal{L} = \log P(\mathbf{X}) - \text{KL}(q \| P(\mathbf{Z}|\mathbf{X}))$।
    • 
        

        

          EM-এর role:
        

        
    
          
  • E-step: $q(\mathbf{Z}) = P(\mathbf{Z}|\mathbf{X}, \theta)$ — KL=0, ELBO = log-likelihood।
    • 
          
  • M-step: ELBO maximize over $\theta$ — log-likelihood lower bound bumped up।
    • 
        

        

          Variational inference:
        

        
    
          
  • Posterior $P(\mathbf{Z}|\mathbf{X})$ intractable।
    • 
          
  • $q(\mathbf{Z})$ family চয়ন (Gaussian) — তার মধ্যে ELBO maximize।
    • 
          
  • KL divergence minimize — posterior approximation।
    • 
        

        

          VAE — ELBO neural:
        

        
    
          
  • $q_\phi(\mathbf{z}|\mathbf{x})$ — encoder neural network।
    • 
          
  • $P_\theta(\mathbf{x}|\mathbf{z})$ — decoder neural network।
    • 
          
  • ELBO loss:
    • 
          
  • $\mathcal{L} = \mathbb{E}_{q_\phi}[\log P_\theta(\mathbf{x}|\mathbf{z})] - \text{KL}(q_\phi(\mathbf{z}|\mathbf{x}) \| P(\mathbf{z}))$।
    • 
          
  • First term — reconstruction।
    • 
          
  • Second — regularization (latent prior match)।
    • 
        

        

          Reparameterization trick:
        

        
    
          
  • $\mathbf{z} = \mu + \sigma \cdot \epsilon, \epsilon \sim \mathcal{N}(0, I)$।
    • 
          
  • Stochastic node deterministic + noise।
    • 
          
  • Gradient backprop through।
    • 
        

        

          Diffusion models:
        

        
    
          
  • Hierarchical VAE — multi-step latent।
    • 
          
  • ELBO simplify হয়ে denoising score matching।
    • 
          
  • State-of-art generation (Stable Diffusion, DALL-E 3)।
    • 
        

        

          Why ELBO popular:
        

        
    
          
  • Tractable lower bound।
    • 
          
  • Gradient-friendly।
    • 
          
  • Probabilistic interpretation retained।
    • 
          
  • Uncertainty quantification।
    • 
          
  • Scalable to big data + neural networks।
    • 
        

        

          Limitations:
        

        
    
          
  • Lower bound — not exact log-likelihood।
    • 
          
  • $q$ family choice biases।
    • 
          
  • "Posterior collapse" in VAE (decoder ignore latent)।
    • 
          
  • Tight bound require complex $q$।
    • 
        

        

          Modern advances:
        

        
    
          
  • Normalizing flows — flexible $q$।
    • 
          
  • Importance-weighted ELBO — tighter bound।
    • 
          
  • SVI (stochastic variational inference) — minibatch।
    • 
        

        

          মূল উপলব্ধি: ELBO — EM-এর modern reformulation, neural network-এ scale। VAE generative AI-র foundation। Bayesian + deep — ELBO setiqu pillar।
        

      

    


    

      
        প্র ০৩
        GMM-এ singular Gaussian (Σ → 0) — degenerate solution। কীভাবে এড়ান?
      
      

        

          GMM EM-এর notorious failure mode।
        

        

          Problem mechanism:
        

        
    
          
  • একটি Gaussian center একটি data point-এ।
    • 
          
  • Variance shrink — likelihood explode।
    • 
          
  • EM trap এই solution।
    • 
          
  • Likelihood unbounded।
    • 
        

        

          Why happens:
        

        
    
          
  • $K$ too large for data।
    • 
          
  • Initialization bad।
    • 
          
  • MLE without regularization unbounded।
    • 
        

        

          Solutions:
        

        

          (১) Covariance regularization:
        

        
    
          
  • $\boldsymbol{\Sigma}_k \to \boldsymbol{\Sigma}_k + \lambda \mathbf{I}$।
    • 
          
  • Minimum eigenvalue floor।
    • 
          
  • sklearn-এ reg_covar=1e-6।
    • 
        

        

          (২) Bayesian prior:
        

        
    
          
  • Inverse-Wishart prior on $\boldsymbol{\Sigma}$।
    • 
          
  • MAP estimate instead of MLE।
    • 
          
  • Singular solution penalized।
    • 
        

        

          (৩) Restart heuristic:
        

        
    
          
  • Detect singular component।
    • 
          
  • Re-initialize that component random।
    • 
          
  • Continue EM।
    • 
        

        

          (৪) Constraint:
        

        
    
          
  • Tied covariance — all components share $\boldsymbol{\Sigma}$।
    • 
          
  • Spherical — $\boldsymbol{\Sigma}_k = \sigma_k^2 \mathbf{I}$।
    • 
          
  • Diagonal — uncorrelated dimensions।
    • 
        

        

          (৫) Cross-validated $K$:
        

        
    
          
  • $K$ too high → singular common।
    • 
          
  • Validation likelihood best $K$।
    • 
          
  • BIC penalize complexity।
    • 
        

        

          (৬) Variational Bayes GMM:
        

        
    
          
  • Dirichlet process prior — automatic $K$।
    • 
          
  • Sparse component — irrelevant ones zero out।
    • 
          
  • sklearn-এ BayesianGaussianMixture।
    • 
        

        

          Detection:
        

        
    
          
  • Component covariance condition number monitor।
    • 
          
  • Likelihood unrealistically high — flag।
    • 
          
  • Tiny effective sample $N_k$ — warning।
    • 
        

        

          sklearn defaults:
        

        
    
          
  • reg_covar default 1e-6।
    • 
          
  • Multiple init (n_init)।
    • 
          
  • Warning if component small।
    • 
        

        

          Production tips:
        

        
    
          
  • Always reg_covar > 0।
    • 
          
  • Bayesian variant — robust automatic।
    • 
          
  • BIC-cv $K$ selection।
    • 
          
  • Multiple init essential।
    • 
        

        

          Bangladesh case:
        

        
    
          
  • Small dataset — fewer cluster, more regularization।
    • 
          
  • Medical image segmentation — 4-6 component, prior regularize।
    • 
          
  • Customer segmentation — 5-10 component, BIC select।
    • 
        

        

          মূল উপলব্ধি: Pure MLE GMM unstable। Regularization or Bayesian — production essential। Singular solution-এর likelihood meaningless — algorithm trap, not insight।
        

      

    


    

      
        প্র ০৪
        Bangladesh-এর rural household survey-এ missing data prevalent — EM-based imputation কীভাবে?
      
      

        

          Real public health/policy challenge — survey data quality।
        

        

          Missing data types (Rubin):
        

        
    
          
  • MCAR: Missing Completely At Random — survey lost page।
    • 
          
  • MAR: Missing At Random — given observed, missing predictable। Income missing more in rural।
    • 
          
  • MNAR: Missing Not At Random — high earner refuse income disclose।
    • 
        

        

          Naive approaches:
        

        
    
          
  • Drop missing rows — sample reduce, bias if not MCAR।
    • 
          
  • Mean imputation — variance underestimate।
    • 
          
  • Mode/median — categorical।
    • 
          
  • "Missingness indicator" — flag missing।
    • 
        

        

          EM-based imputation:
        

        
    
          
  1. Model joint distribution $P(\mathbf{X})$ — multivariate normal common।
    2. 
          
  2. E-step: missing values' expected value given observed।
    3. 
          
  3. M-step: parameters update with completed data।
    4. 
          
  4. Iterate to convergence।
    5. 
          
  5. Final imputed value — posterior mean।
    6. 
        

        

          Multivariate Gaussian EM:
        

        
    
          
  • Parameter $\boldsymbol{\mu}, \boldsymbol{\Sigma}$।
    • 
          
  • Conditional Gaussian — partition observed and missing।
    • 
          
  • Closed-form E-step।
    • 
        

        

          Multiple Imputation (Rubin):
        

        
    
          
  • $M$ different imputations sample।
    • 
          
  • Analyze each separately।
    • 
          
  • Pool result (Rubin's rule) — uncertainty proper।
    • 
          
  • Better than single EM imputation।
    • 
        

        

          Pipeline for Bangladesh survey:
        

        
    
          
  1. Exploratory: missingness pattern (visualize)।
    2. 
          
  2. Mechanism assessment: Little's MCAR test।
    3. 
          
  3. Variable transformation: log income, normalize age।
    4. 
          
  4. EM imputation: mvnorm or chained equations।
    5. 
          
  5. Validation: hold-out test।
    6. 
          
  6. Sensitivity analysis: imputation choice impact।
    7. 
        

        

          Software:
        

        
    
          
  • sklearn IterativeImputer — MICE।
    • 
          
  • R mice — gold standard।
    • 
          
  • Python fancyimpute।
    • 
          
  • missForest — non-parametric।
    • 
        

        

          Bangladesh context:
        

        
    
          
  • HIES (Household Income Expenditure Survey) — significant missing।
    • 
          
  • BBS census — large but missing in remote areas।
    • 
          
  • HDI calculation — imputation impacts national stats।
    • 
          
  • Demographic and Health Survey (BDHS)।
    • 
        

        

          Domain considerations:
        

        
    
          
  • Religion, ethnicity — sensitive, imputation politically charged।
    • 
          
  • Income — heavy tail, log transform essential।
    • 
          
  • Age — typically observed reliably।
    • 
          
  • Education, occupation — categorical।
    • 
        

        

          Validation methods:
        

        
    
          
  • Hold out fully observed records as missing.
    • 
          
  • Imputation error metric।
    • 
          
  • Downstream analysis sensitivity।
    • 
        

        

          Caveats:
        

        
    
          
  • MNAR — EM cannot fix without model।
    • 
          
  • Joint distribution assumption strong।
    • 
          
  • Bias in imputed values — careful interpretation।
    • 
          
  • Privacy — imputed value, not synthetic।
    • 
        

        

          Modern alternative:
        

        
    
          
  • GAIN — generative adversarial imputation।
    • 
          
  • VAE-based — latent variable principled।
    • 
          
  • Transformer-based — contextual imputation।
    • 
        

        

          মূল উপলব্ধি: Missing data inevitable in Bangladesh field surveys। EM imputation principled approach — assumption-aware, validation-driven। Multiple imputation gold standard for analysis, single imputation only for prediction। Public policy data integrity critical।
        

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        হিসাব করুন: 1D 2-Gaussian mixture, π=(0.5, 0.5), μ=(0, 5), σ=(1, 1)। Point x=2-এর responsibility?
        
    
          
          
    
            
      
              
    • $\mathcal{N}(2 | 0, 1) = \frac{1}{\sqrt{2\pi}} e^{-2} \approx 0.054$।
      • 
              
    • $\mathcal{N}(2 | 5, 1) = \frac{1}{\sqrt{2\pi}} e^{-4.5} \approx 0.00443$।
      • 
              
    • γ₁ = 0.5×0.054 / (0.5×0.054 + 0.5×0.00443) ≈ 0.924।
      • 
              
    • γ₂ ≈ 0.076।
      • 
              
    • Point 2 mostly cluster 1।
      • 
            
    
          
    
        
    
      
    2. 

      
  2. 
        sklearn-এ চেষ্টা: Bangladesh income distribution — 2-component GMM fit।
        
    
          
          
    
            
    from sklearn.mixture import GaussianMixture
import numpy as np

# Synthetic bimodal income (low + middle income clusters)
np.random.seed(0)
income = np.concatenate([
    np.random.lognormal(mean=8.5, sigma=0.4, size=700),  # low
    np.random.lognormal(mean=10.0, sigma=0.5, size=300), # middle
])
X = income.reshape(-1, 1)

gmm = GaussianMixture(n_components=2, random_state=0).fit(X)
print(f"Means: {gmm.means_.ravel().round(0)}")
print(f"Weights: {gmm.weights_.round(3)}")
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Bangladesh-এর hospital-এর patient symptoms (categorical) — Latent Class Analysis (categorical EM)। কী useful?
        
    
          
          
    
            
      
              
    • Idea: latent disease group, observed symptom pattern।
      • 
              
    • EM: E-step — patient-disease responsibility। M-step — disease symptom probability।
      • 
              
    • Use: data-driven disease subtype discovery।
      • 
              
    • Bangladesh: dengue, typhoid mixed — separate cluster?
      • 
              
    • Validation: doctor agreement।
      • 
              
    • Limit: co-morbidity overlap, atypical presentation।
      • 
              
    • Action: diagnostic guideline refine।
      • 
            
    
          
    
        
    
      
    4. 
    

  


  
  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    

    

      কোড রানার কাজ না করলে? Google Colab ব্যবহার করুন।
    

  








  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ৩৮ · Viterbi ও Baum-Welch