পাঠ ২৯ · ৩৫-এর মধ্যে · মডিউল ৪
Home / AI Courses / Computer Vision / Stable Diffusion

Stable Diffusion — image generation

Stable Diffusion — open-source text-to-image generation
৯ মিনিট পড়া উচ্চ · Advanced diffusers কোড

এই পাঠে যা শিখবেন

  • Diffusion model basics — forward + reverse process
  • Latent diffusion — efficiency trick
  • U-Net + CLIP cross-attention
  • Practical use — Hugging Face diffusers

১ · Diffusion model intuition

Diffusion modelDiffusion ModelGenerative model — image-এ gradually noise add (forward), reverse-এ denoise step by step (reverse). DDPM (Ho et al., 2020) original formulation. two process:

  • Forward: image → gradually noise add → pure Gaussian noise.
  • Reverse: noise → gradually denoise → image.

Train: at any noise level, network predict noise added. Inference: start from noise, denoise iteratively.

কেন্দ্রীয় ধারণা

Generation = "iterative denoising". Each step small refinement. 1000-step process (DDPM) → 20-50 step (DDIM, Euler) modern. Quality vs speed trade-off.

২ · Forward diffusion

$$x_t = \sqrt{\alpha_t} x_0 + \sqrt{1 - \alpha_t} \epsilon, \quad \epsilon \sim \mathcal{N}(0, I)$$

  • $x_0$: original image.
  • $x_t$: noisy version at step $t$।
  • $\alpha_t$: noise schedule (small for high noise).
  • $\epsilon$: random Gaussian noise.
  • $t = 0$ original, $t = T$ pure noise.

৩ · Reverse process — denoise

Network $\epsilon_\theta$ — predict noise added. Iteratively:

$$x_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( x_t - \frac{1 - \alpha_t}{\sqrt{1 - \bar{\alpha}_t}} \epsilon_\theta(x_t, t) \right) + \sigma_t z$$

  • $\epsilon_\theta$ trained to predict noise.
  • $z$: random Gaussian (stochasticity).
  • Step by step — clean image emerge.

৪ · Loss function

$$\mathcal{L} = \mathbb{E}_{x_0, t, \epsilon} \big[ \| \epsilon - \epsilon_\theta(x_t, t) \|^2 \big]$$

  • Training: take random image, random noise level, predict noise.
  • Simple MSE — surprisingly effective.

৫ · Latent diffusion — Stable Diffusion-এর key

Pixel-space diffusion expensive (1024×1024 = 3M pixel). Stable Diffusion idea:

  1. VAE encoder: 1024×1024 image → 128×128 latent (factor 8).
  2. Diffusion in latent space: 64x compute reduce.
  3. VAE decoder: denoised latent → image.

Original DDPM 4 days A100, SD same quality 1 day. Hardware affordable.

৬ · Architecture

  • VAE: Image ↔ latent (4×64×64 for 512×512 image).
  • U-Net: Latent + text + time → noise prediction. ~860M params.
  • CLIP text encoder: text prompt → embedding for cross-attention.
  • Scheduler: noise schedule (DDIM, Euler, DPM++).

৭ · Cross-attention with text

  • U-Net-এর each block — self-attention (image-internal).
  • + Cross-attention — image query, text key+value.
  • Text guides denoising.
  • "A cat" — denoise toward cat-like pattern.
Stable Diffusion = "sculptor with text instruction". Block of marble (random noise) → clear sculpture (image). Each chisel stroke (denoise step) refine. Text prompt = sculptor's intent.
Stable Diffusion — text-to-image generation 📝 "A cat" CLIP Text embedding 🌫 Random noise Latent 4×64×64 (8x smaller) U-Net Denoiser ~860M params cross-attn with text predict noise 20-50 step iterate Latent denoised VAE Decoder → 512×512 cat Latent diffusion — pixel-space-এর 64x সাশ্রয় LAION-5B trained, ~860M params, runs on consumer GPU SDXL, SD3, FLUX — progressive improvements
Stable Diffusion — VAE, U-Net, CLIP-এর integration. Latent space-এ diffusion compute-affordable.

৮ · Hugging Face diffusers

Python · diffusers 
# pip install diffusers torch
from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(
    model_id,
    torch_dtype=torch.float16
).to("cuda")

prompt = "A photo of a Bengali cat sitting on a rickshaw, sunset, Dhaka"
image = pipe(
    prompt,
    num_inference_steps=30,
    guidance_scale=7.5,
).images[0]

image.save("output.png")
print("Generated!")

    


    

      Few line of code, professional-quality image generate. 30 step typical. Guidance scale — text adherence (7-15).
    


    
৯ · Stable Diffusion versions

    
    
      
  • SD 1.4/1.5: 512×512, original.
    • 
      
  • SD 2.0/2.1: 768×768, OpenCLIP.
    • 
      
  • SDXL (২০২৩): 1024×1024, dual encoder, 6.6B params.
    • 
      
  • SD 3 (২০২৪): 8B params, T5 + CLIP, MMDiT architecture.
    • 
      
  • FLUX (Black Forest Labs, ২০২৪): 12B, current SOTA open.
    • 
    


    
১০ · Practical features

    
    
      
  • Image-to-image: input image + text → modified image.
    • 
      
  • Inpainting: mask region + text → fill in.
    • 
      
  • ControlNet: pose/depth/edge conditioning.
    • 
      
  • LoRA: efficient fine-tune for style.
    • 
      
  • DreamBooth: personalize on few subject images.
    • 
      
  • Textual inversion: learn new concept token.
    • 
    


    
১১ · Sampling schedulers

    
    
      
  • DDPM (1000 step): original, slow, high quality.
    • 
      
  • DDIM (50 step): deterministic, fast.
    • 
      
  • DPM-Solver++: 20 step competitive.
    • 
      
  • Euler: simple, popular.
    • 
      
  • UniPC, LCM: 1-4 step, real-time.
    • 
    


    
১২ · Bangladesh applications

    
    
      
  • Bangla art digitization.
    • 
      
  • Cultural content (festivals, rural scenes).
    • 
      
  • Educational visualization.
    • 
      
  • Marketing material generation.
    • 
      
  • Game asset for Bangla content creators.
    • 
      
  • Bangladesh-specific finetune — jamdani style, Dhaka skyline.
    • 
    


    

      Stable Diffusion-এর responsibility — copyright, deepfake, NSFW concern. Bangladesh law gradually addressing. Personal/research use OK, commercial — license check.
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      
        প্র ০১
        "Latent diffusion" — pixel-space-এ diffusion থেকে কতটা সাশ্রয়? Quality compromise কী?
      
      

        
SD-এর key innovation. Trade-off carefully designed.

        
Pixel-space diffusion (DDPM):

        
    
          
  • 512×512 RGB = 786K dimension.
    • 
          
  • U-Net process this directly.
    • 
          
  • Compute O(786K × layer × step).
    • 
          
  • Training expensive — months on multi-GPU.
    • 
        

        
Latent diffusion:

        
    
          
  • VAE encode 512² → 4×64²= 16K dim. 48x reduction.
    • 
          
  • U-Net process 16K dim.
    • 
          
  • ~50x compute savings.
    • 
          
  • Single GPU training feasible.
    • 
        

        
VAE quality:

        
    
          
  • Encode-decode round-trip near-lossless.
    • 
          
  • 4-channel latent — semantic preserved.
    • 
          
  • Subtle texture lose minor.
    • 
          
  • Acceptable for generation quality.
    • 
        

        
Limitations:

        
    
          
  • Fine detail (text, eyes) — sometimes inaccurate.
    • 
          
  • Latent VAE bottleneck.
    • 
          
  • Pixel-space diffusion (Imagen, Sora) sometimes higher quality.
    • 
        

        
Modern variant:

        
    
          
  • SDXL: 4×128² latent (1024² image).
    • 
          
  • SD3: continuous latent (DiT-based).
    • 
          
  • FLUX: 16-channel latent.
    • 
        

        
Quality benchmark:

        
    
          
  • SD 1.5 FID 22.65.
    • 
          
  • SDXL FID 12.32.
    • 
          
  • FLUX FID 9.6.
    • 
          
  • Imagen (pixel) — 7.27.
    • 
        

        
Practical:

        
    
          
  • Affordable open access > marginal quality gain.
    • 
          
  • SD democratized image generation.
    • 
        

        
মূল উপলব্ধি: "Compress and process" — efficiency principle. Quality acceptable trade for accessibility.

      

    


    

      
        প্র ০২
        Guidance scale 7.5 — কেন এই specific value? Higher দিলে কী হয়?
      
      

        
Classifier-free guidance (CFG) — diffusion-এর crucial concept.

        
CFG mechanism:

        
    
          
  • Two prediction: conditional ($\epsilon_c$ with text), unconditional ($\epsilon_u$ empty prompt).
    • 
          
  • Combined: $\epsilon = \epsilon_u + s \cdot (\epsilon_c - \epsilon_u)$।
    • 
          
  • $s$ = guidance scale.
    • 
          
  • $s = 1$: condition-only baseline.
    • 
          
  • $s > 1$: amplify text influence.
    • 
        

        
Effect of scale:

        
    
          
  • $s = 1$: faithful to text but vague.
    • 
          
  • $s = 5$: good balance.
    • 
          
  • $s = 7.5$: sweet spot — text strong, quality.
    • 
          
  • $s = 15$: over-saturated, artifacts.
    • 
          
  • $s = 30$: "burnt" appearance, unnatural.
    • 
        

        
Why over-amplification bad:

        
    
          
  • Push prediction far from learned distribution.
    • 
          
  • Out-of-distribution image.
    • 
          
  • Quality degrade.
    • 
        

        
Empirical guidance scale by version:

        
    
          
  • SD 1.5: 7-8 default.
    • 
          
  • SDXL: 7-8.
    • 
          
  • SD 3: 7-9.
    • 
          
  • FLUX: 3.5 (different scale).
    • 
        

        
Negative prompts:

        
    
          
  • "low quality, ugly" — push away from these.
    • 
          
  • $\epsilon = \epsilon_u + s \cdot (\epsilon_c - \epsilon_n)$ where $n$ = negative.
    • 
          
  • Quality boost typically.
    • 
        

        
Tuning practice:

        
    
          
  • Photorealistic: 5-7.
    • 
          
  • Stylized: 8-12.
    • 
          
  • Abstract: 3-5.
    • 
          
  • Test variations.
    • 
        

        
মূল উপলব্ধি: CFG — text adherence dial. Tuning matters more than people realize.

      

    


    

      
        প্র ০৩
        SD-এর LAION-5B training data — copyright, bias. Stability AI কী legal challenge face করছে?
      
      

        
Generative AI legal landscape — actively evolving.

        
LAION-5B description:

        
    
          
  • 5 billion image-text pair.
    • 
          
  • Web-scraped (Common Crawl).
    • 
          
  • Includes copyrighted artwork, photo.
    • 
          
  • Released for research.
    • 
        

        
Major lawsuits:

        
    
          
  • Getty Images v. Stability AI (২০২৩): 12M Getty images allegedly trained. UK lawsuit.
    • 
          
  • Artist class action (২০২৩): Sarah Andersen, Karla Ortiz — copyright.
    • 
          
  • EU AI Act: training data transparency required.
    • 
        

        
Defendant arguments:

        
    
          
  • "Fair use" for transformative purpose.
    • 
          
  • Model doesn't store image directly.
    • 
          
  • Statistical pattern, not copy.
    • 
        

        
Plaintiff arguments:

        
    
          
  • Output sometimes near-copy of training image.
    • 
          
  • Style mimicry harmful.
    • 
          
  • Commercial use without permission.
    • 
        

        
Outcomes-so-far:

        
    
          
  • ২০২৪-এর অনেক case ongoing.
    • 
          
  • Some companies opt-out tools (NoAI metadata).
    • 
          
  • Adobe Firefly — licensed-only data training.
    • 
          
  • OpenAI DALL-E — partial license deal.
    • 
        

        
Bias issues:

        
    
          
  • "CEO" → mostly white male generate.
    • 
          
  • "Nurse" → mostly female.
    • 
          
  • Race, gender, geography bias clear.
    • 
          
  • Underrepresented culture (Bangladesh) — limited samples.
    • 
        

        
NSFW filter:

        
    
          
  • SD safety classifier.
    • 
          
  • Some users disable — controversy.
    • 
          
  • Deepfake concern.
    • 
        

        
Bangladesh implications:

        
    
          
  • Local artist style mimicry — copyright unclear.
    • 
          
  • Religious-cultural sensitivity.
    • 
          
  • No specific regulation yet.
    • 
          
  • Best practice: Bangladesh-specific dataset finetune (licensed).
    • 
        

        
Future direction:

        
    
          
  • Training data transparency mandates.
    • 
          
  • Opt-in/opt-out tools.
    • 
          
  • Compensation models (Adobe paid contributors).
    • 
          
  • License-only model proliferate.
    • 
        

        
মূল উপলব্ধি: Generative AI ethical-legal pipeline still developing. Engineer responsibility — informed deployment. "Just because possible doesn't mean ethical."

      

    


    

      
        প্র ০৪
        Bangladesh-এ Stable Diffusion local finetune — কীভাবে? "Bangladesh-themed" image generate-এ কী strategy?
      
      

        
Real opportunity for Bangladesh creators.

        
Why local finetune:

        
    
          
  • Default SD weak on Bangladesh imagery.
    • 
          
  • "Dhaka street" — stereotypical Asian street, wrong.
    • 
          
  • "Jamdani sari" — generic floral, missing actual pattern.
    • 
          
  • "Eid prayer" — partial Middle East pattern, not Bangladesh.
    • 
        

        
Finetune approach 1 — Full finetune:

        
    
          
  • 10K+ Bangladesh-themed image.
    • 
          
  • Full model (860M params) update.
    • 
          
  • Cost: $500-2000 cloud GPU.
    • 
          
  • Most flexible.
    • 
        

        
Approach 2 — DreamBooth:

        
    
          
  • 5-20 image of specific subject.
    • 
          
  • "Sks rickshaw" → trained tag.
    • 
          
  • Cost: $5-50.
    • 
          
  • Subject-specific.
    • 
        

        
Approach 3 — LoRA (most popular):

        
    
          
  • Low-rank adaptation — small additional weights.
    • 
          
  • 50-200 images per concept.
    • 
          
  • Cost: $5-20.
    • 
          
  • Easy combine multiple LoRAs.
    • 
        

        
Approach 4 — Textual inversion:

        
    
          
  • Learn new token embedding.
    • 
          
  • Few image.
    • 
          
  • No model weight change.
    • 
          
  • Smallest cost.
    • 
        

        
Bangladesh dataset sources:

        
    
          
  • Public domain art (museums).
    • 
          
  • Wikimedia Bangladesh photos.
    • 
          
  • Creative Commons galleries.
    • 
          
  • Custom photography.
    • 
          
  • Museum partnership (Bangladesh National Museum).
    • 
        

        
Concept categories:

        
    
          
  • Architecture: Lalbagh, Ahsan Manzil, Sundarbans.
    • 
          
  • Festival: Pohela Boishakh, Eid, Durga Puja.
    • 
          
  • Clothing: jamdani, panjabi, lungi.
    • 
          
  • Food: biryani, hilsa, pithas.
    • 
          
  • Vehicle: rickshaw, CNG, nouka.
    • 
          
  • Nature: Padma, tea garden, paddy field.
    • 
        

        
Toolkits:

        
    
          
  • Hugging Face autotrain.
    • 
          
  • Kohya_ss (popular LoRA trainer).
    • 
          
  • OneTrainer.
    • 
          
  • Cloud: RunPod, Vast.ai.
    • 
        

        
Quality assurance:

        
    
          
  • Cultural review by Bangladeshi artists.
    • 
          
  • Avoid stereotype reinforcement.
    • 
          
  • Religious sensitivity check.
    • 
          
  • Test diverse demographic prompts.
    • 
        

        
Distribution:

        
    
          
  • Hugging Face / CivitAI publish.
    • 
          
  • Open license (Apache 2.0) preferred.
    • 
          
  • Community contribution.
    • 
        

        
Business opportunity:

        
    
          
  • Local marketing — culturally accurate visual.
    • 
          
  • Educational content.
    • 
          
  • Game/comic creator support.
    • 
          
  • Tourism promotion visual.
    • 
        

        
মূল উপলব্ধি: Foundation model + local finetune = Bangladesh creative AI ecosystem. Investment small, impact significant. Cultural representation critical.

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        SD inference: Hugging Face diffusers দিয়ে একটি simple text-to-image generate.
        
    
          
          
    
            
    Code section ৮-এ। Free Colab GPU দিয়ে চালানো যায়।
    
          
    
        
    
      
    2. 

      
  2. 
        Img-to-img: input image + prompt, modify.
        
    
          
          
    
            
    from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("...", torch_dtype=torch.float16).to("cuda")
result = pipe(prompt="anime style", image=src_image, strength=0.7).images[0]
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Bangladesh marketing agency — SDXL Bangla theme content generate. Workflow design করুন।
        
    
          
          
    
            
    (1) LoRA finetune — local culture. (2) Prompt template Bangla concept-এ। (3) ControlNet pose-অনুযায়ী। (4) Manual curation. (5) Cultural review committee.
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    
    

      কোড রানার কাজ না করলে? Google Colab use করুন।
    

  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ২৮ · DETR