পাঠ ২৯ · ৩৫-এর মধ্যে · মডিউল ৪
Home / AI Courses / Computer Vision / Stable Diffusion

Stable Diffusion — image generation

Stable Diffusion — open-source text-to-image generation
৯ মিনিট পড়া উচ্চ · Advanced diffusers কোড

এই পাঠে যা শিখবেন

  • Diffusion model basics — forward + reverse process
  • Latent diffusion — efficiency trick
  • U-Net + CLIP cross-attention
  • Practical use — Hugging Face diffusers

১ · Diffusion model intuition

Diffusion modelDiffusion ModelGenerative model — image-এ gradually noise add (forward), reverse-এ denoise step by step (reverse)। DDPM (Ho et al., 2020) original formulation। two process:

  • Forward: image → gradually noise add → pure Gaussian noise।
  • Reverse: noise → gradually denoise → image।

Train: at any noise level, network predict noise added। Inference: start from noise, denoise iteratively।

কেন্দ্রীয় ধারণা

Generation = "iterative denoising"। Each step small refinement। 1000-step process (DDPM) → 20-50 step (DDIM, Euler) modern। Quality vs speed trade-off।

২ · Forward diffusion

$$x_t = \sqrt{\alpha_t} x_0 + \sqrt{1 - \alpha_t} \epsilon, \quad \epsilon \sim \mathcal{N}(0, I)$$

  • $x_0$: original image।
  • $x_t$: noisy version at step $t$।
  • $\alpha_t$: noise schedule (small for high noise)।
  • $\epsilon$: random Gaussian noise।
  • $t = 0$ original, $t = T$ pure noise।

৩ · Reverse process — denoise

Network $\epsilon_\theta$ — predict noise added। Iteratively:

$$x_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( x_t - \frac{1 - \alpha_t}{\sqrt{1 - \bar{\alpha}_t}} \epsilon_\theta(x_t, t) \right) + \sigma_t z$$

  • $\epsilon_\theta$ trained to predict noise।
  • $z$: random Gaussian (stochasticity)।
  • Step by step — clean image emerge।

৪ · Loss function

$$\mathcal{L} = \mathbb{E}_{x_0, t, \epsilon} \big[ \| \epsilon - \epsilon_\theta(x_t, t) \|^2 \big]$$

  • Training: take random image, random noise level, predict noise।
  • Simple MSE — surprisingly effective।

৫ · Latent diffusion — Stable Diffusion-এর key

Pixel-space diffusion expensive (1024×1024 = 3M pixel)। Stable Diffusion idea:

  1. VAE encoder: 1024×1024 image → 128×128 latent (factor 8)।
  2. Diffusion in latent space: 64x compute reduce।
  3. VAE decoder: denoised latent → image।

Original DDPM 4 days A100, SD same quality 1 day। Hardware affordable।

৬ · Architecture

  • VAE: Image ↔ latent (4×64×64 for 512×512 image)।
  • U-Net: Latent + text + time → noise prediction। ~860M params।
  • CLIP text encoder: text prompt → embedding for cross-attention।
  • Scheduler: noise schedule (DDIM, Euler, DPM++)।

৭ · Cross-attention with text

  • U-Net-এর each block — self-attention (image-internal)।
  • + Cross-attention — image query, text key+value।
  • Text guides denoising।
  • "A cat" — denoise toward cat-like pattern।
Stable Diffusion = "sculptor with text instruction"। Block of marble (random noise) → clear sculpture (image)। Each chisel stroke (denoise step) refine। Text prompt = sculptor's intent।
Stable Diffusion — text-to-image generation 📝 "A cat" CLIP Text embedding 🌫 Random noise Latent 4×64×64 (8x smaller) U-Net Denoiser ~860M params cross-attn with text predict noise 20-50 step iterate Latent denoised VAE Decoder → 512×512 cat Latent diffusion — pixel-space-এর 64x সাশ্রয় LAION-5B trained, ~860M params, runs on consumer GPU SDXL, SD3, FLUX — progressive improvements
Stable Diffusion — VAE, U-Net, CLIP-এর integration। Latent space-এ diffusion compute-affordable।

৮ · Hugging Face diffusers

Python · diffusers 
# pip install diffusers torch
from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(
    model_id,
    torch_dtype=torch.float16
).to("cuda")

prompt = "A photo of a Bengali cat sitting on a rickshaw, sunset, Dhaka"
image = pipe(
    prompt,
    num_inference_steps=30,
    guidance_scale=7.5,
).images[0]

image.save("output.png")
print("Generated!")

    


    

      Few line of code, professional-quality image generate। 30 step typical। Guidance scale — text adherence (7-15)।
    


    
৯ · Stable Diffusion versions

    
    
      
  • SD 1.4/1.5: 512×512, original।
    • 
      
  • SD 2.0/2.1: 768×768, OpenCLIP।
    • 
      
  • SDXL (২০২৩): 1024×1024, dual encoder, 6.6B params।
    • 
      
  • SD 3 (২০২৪): 8B params, T5 + CLIP, MMDiT architecture।
    • 
      
  • FLUX (Black Forest Labs, ২০২৪): 12B, current SOTA open।
    • 
    


    
১০ · Practical features

    
    
      
  • Image-to-image: input image + text → modified image।
    • 
      
  • Inpainting: mask region + text → fill in।
    • 
      
  • ControlNet: pose/depth/edge conditioning।
    • 
      
  • LoRA: efficient fine-tune for style।
    • 
      
  • DreamBooth: personalize on few subject images।
    • 
      
  • Textual inversion: learn new concept token।
    • 
    


    
১১ · Sampling schedulers

    
    
      
  • DDPM (1000 step): original, slow, high quality।
    • 
      
  • DDIM (50 step): deterministic, fast।
    • 
      
  • DPM-Solver++: 20 step competitive।
    • 
      
  • Euler: simple, popular।
    • 
      
  • UniPC, LCM: 1-4 step, real-time।
    • 
    


    
১২ · Bangladesh applications

    
    
      
  • Bangla art digitization।
    • 
      
  • Cultural content (festivals, rural scenes)।
    • 
      
  • Educational visualization।
    • 
      
  • Marketing material generation।
    • 
      
  • Game asset for Bangla content creators।
    • 
      
  • Bangladesh-specific finetune — jamdani style, Dhaka skyline।
    • 
    


    

      Stable Diffusion-এর responsibility — copyright, deepfake, NSFW concern। Bangladesh law gradually addressing। Personal/research use OK, commercial — license check।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      
        প্র ০১
        "Latent diffusion" — pixel-space-এ diffusion থেকে কতটা সাশ্রয়? Quality compromise কী?
      
      

        
SD-এর key innovation। Trade-off carefully designed।

        
Pixel-space diffusion (DDPM):

        
    
          
  • 512×512 RGB = 786K dimension।
    • 
          
  • U-Net process this directly।
    • 
          
  • Compute O(786K × layer × step)।
    • 
          
  • Training expensive — months on multi-GPU।
    • 
        

        
Latent diffusion:

        
    
          
  • VAE encode 512² → 4×64²= 16K dim। 48x reduction।
    • 
          
  • U-Net process 16K dim।
    • 
          
  • ~50x compute savings।
    • 
          
  • Single GPU training feasible।
    • 
        

        
VAE quality:

        
    
          
  • Encode-decode round-trip near-lossless।
    • 
          
  • 4-channel latent — semantic preserved।
    • 
          
  • Subtle texture lose minor।
    • 
          
  • Acceptable for generation quality।
    • 
        

        
Limitations:

        
    
          
  • Fine detail (text, eyes) — sometimes inaccurate।
    • 
          
  • Latent VAE bottleneck।
    • 
          
  • Pixel-space diffusion (Imagen, Sora) sometimes higher quality।
    • 
        

        
Modern variant:

        
    
          
  • SDXL: 4×128² latent (1024² image)।
    • 
          
  • SD3: continuous latent (DiT-based)।
    • 
          
  • FLUX: 16-channel latent।
    • 
        

        
Quality benchmark:

        
    
          
  • SD 1.5 FID 22.65।
    • 
          
  • SDXL FID 12.32।
    • 
          
  • FLUX FID 9.6।
    • 
          
  • Imagen (pixel) — 7.27।
    • 
        

        
Practical:

        
    
          
  • Affordable open access > marginal quality gain।
    • 
          
  • SD democratized image generation।
    • 
        

        
মূল উপলব্ধি: "Compress and process" — efficiency principle। Quality acceptable trade for accessibility।

      

    


    

      
        প্র ০২
        Guidance scale 7.5 — কেন এই specific value? Higher দিলে কী হয়?
      
      

        
Classifier-free guidance (CFG) — diffusion-এর crucial concept।

        
CFG mechanism:

        
    
          
  • Two prediction: conditional ($\epsilon_c$ with text), unconditional ($\epsilon_u$ empty prompt)।
    • 
          
  • Combined: $\epsilon = \epsilon_u + s \cdot (\epsilon_c - \epsilon_u)$।
    • 
          
  • $s$ = guidance scale।
    • 
          
  • $s = 1$: condition-only baseline।
    • 
          
  • $s > 1$: amplify text influence।
    • 
        

        
Effect of scale:

        
    
          
  • $s = 1$: faithful to text but vague।
    • 
          
  • $s = 5$: good balance।
    • 
          
  • $s = 7.5$: sweet spot — text strong, quality।
    • 
          
  • $s = 15$: over-saturated, artifacts।
    • 
          
  • $s = 30$: "burnt" appearance, unnatural।
    • 
        

        
Why over-amplification bad:

        
    
          
  • Push prediction far from learned distribution।
    • 
          
  • Out-of-distribution image।
    • 
          
  • Quality degrade।
    • 
        

        
Empirical guidance scale by version:

        
    
          
  • SD 1.5: 7-8 default।
    • 
          
  • SDXL: 7-8।
    • 
          
  • SD 3: 7-9।
    • 
          
  • FLUX: 3.5 (different scale)।
    • 
        

        
Negative prompts:

        
    
          
  • "low quality, ugly" — push away from these।
    • 
          
  • $\epsilon = \epsilon_u + s \cdot (\epsilon_c - \epsilon_n)$ where $n$ = negative।
    • 
          
  • Quality boost typically।
    • 
        

        
Tuning practice:

        
    
          
  • Photorealistic: 5-7।
    • 
          
  • Stylized: 8-12।
    • 
          
  • Abstract: 3-5।
    • 
          
  • Test variations।
    • 
        

        
মূল উপলব্ধি: CFG — text adherence dial। Tuning matters more than people realize।

      

    


    

      
        প্র ০৩
        SD-এর LAION-5B training data — copyright, bias। Stability AI কী legal challenge face করছে?
      
      

        
Generative AI legal landscape — actively evolving।

        
LAION-5B description:

        
    
          
  • 5 billion image-text pair।
    • 
          
  • Web-scraped (Common Crawl)।
    • 
          
  • Includes copyrighted artwork, photo।
    • 
          
  • Released for research।
    • 
        

        
Major lawsuits:

        
    
          
  • Getty Images v. Stability AI (২০২৩): 12M Getty images allegedly trained। UK lawsuit।
    • 
          
  • Artist class action (২০২৩): Sarah Andersen, Karla Ortiz — copyright।
    • 
          
  • EU AI Act: training data transparency required।
    • 
        

        
Defendant arguments:

        
    
          
  • "Fair use" for transformative purpose।
    • 
          
  • Model doesn't store image directly।
    • 
          
  • Statistical pattern, not copy।
    • 
        

        
Plaintiff arguments:

        
    
          
  • Output sometimes near-copy of training image।
    • 
          
  • Style mimicry harmful।
    • 
          
  • Commercial use without permission।
    • 
        

        
Outcomes-so-far:

        
    
          
  • ২০২৪-এর অনেক case ongoing।
    • 
          
  • Some companies opt-out tools (NoAI metadata)।
    • 
          
  • Adobe Firefly — licensed-only data training।
    • 
          
  • OpenAI DALL-E — partial license deal।
    • 
        

        
Bias issues:

        
    
          
  • "CEO" → mostly white male generate।
    • 
          
  • "Nurse" → mostly female।
    • 
          
  • Race, gender, geography bias clear।
    • 
          
  • Underrepresented culture (Bangladesh) — limited samples।
    • 
        

        
NSFW filter:

        
    
          
  • SD safety classifier।
    • 
          
  • Some users disable — controversy।
    • 
          
  • Deepfake concern।
    • 
        

        
Bangladesh implications:

        
    
          
  • Local artist style mimicry — copyright unclear।
    • 
          
  • Religious-cultural sensitivity।
    • 
          
  • No specific regulation yet।
    • 
          
  • Best practice: Bangladesh-specific dataset finetune (licensed)।
    • 
        

        
Future direction:

        
    
          
  • Training data transparency mandates।
    • 
          
  • Opt-in/opt-out tools।
    • 
          
  • Compensation models (Adobe paid contributors)।
    • 
          
  • License-only model proliferate।
    • 
        

        
মূল উপলব্ধি: Generative AI ethical-legal pipeline still developing। Engineer responsibility — informed deployment। "Just because possible doesn't mean ethical।"

      

    


    

      
        প্র ০৪
        Bangladesh-এ Stable Diffusion local finetune — কীভাবে? "Bangladesh-themed" image generate-এ কী strategy?
      
      

        
Real opportunity for Bangladesh creators।

        
Why local finetune:

        
    
          
  • Default SD weak on Bangladesh imagery।
    • 
          
  • "Dhaka street" — stereotypical Asian street, wrong।
    • 
          
  • "Jamdani sari" — generic floral, missing actual pattern।
    • 
          
  • "Eid prayer" — partial Middle East pattern, not Bangladesh।
    • 
        

        
Finetune approach 1 — Full finetune:

        
    
          
  • 10K+ Bangladesh-themed image।
    • 
          
  • Full model (860M params) update।
    • 
          
  • Cost: $500-2000 cloud GPU।
    • 
          
  • Most flexible।
    • 
        

        
Approach 2 — DreamBooth:

        
    
          
  • 5-20 image of specific subject।
    • 
          
  • "Sks rickshaw" → trained tag।
    • 
          
  • Cost: $5-50।
    • 
          
  • Subject-specific।
    • 
        

        
Approach 3 — LoRA (most popular):

        
    
          
  • Low-rank adaptation — small additional weights।
    • 
          
  • 50-200 images per concept।
    • 
          
  • Cost: $5-20।
    • 
          
  • Easy combine multiple LoRAs।
    • 
        

        
Approach 4 — Textual inversion:

        
    
          
  • Learn new token embedding।
    • 
          
  • Few image।
    • 
          
  • No model weight change।
    • 
          
  • Smallest cost।
    • 
        

        
Bangladesh dataset sources:

        
    
          
  • Public domain art (museums)।
    • 
          
  • Wikimedia Bangladesh photos।
    • 
          
  • Creative Commons galleries।
    • 
          
  • Custom photography।
    • 
          
  • Museum partnership (Bangladesh National Museum)।
    • 
        

        
Concept categories:

        
    
          
  • Architecture: Lalbagh, Ahsan Manzil, Sundarbans।
    • 
          
  • Festival: Pohela Boishakh, Eid, Durga Puja।
    • 
          
  • Clothing: jamdani, panjabi, lungi।
    • 
          
  • Food: biryani, hilsa, pithas।
    • 
          
  • Vehicle: rickshaw, CNG, nouka।
    • 
          
  • Nature: Padma, tea garden, paddy field।
    • 
        

        
Toolkits:

        
    
          
  • Hugging Face autotrain।
    • 
          
  • Kohya_ss (popular LoRA trainer)।
    • 
          
  • OneTrainer।
    • 
          
  • Cloud: RunPod, Vast.ai।
    • 
        

        
Quality assurance:

        
    
          
  • Cultural review by Bangladeshi artists।
    • 
          
  • Avoid stereotype reinforcement।
    • 
          
  • Religious sensitivity check।
    • 
          
  • Test diverse demographic prompts।
    • 
        

        
Distribution:

        
    
          
  • Hugging Face / CivitAI publish।
    • 
          
  • Open license (Apache 2.0) preferred।
    • 
          
  • Community contribution।
    • 
        

        
Business opportunity:

        
    
          
  • Local marketing — culturally accurate visual।
    • 
          
  • Educational content।
    • 
          
  • Game/comic creator support।
    • 
          
  • Tourism promotion visual।
    • 
        

        
মূল উপলব্ধি: Foundation model + local finetune = Bangladesh creative AI ecosystem। Investment small, impact significant। Cultural representation critical।

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        SD inference: Hugging Face diffusers দিয়ে একটি simple text-to-image generate।
        
    
          
          
    
            
    Code section ৮-এ। Free Colab GPU দিয়ে চালানো যায়।
    
          
    
        
    
      
    2. 

      
  2. 
        Img-to-img: input image + prompt, modify।
        
    
          
          
    
            
    from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("...", torch_dtype=torch.float16).to("cuda")
result = pipe(prompt="anime style", image=src_image, strength=0.7).images[0]
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Bangladesh marketing agency — SDXL Bangla theme content generate। Workflow design করুন।
        
    
          
          
    
            
    (1) LoRA finetune — local culture। (2) Prompt template Bangla concept-এ। (3) ControlNet pose-অনুযায়ী। (4) Manual curation। (5) Cultural review committee।
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    
    

      কোড রানার কাজ না করলে? Google Colab use করুন।
    

  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ২৮ · DETR