পাঠ ১৮ · ২৮-এর মধ্যে · মডিউল ৩
Home / AI Courses / Generative AI / ControlNet

ControlNet — নিয়ন্ত্রিত generation

ControlNet — spatial conditioning for diffusion
৭ মিনিট পড়া মাঝারি+ · Intermediate+ Spatial control

এই পাঠে যা শিখবেন

  • Text condition-এর সীমাবদ্ধতা — কেন spatial control দরকার
  • ControlNet architecture — frozen base + trainable copy + zero-conv
  • Different control modalities — canny, depth, pose, segmentation
  • Multi-ControlNet, ControlNet-LoRA, IP-Adapter — সাম্প্রতিক extension

১ · Text-only condition-এর সীমা

"A man running on the beach" — SD generate করবে কিন্তু:

  • কোন pose-এ?
  • Camera angle?
  • Beach-এর সঠিক layout?
  • Architectural detail?

Text এসব specify করতে পারে না। আগে ControlNet-এর — এই precision impossible ছিল diffusion-এ।

২ · ControlNet — Zhang, Rao, Agrawala 2023

ICCV 2023 paper "Adding Conditional Control to Text-to-Image Diffusion Models"। Stanford-এর Lvmin Zhang-এর breakthrough।

Core idea

Pretrained SD U-Net-এর encoder + middle block clone করুন (trainable copy)। Original frozen — capabilities preserve। Copy fine-tune control image-এর উপর। দু'টিকে যোগ করুন zero-conv দিয়ে — initialization-এ output 0, শুধু training-এ contribution গড়ে।

৩ · Architecture detail

Inputs:

  • $\mathbf{z}_t$ — noisy latent (standard SD input)।
  • $\mathbf{c}_{text}$ — text condition (CLIP)।
  • $\mathbf{c}_f$ — control image (canny edge, depth map, pose skeleton, etc.)।

Three components:

  1. Frozen SD U-Net: Encoder + middle + decoder — original weights, no gradient।
  2. Trainable copy: Encoder + middle clone (initially same weights)। Gradient flows।
  3. Zero-conv layers: $1\times 1$ convolutions, weights initialized to 0। Connect copy → frozen decoder।

৪ · Zero-conv — kya hai?

Zero-conv = $1\times 1$ conv, weight & bias both initialized 0।

প্রভাব:

  • Training শুরুতে output = 0 → frozen U-Net-এর output unchanged। SD পুরোপুরি কাজ করে।
  • Gradient first step-এ 0 hold করে — কিন্তু input non-zero, তাই weight update হতে শুরু করে (chain rule)।
  • Gradually copy contribution build up — disrupt না করে।
Zero-init brilliant trick — fine-tuning instability avoid করে। ControlNet ছোট dataset (50K image)-এ train হয় কারণ original SD knowledge intact।

৫ · Control image preprocessors

  • Canny edge: OpenCV Canny detector → binary edge map। Sketch-style control।
  • Depth: MiDaS or Depth-Anything → relative depth। 3D structure preserve।
  • OpenPose: Body keypoint skeleton → human pose control।
  • Segmentation: ADE20K semantic mask → scene layout।
  • Scribble: Hand-drawn sketch → loose composition guide।
  • Normal map: Surface normals → 3D-aware lighting।
  • Lineart: Anime/illustration line drawing।
  • MLSD: Straight lines → architectural perspectives।
ControlNet: Frozen SD + Trainable Copy + Zero-Conv z_t (noisy) text c control c_f (canny/depth/pose) 🔒 Frozen SD U-Net Encoder Mid Decoder Original SD knowledge preserved No gradient — never updated 🔧 Trainable Copy (encoder + mid only) Encoder' Mid' Init from SD weights, fine-tune Zero-Conv 1×1 conv w=0, b=0 init add to decoder skip ε̂ controlled noise Control modalities: • Canny edges • Depth map (MiDaS) • OpenPose skeleton • Segmentation • Scribble / sketch • Normal map • Lineart, MLSD Train: 50K controlled images, 1 GPU, ~5 days Inference: same SD speed, no fundamental slowdown Zero-conv: starts at 0 → SD untouched. Gradient builds copy contribution gradually.
Frozen SD ভাঙে না — trainable copy control image থেকে spatial info encode। Zero-conv মাধ্যমে frozen decoder-এ যোগ। সম্পূর্ণ noninvasive fine-tuning।

৬ · PyTorch — using ControlNet

Python · diffusers 
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from PIL import Image
import cv2, numpy as np, torch

# 1. Load ControlNet (canny variant)
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16
)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", controlnet=controlnet,
    torch_dtype=torch.float16,
).to("cuda")

# 2. Prepare control image
input_img = Image.open("input.jpg").convert("RGB")
arr = np.array(input_img)
edges = cv2.Canny(arr, 100, 200)              # canny edges
canny_img = Image.fromarray(np.stack([edges]*3, axis=-1))

# 3. Generate
out = pipe(
    "a photo of a cat in a Bangladeshi village courtyard, golden hour",
    image=canny_img,
    num_inference_steps=30,
    controlnet_conditioning_scale=1.0,        # control strength
    guidance_scale=7.5,
).images[0]
out.save("controlled.png")

    


    

      controlnet_conditioning_scale: 0 = ignore control, 1.0 = strong, 2.0 = over-rigid। Typical 0.7-1.2।
    


    
৭ · Multi-ControlNet

    

      একসাথে multiple control — যেমন pose + canny:
    


    

      

        
        Python · multi-ControlNet
      

      
from diffusers import ControlNetModel

cn_pose = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-openpose", torch_dtype=torch.float16)
cn_depth = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-depth", torch_dtype=torch.float16)

pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=[cn_pose, cn_depth],
    torch_dtype=torch.float16,
).to("cuda")

out = pipe(
    "a Bangladeshi dancer in traditional dress, ornate stage",
    image=[pose_img, depth_img],
    controlnet_conditioning_scale=[1.0, 0.5],     # pose strong, depth softer
).images[0]

      

        
        
      

      

    


    
৮ · Recent extensions

    
    
      
  • ControlNet-LoRA: ControlNet-কে LoRA-এ compress — smaller, faster।
    • 
      
  • T2I-Adapter (Mou et al. 2023): Lightweight alternative — ControlNet-এর ~1/10 size।
    • 
      
  • IP-Adapter (Ye et al. 2023): Image prompt — reference image direct condition।
    • 
      
  • InstantID, PhotoMaker: Identity-preserving generation — face reference।
    • 
      
  • SDXL ControlNet: Promax, Union — multiple modalities one model।
    • 
      
  • Flux Tools: Flux-Canny, Flux-Depth, Flux-Redux — Flux-এর native control।
    • 
    


    

      ControlNet quality control image-এর quality-নির্ভর। Bad canny edges → bad output। Pre-process carefully।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন

    
প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।


    

      
        প্র ০১
        Zero-conv-এর "starts at zero, gradient builds up" — গাণিতিকভাবে কীভাবে কাজ করে? Naive initialization (Gaussian, Xavier)-এর তুলনায় কেন superior?
      
      

        

          Zero-conv ControlNet-এর সবচেয়ে subtle ও brilliant component।
        

        

          Naive fine-tuning সমস্যা:
        

        
    
          
  • Pretrained SD-এ control image inject — random init সাথে — early gradient signal noisy।
    • 
          
  • Network output drastically perturbed শুরুতে — SD-এর pretrained capabilities corrupt।
    • 
          
  • "Catastrophic forgetting" — fine-tuning early phase-এ original knowledge erase।
    • 
          
  • Need careful learning rate, gradual unfreezing — fragile।
    • 
        

        

          Zero-conv mathematics:
        

        
    
          
  • $\mathcal{Z}(\mathbf{x}; \mathbf{w}, \mathbf{b}) = \mathbf{w} \cdot \mathbf{x} + \mathbf{b}$।
    • 
          
  • Init $\mathbf{w} = 0, \mathbf{b} = 0$।
    • 
          
  • Output: $\mathcal{Z}(\mathbf{x}) = 0$ initially।
    • 
          
  • $\partial \mathcal{Z}/\partial \mathbf{w} = \mathbf{x}^T \neq 0$ (input non-zero)।
    • 
          
  • $\partial L/\partial \mathbf{w} = (\partial L/\partial \mathcal{Z}) \cdot \mathbf{x}^T$ — non-zero gradient।
    • 
        

        

          Why this works:
        

        
    
          
  • Step 0: SD output exactly same as pre-ControlNet। Loss = standard SD reconstruction loss।
    • 
          
  • Gradient backward through zero-conv — but $\mathbf{w}$ updates because input $\mathbf{x}$ non-zero।
    • 
          
  • Step 1: $\mathbf{w}$ slightly non-zero → tiny perturbation in SD output।
    • 
          
  • Step $N$: $\mathbf{w}$ has learned meaningful values, control influence builds up।
    • 
        

        

          Comparison with alternatives:
        

        
    
          
  • Random init (Gaussian σ=0.01): Initial output non-zero, perturbation immediate. SD can drift।
    • 
          
  • Xavier/He init: Designed for forward signal preservation, but initial change in SD large।
    • 
          
  • Identity init (residual): Add identity, but for $1\times 1$ conv with channel mismatch tricky।
    • 
          
  • Zero init: Guaranteed zero initial impact, smooth integration।
    • 
        

        

          Theoretical perspective:
        

        
    
          
  • Reminiscent of "warm start" optimization — start from known-good solution।
    • 
          
  • Hypernetwork connections: zero gating signal initially।
    • 
          
  • LoRA (পাঠ ১৯) uses similar principle: $\Delta W = BA$, $A$ Gaussian small, $B$ zero।
    • 
          
  • Gating mechanisms: ResNet-Adapter, AdaLN — all have "start with no influence" insight।
    • 
        

        

          Beyond ControlNet — design principle:
        

        
    
          
  • Whenever adding modules to pretrained model, ensure "soft start"।
    • 
          
  • Dora, OFT, IA3 — recent PEFT methods use variants।
    • 
          
  • Gating with sigmoid initialized at 0 input → 0.5 output, then learn।
    • 
          
  • Adapter modules (Houlsby 2019): small bottleneck, zero-init final projection।
    • 
        

        

          Practical implications for ControlNet training:
        

        
    
          
  • Stable training even with small datasets (50K images sufficient)।
    • 
          
  • Higher learning rates feasible (~$10^{-5}$ to $10^{-4}$)।
    • 
          
  • Single GPU + 5 days enough for new ControlNet variant।
    • 
          
  • SD's pretrained knowledge intact even after long training।
    • 
        

        

          Subtleties & failure modes:
        

        
    
          
  • Gradient calculation error in PyTorch with zero weights — sometimes dead path।
    • 
          
  • Solution: ensure non-zero bias gradient flow (small bias init OK, zero-conv specifically zero weight)।
    • 
          
  • Verify zero-conv: add hook to print weight norm during training — should grow।
    • 
        

        

          মূল উপলব্ধি: Zero-conv "harmless start" principle-এর elegant instantiation। Pretrained model + new module = "build slowly, don't disrupt"। Modern PEFT (LoRA, adapters) all share this DNA। Zhang's 2023 ControlNet paper made this principle widely known।
        

      

    


    

      
        প্র ০২
        Canny vs Depth vs OpenPose — কখন কোন modality চয়ন করব? Photographic realism, anime, architectural, fashion — domain-specific guidance কেমন?
      
      

        

          ControlNet modality selection — designer/developer-এর core skill।
        

        

          Canny edges:
        

        
    
          
  • Strength: Crisp object outlines, mid-level detail।
    • 
          
  • Weakness: Noise-sensitive, miss soft edges, no depth info।
    • 
          
  • Best for:
            
      
              
    • Sketch → photo conversion।
      • 
              
    • Logo / typography preservation।
      • 
              
    • Object outline-based composition।
      • 
              
    • Product photography variants।
      • 
            
    
          
    • 
          
  • Tuning: Threshold parameters (low_threshold, high_threshold)। Typical: 100, 200।
    • 
        

        

          Depth (MiDaS, Depth-Anything):
        

        
    
          
  • Strength: 3D structure, scene composition, perspective।
    • 
          
  • Weakness: No detail (texture, color)। Soft boundaries।
    • 
          
  • Best for:
            
      
              
    • Architectural scenes — preserve building geometry।
      • 
              
    • Landscape — depth-driven composition।
      • 
              
    • Style transfer with structure preservation।
      • 
              
    • Re-lighting (depth + new prompt = new lighting)।
      • 
            
    
          
    • 
          
  • Modern: Depth-Anything V2 — significantly improved depth estimation।
    • 
        

        

          OpenPose:
        

        
    
          
  • Strength: Body joint precision, multi-person, action capture।
    • 
          
  • Weakness: Pose only — clothing, environment uncontrolled।
    • 
          
  • Best for:
            
      
              
    • Character pose specification।
      • 
              
    • Dance, sports, action illustrations।
      • 
              
    • Multi-character scene composition।
      • 
              
    • Animation reference frames।
      • 
            
    
          
    • 
          
  • Variants: Hand pose, face landmark — finer control।
    • 
        

        

          Segmentation (ADE20K-style):
        

        
    
          
  • Strength: Semantic regions (sky, building, person, road)। High-level layout।
    • 
          
  • Weakness: No fine detail, requires pre-defined classes।
    • 
          
  • Best for:
            
      
              
    • Scene layout from rough painting।
      • 
              
    • Architectural site plans।
      • 
              
    • "Color block painting → realistic render"।
      • 
              
    • Auto-generated room layouts।
      • 
            
    
          
    • 
        

        

          Scribble:
        

        
    
          
  • Strength: Lossy, allows creative interpretation।
    • 
          
  • Best for:
            
      
              
    • Quick concept sketches → final art।
      • 
              
    • Children's drawing → realistic image (popular meme)।
      • 
              
    • Loose composition guidance।
      • 
            
    
          
    • 
        

        

          Lineart:
        

        
    
          
  • Anime/manga line drawings → colored illustration।
    • 
          
  • Coarse to fine variants।
    • 
          
  • Best for: comic colorization, illustration workflow।
    • 
        

        

          MLSD (Mobile Line Segment Detection):
        

        
    
          
  • Straight lines only — interior, architecture।
    • 
          
  • Best for: room layouts, building facades।
    • 
        

        

          Normal map:
        

        
    
          
  • Surface orientation — 3D-aware lighting।
    • 
          
  • Best for: re-lighting, cinematic mood control।
    • 
        

        

          Tile (specialty):
        

        
    
          
  • Low-res image → high-res with detail restoration।
    • 
          
  • Best for: upscaling, detail injection।
    • 
        

        

          Domain-specific recommendations:
        

        
    
          
  • Photographic realism: Depth + Canny combo। Gentle structure।
    • 
          
  • Anime/illustration: Lineart + OpenPose। Crisp outlines + character pose।
    • 
          
  • Architectural: MLSD + Depth। Geometric precision।
    • 
          
  • Fashion: OpenPose + Canny clothing। Pose + garment outline।
    • 
          
  • Product photography: Canny + Depth। Outline + 3D presence।
    • 
          
  • Bangladeshi cultural (rickshaw art): Lineart + custom LoRA। Style + structure।
    • 
          
  • Portrait: OpenPose face + Canny। Identity guidance।
    • 
        

        

          Multi-ControlNet strategy:
        

        
    
          
  • 2 modalities optimal (more = over-constrain, slow)।
    • 
          
  • Conditioning scales: primary 1.0, secondary 0.5-0.7।
    • 
          
  • Common pairs:
            
      
              
    • Pose (1.0) + Depth (0.6) — character in scene।
      • 
              
    • Canny (0.8) + Color (0.5) — re-coloring।
      • 
              
    • OpenPose (1.0) + Lineart (0.7) — character art।
      • 
            
    
          
    • 
        

        

          Strength control nuance:
        

        
    
          
  • $0$: ignore control (defeats purpose)।
    • 
          
  • $0.3-0.5$: loose guidance, creative freedom।
    • 
          
  • $0.7-1.0$: strong adherence (default)।
    • 
          
  • $1.5-2.0$: rigid, may conflict with prompt।
    • 
        

        

          Common pitfalls:
        

        
    
          
  • Bad preprocessing — poor canny → poor output।
    • 
          
  • Mismatched control + prompt — pose says "running", prompt says "sitting"।
    • 
          
  • Too many controls — over-constrain, lifeless output।
    • 
          
  • Modality wrong for domain — depth for flat icon design unhelpful।
    • 
        

        

          মূল উপলব্ধি: ControlNet modality = "what aspect of structure to preserve"। Mastery comes from understanding image components: outlines (canny), 3D structure (depth), human pose (OpenPose), semantic regions (segmentation)। Different art form → different modality preference।
        

      

    


    

      
        প্র ০৩
        ControlNet ছোট 50K dataset-এ trained — কিন্তু effective। Pretrained backbone-এর role কী এবং fine-tuning paradigm-এ এটি কী principle establish করেছে?
      
      

        

          ControlNet — efficient fine-tuning-এর landmark example।
        

        

          Pretrained backbone advantage:
        

        
    
          
  • SD trained on LAION-5B (~5 billion image-text pairs)।
    • 
          
  • Encoder learned diverse visual features (edges, textures, objects, scenes)।
    • 
          
  • ControlNet-এর copy initialized from SD encoder — already strong feature extractor।
    • 
          
  • Only need to learn "control image → control feature" mapping।
    • 
        

        

          50K dataset feasibility:
        

        
    
          
  • Random 50K image subset of LAION + extracted control (canny/depth/pose)।
    • 
          
  • 1 GPU (A100), ~5 days per modality।
    • 
          
  • Small lab/individual feasible — democratized advancement।
    • 
        

        

          Why so few examples enough:
        

        
    
          
  • Backbone provides 99% of "what to generate" knowledge।
    • 
          
  • ControlNet learns 1% — "how to incorporate control signal"।
    • 
          
  • Map: control (sparse signal) → control feature (rich signal aligned with SD's representation)।
    • 
          
  • Akin to learning a "translation layer" — small mapping, big effect।
    • 
        

        

          Compared to from-scratch training:
        

        
    
          
  • From-scratch text-to-image: 100M+ images, 1000s GPU-days।
    • 
          
  • ControlNet: 50K images, 5 GPU-days।
    • 
          
  • ~$10^4 \times$ more efficient।
    • 
        

        

          Principles ControlNet established:
        

        
    
          
  1. Frozen backbone preservation: Don't disturb expensive pretrained weights।
    2. 
          
  2. Trainable adapter modules: Add new capabilities via small modules।
    3. 
          
  3. Zero-init for stability: Smooth integration, no catastrophic forgetting।
    4. 
          
  4. Modular design: Multiple ControlNets composable at inference।
    5. 
          
  5. Reusability: One ControlNet works on community fine-tuned SD checkpoints।
    6. 
        

        

          Influence on broader field:
        

        
    
          
  • LoRA (Hu 2021): Predates ControlNet, similar principle in LLMs। Low-rank trainable matrix।
    • 
          
  • QLoRA (Dettmers 2023): Quantized backbone + LoRA — even more efficient।
    • 
          
  • Adapters (Houlsby 2019): Bottleneck modules in transformers।
    • 
          
  • Prefix Tuning, P-Tuning: Trainable input prefixes only।
    • 
          
  • IA3, LoHa, LoKr, OFT: Variations of efficient fine-tuning।
    • 
        

        

          PEFT (Parameter-Efficient Fine-Tuning) revolution:
        

        
    
          
  • 2021-2024: full fine-tuning → PEFT default in most domains।
    • 
          
  • Hugging Face PEFT library: unified interface।
    • 
          
  • Production: thousands of LoRAs swappable on same base model।
    • 
        

        

          Composability advantage:
        

        
    
          
  • Multiple ControlNets at inference — single training each।
    • 
          
  • SD checkpoint + ControlNet + LoRA + Textual Inversion = layered customization।
    • 
          
  • Civitai marketplace: thousands of variants, each modular।
    • 
        

        

          Cost democratization:
        

        
    
          
  • Pre-2023: Image generation control = $50K-500K compute। Big tech only।
    • 
          
  • Post-ControlNet: hobbyist can train custom ControlNet for < $100।
    • 
          
  • New modalities (e.g., medical scan → diagnostic illustration) viable for small teams।
    • 
        

        

          Limitations of paradigm:
        

        
    
          
  • Requires strong pretrained backbone — frontier model dependency।
    • 
          
  • Specific to base model architecture — SD1.5 ControlNet ≠ SDXL ControlNet।
    • 
          
  • Inference cost: extra forward pass through ControlNet copy।
    • 
          
  • Quality cap: can't exceed backbone's intrinsic capacity।
    • 
        

        

          Modern evolutions:
        

        
    
          
  • T2I-Adapter: 1/10 size of ControlNet, similar quality।
    • 
          
  • ControlNet-XS: Ultra-lightweight variant।
    • 
          
  • Union ControlNet (SDXL): Single model, multiple modalities।
    • 
          
  • Promax (SDXL): Multi-modal, multi-task।
    • 
          
  • FLUX.1 Tools (BFL 2024): Native control for Flux।
    • 
        

        

          Lessons for future:
        

        
    
          
  • Foundation model + small adapter = capability extension।
    • 
          
  • Modular AI ecosystem viable।
    • 
          
  • Compute democratization through smart architecture choices।
    • 
          
  • Open-source backbone (SD) + community innovations (ControlNet, LoRA) outpace proprietary monoliths in some metrics।
    • 
        

        

          মূল উপলব্ধি: ControlNet's 50K-dataset success proves "transfer learning at scale" — pretrained foundation + minimal fine-tuning = massive functionality gain। This paradigm now standard across modalities (vision, NLP, audio)। বাংলাদেশী researcher/startup-এর জন্য — frontier compete করতে full pretrain না, custom ControlNet/LoRA তৈরি করে niche domination feasible।
        

      

    


    

      
        প্র ০৪
        আপনি একটি Bangladeshi e-commerce platform-এ কাজ করছেন। প্রতিটি product-এর AI-generated lifestyle photograph লাগবে (model wearing kurta, saree, etc.)। ControlNet workflow design করুন। Scalability, brand consistency, cost — কীভাবে handle?
      
      

        

          E-commerce production AI workflow — real-world, high-stakes use case।
        

        

          Business context:
        

        
    
          
  • Daraz, Pickaboo, Chaldal — thousands of products, photography expensive।
    • 
          
  • Traditional shoot: ৳5,000-50,000/product depending on model/scene।
    • 
          
  • AI alternative: ৳50-500/product, faster turnaround, more variations।
    • 
          
  • Catch: brand consistency, product accuracy, ethical considerations।
    • 
        

        

          Workflow design — the pipeline:
        

        
    
          
  1. Product photography (real, isolated): Plain background, multiple angles। Existing process।
    2. 
          
  2. Background removal: rembg or SAM (Segment Anything) — clean PNG with alpha।
    3. 
          
  3. Pose generation: Standard model poses (front, side, walking, sitting)। OpenPose skeleton library।
    4. 
          
  4. ControlNet composite:
            
      
              
    • OpenPose: model pose।
      • 
              
    • Canny: product overlaid on body region।
      • 
              
    • Depth: scene context (background)।
      • 
            
    
          
    5. 
          
  5. Generation: SDXL + ControlNet + brand-specific LoRA।
    6. 
          
  6. Refinement:
            
      
              
    • Inpainting for face details।
      • 
              
    • Upscaling (4× via RealESRGAN)।
      • 
              
    • Quality check।
      • 
            
    
          
    7. 
          
  7. Approval workflow: Human reviewer checks (50-100/day)।
    8. 
        

        

          Brand consistency:
        

        
    
          
  • Brand LoRA: Train on existing brand photography (~200 images)। Captures aesthetic — lighting, colors, mood।
    • 
          
  • Style guide prompts: Standardized "in the style of [brand X], golden hour, warm tones, professional fashion photography"।
    • 
          
  • Consistent model: InstantID/PhotoMaker — same face across catalog (or 5-10 model rotation)।
    • 
          
  • Negative prompts: "low quality, distorted, watermark, generic stock photo"।
    • 
        

        

          Scalability — production architecture:
        

        
    
          
  • Self-hosted GPU cluster: 4× RTX 4090 — ~৳800K initial।
    • 
          
  • ComfyUI workflow as JSON: Reproducible pipeline।
    • 
          
  • Queue system: Redis + Celery — async job processing।
    • 
          
  • Throughput: Per GPU 100-200 images/hour (SDXL + ControlNet)। 4 GPUs = 400-800/hour।
    • 
          
  • Daily capacity: 10,000-20,000 images।
    • 
        

        

          Alternative: Cloud-based:
        

        
    
          
  • Replicate, RunPod — pay per image।
    • 
          
  • $0.02-0.10 per image।
    • 
          
  • 10,000 images/day = $200-1,000/day।
    • 
          
  • OpEx vs CapEx trade-off।
    • 
          
  • Hybrid: peak load to cloud, baseline self-host।
    • 
        

        

          Cost analysis (per image):
        

        
    
          
  • Compute: ৳3-10 (self-host amortized) or ৳5-25 (cloud)।
    • 
          
  • Human review: ৳5-15।
    • 
          
  • Failed/rejected: 20-30% rate → cost ÷ acceptance rate।
    • 
          
  • Final per-image cost: ৳25-100।
    • 
          
  • vs Traditional: ৳5,000-50,000 — 50-200× cheaper।
    • 
        

        

          Quality assurance:
        

        
    
          
  • Automated checks:
            
      
              
    • NSFW detection (essential)।
      • 
              
    • Face count (no extra faces)।
      • 
              
    • Hand/finger anomaly detection (notorious problem)।
      • 
              
    • Product visibility (not occluded)।
      • 
            
    
          
    • 
          
  • Human reviewer: Cultural appropriateness, brand alignment, technical errors।
    • 
          
  • A/B test: AI vs traditional photo — conversion rate measure।
    • 
        

        

          Bangladesh-specific considerations:
        

        
    
          
  • Cultural appropriateness:
            
      
              
    • Saree drape style accurate (different across regions)।
      • 
              
    • Modest poses for traditional wear।
      • 
              
    • Religious sensitivity (avoid clashing motifs)।
      • 
            
    
          
    • 
          
  • Diverse representation:
            
      
              
    • Skin tones spanning South Asian variation।
      • 
              
    • Body types (not just thin model)।
      • 
              
    • Age diversity (young to mature)।
      • 
            
    
          
    • 
          
  • Local context:
            
      
              
    • Bangladeshi setting (not generic Western)।
      • 
              
    • Festivals (Eid, Pohela Boishakh, Pohela Falgun)।
      • 
              
    • Weather (monsoon, summer)।
      • 
            
    
          
    • 
        

        

          Ethical & legal:
        

        
    
          
  • Disclosure: Bangladesh Advertising Standards Council guidelines — AI-generated should be labeled where required।
    • 
          
  • Model rights: Real model likeness — license carefully। Synthetic identity safer।
    • 
          
  • Consent for training data: Brand's own images OK. Web-scraped face data risky।
    • 
          
  • Deceptive practices: Don't imply real human endorsement।
    • 
          
  • Returns/sizing: AI image must accurately represent product fit (legal liability)।
    • 
        

        

          Iterative improvements:
        

        
    
          
  • Phase 1: Background only AI (real product, AI scene)।
    • 
          
  • Phase 2: AI model + real product (composite)।
    • 
          
  • Phase 3: Full AI generation with strong product accuracy।
    • 
          
  • Phase 4: Personalized — user upload photo, see how product looks on them।
    • 
        

        

          Tech stack recommendation:
        

        
    
          
  • Base model: SDXL or Flux (preferred for fashion fidelity)।
    • 
          
  • ControlNet: OpenPose + Depth + Canny (multi)।
    • 
          
  • LoRA: Brand-specific style + cultural specificity।
    • 
          
  • Adapter: IP-Adapter for product image conditioning।
    • 
          
  • Workflow: ComfyUI (production-ready)।
    • 
          
  • Backend: FastAPI + PostgreSQL + S3-compatible storage।
    • 
          
  • Frontend: Internal admin panel for review।
    • 
        

        

          ROI timeline:
        

        
    
          
  • Month 1-2: Setup, train custom LoRAs, refine pipeline।
    • 
          
  • Month 3: 1,000 products processed, validation।
    • 
          
  • Month 4-6: Scale to 10,000+ products। 70-80% cost saving achieved।
    • 
          
  • Year 1: Full catalog AI-augmented। ROI 5-10× depending on scale।
    • 
        

        

          Common pitfalls:
        

        
    
          
  • Underestimating human review effort।
    • 
          
  • Not investing in custom LoRA — generic SDXL doesn't match brand।
    • 
          
  • Ignoring cultural context — generic Asian model failures।
    • 
          
  • Over-reliance on AI — losing brand authenticity।
    • 
          
  • Legal/ethical shortcuts — backlash risk।
    • 
        

        

          মূল উপলব্ধি: Production ControlNet workflow = engineering + design + ethics। Tech is now mature; differentiation comes from cultural specificity, brand consistency, ethical execution। বাংলাদেশের e-commerce-এ first-mover with cultural-aware AI imagery massive opportunity। Daraz alone has 5M+ products — visualization opportunity কত!
        

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Canny preprocessing: একটি ছবির canny edge OpenCV-তে extract করুন। থ্রেশহোল্ড vary করে কী changes দেখলেন?
        
    
          
          
    
            
    import cv2, numpy as np
img = cv2.imread("input.jpg", cv2.IMREAD_GRAYSCALE)
for low, high in [(50, 100), (100, 200), (200, 400)]:
    edges = cv2.Canny(img, low, high)
    cv2.imwrite(f"canny_{low}_{high}.png", edges)
    
            
    Low threshold: more edges (noisy)। High threshold: fewer edges (cleaner but may lose details)। Default 100/200 typical।
    
          
    
        
    
      
    2. 
      
  2. 
        Multi-ControlNet composition: "A traditional Bangladeshi dancer" — pose + canny দিয়ে ControlNet সাজান। কোন strength values trial করবেন?
        
    
          
          
    
            
    Pose strength 1.0 (rigid pose), Canny 0.5-0.7 (clothing outline guide, but allow artistic flexibility)।
    
            
    Test variations: pose 1.0 / canny 0.3 (loose), pose 1.0 / canny 0.8 (tight)।
    
            
    Cultural: dancer pose may not be in standard OpenPose dataset — custom pose skeleton helpful।
    
          
    
        
    
      
    3. 
      
  3. 
        Conceptual: SDXL ControlNet vs SD1.5 ControlNet — কেন SDXL-এর জন্য আলাদা ControlNet দরকার?
        
    
          
          
    
            
    ControlNet copy initializes from SD encoder। SDXL encoder architecture (channel sizes, layer count) SD1.5 থেকে ভিন্ন। Same ControlNet incompatible — both encoders different feature dimensionality।
    
            
    SDXL-specific ControlNets — initialize from SDXL encoder, train separately।
    
            
    Recent: Union ControlNet — single model multiple modality, simpler ecosystem।
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    

    

      Hands-on:
      Colab-এ
      lllyasviel-এর HuggingFace ControlNet checkpoints try করুন — canny, depth, pose ভিন্ন মডেল available।
    

  







  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৭ · Classifier-free guidance