পাঠ ২৩ · ৩৫-এর মধ্যে · মডিউল ৩
Home / AI Courses / Computer Vision / Mask R-CNN

Mask R-CNN

Mask R-CNN — Faster R-CNN with mask head
৭ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

  • Mask R-CNN — Faster R-CNN-এর extension
  • RoI Align — quantization-free region pooling
  • Mask head architecture
  • PyTorch-এ inference + finetune

১ · Mask R-CNN-এর জন্ম

Kaiming He et al. (২০১৭, Facebook AI Research)। Faster R-CNN-এর extension। Conceptually simple, accuracy high।

Goal: instance segmentation — per-object pixel mask। প্রতিটি car-এর আলাদা mask। Object detection + per-instance segmentation।

কেন্দ্রীয় ধারণা

Faster R-CNN-এর প্রতিটি RoI-এ — মাত্র এক additional branch (mask head)। Multi-task learning, decoupled prediction। Simple addition, dramatic capability।

২ · Mask R-CNN architecture

Shared components (Faster R-CNN থেকে)

  • Backbone: ResNet-50/101 + FPN।
  • RPN: region proposal।
  • RoI Align: feature extract (RoI Pool → RoI Align)।

Three parallel heads

  • Classification head: RoI → class probability।
  • Bbox regression head: RoI → refined bbox।
  • Mask head (new): RoI → 28×28 binary mask per class।

৩ · RoI Align

RoI Pool-এর problem: integer quantization-এ sub-pixel misalignment। Mask precision-এ এটি critical।

  • RoI Pool: integer round → bin division integer।
  • RoI Align: continuous coordinate, bilinear interpolation।
  • 4 sample point per bin, average।
  • Sub-pixel accurate।

Effect on mask AP: 24.5 → 30.3 (+5.8%) — vital improvement।

৪ · Mask head detail

  • RoI feature 14×14 → 4 conv 3×3 → upsample 28×28 → 1 conv 1×1।
  • Per-class mask channel: $K \times 28 \times 28$ (where $K$ = num classes)।
  • Inference: only target class-এর mask use।
  • Output binary — per pixel sigmoid।

৫ · Loss function

Multi-task loss:

$$\mathcal{L} = \mathcal{L}_{\text{cls}} + \mathcal{L}_{\text{bbox}} + \mathcal{L}_{\text{mask}}$$

  • $\mathcal{L}_{\text{cls}}$: softmax cross-entropy।
  • $\mathcal{L}_{\text{bbox}}$: smooth L1 (Faster R-CNN)।
  • $\mathcal{L}_{\text{mask}}$: per-pixel binary cross-entropy (only on positive RoI)।

Decoupling: mask loss only on ground-truth class — class prediction-এর সাথে competition নেই।

৬ · Why decoupling matters

Standard FCN-based segmentation: mask head per-pixel softmax over all class। Class compete।

Mask R-CNN: per-class binary mask। Classification & mask separate task। Each class own mask binary।

Result: mask quality higher, training stable।

Mask R-CNN = Faster R-CNN + Photoshop "magic wand" per object। Detect first, then trace boundary precisely। Two-stage philosophy ideal for fine-grained mask।
Mask R-CNN — Faster R-CNN + mask head 📷 Image Backbone+FPN RPN RoI Align Class head FC → softmax Bbox head FC → 4 coord Mask head ⭐ conv → 28×28 mask Per-instance: class + bbox + binary mask decoupled — class prediction independent of mask Multi-task heads — RoI Align — sub-pixel accuracy
Mask R-CNN — Faster R-CNN-এর সাথে mask head। RoI Align দিয়ে precision।

৭ · PyTorch-এ Mask R-CNN

Python · PyTorch 
import torch
from torchvision.models.detection import maskrcnn_resnet50_fpn
from torchvision.models.detection import MaskRCNN_ResNet50_FPN_Weights

w = MaskRCNN_ResNet50_FPN_Weights.COCO_V1
model = maskrcnn_resnet50_fpn(weights=w)
model.eval()

img = torch.rand(3, 800, 800)
with torch.no_grad():
    out = model([img])

det = out[0]
print("Boxes:", det['boxes'].shape)         # (N, 4)
print("Labels:", det['labels'].shape)       # (N,)
print("Scores:", det['scores'].shape)       # (N,)
print("Masks:", det['masks'].shape)         # (N, 1, H, W) — float

# Filter & threshold mask
keep = det['scores'] > 0.7
final_masks = det['masks'][keep]
binary_masks = (final_masks > 0.5).squeeze(1)
print("Confident detections with masks:", binary_masks.shape)

    


    

      Mask output float (sigmoid) — threshold 0.5-এ binary mask। Per-pixel confidence reveal।
    


    
৮ · Mask R-CNN extensions

    
    
      
  • Cascade Mask R-CNN: multi-stage refinement।
    • 
      
  • HTC (Hybrid Task Cascade): task interleaving।
    • 
      
  • PointRend: uncertain pixel refine separately।
    • 
      
  • Mask2Former: universal segmentation, transformer-based।
    • 
    


    
৯ · Use cases

    
    
      
  • Autonomous driving: per-vehicle, per-pedestrian instance।
    • 
      
  • Robotics: object grasping।
    • 
      
  • Retail: Amazon Go-style automatic checkout।
    • 
      
  • Medical: per-cell, per-lesion analysis।
    • 
      
  • Wildlife: animal census।
    • 
      
  • AR: virtual try-on, occlusion।
    • 
      
  • Bangladesh use: garment defect detect, fish counting।
    • 
    


    
১০ · Detectron2

    
    
      
  • Facebook AI Research-এর library।
    • 
      
  • Mask R-CNN, Cascade, PointRend included।
    • 
      
  • Modular, well-documented।
    • 
      
  • Production-grade।
    • 
    


    

      Mask R-CNN slow on CPU (~5 sec/image)। Realtime mobile-এ MobileNet backbone use। Server inference fine — 10 FPS GPU।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      
        প্র ০১
        Mask R-CNN-এ per-class binary mask — vs FCN-style multi-class softmax mask। Decoupling-এর rationale?
      
      

        
এটি instance segmentation-এর fundamental insight।

        
FCN-style (semantic):

        
    
          
  • Per pixel — softmax over $C$ class।
    • 
          
  • Pixel can be only one class।
    • 
          
  • Class compete at pixel level।
    • 
        

        
Mask R-CNN per-class binary:

        
    
          
  • $K$ binary mask, one per class।
    • 
          
  • Sigmoid (not softmax)।
    • 
          
  • Pixel can be multiple class technically।
    • 
          
  • RoI-level class decide first।
    • 
        

        
Why decouple:

        
    
          
  • Class prediction independent of mask।
    • 
          
  • Mask training simpler — no inter-class compete।
    • 
          
  • Per-instance binary easier to learn।
    • 
          
  • Empirically: AP 30.3 (decoupled) vs 24.7 (coupled)।
    • 
        

        
Mask quality reason:

        
    
          
  • Coupled: pixel must "compete" — boundary fuzzy।
    • 
          
  • Decoupled: per-class focused — boundary sharp।
    • 
        

        
Inference:

        
    
          
  • RoI classify first → predicted class।
    • 
          
  • Use that class's mask channel।
    • 
          
  • Other class's mask discard।
    • 
        

        
Memory cost:

        
    
          
  • $K$ mask per RoI — bigger output।
    • 
          
  • COCO 80 class — 80 × 28 × 28 mask per RoI।
    • 
          
  • Acceptable trade-off।
    • 
        

        
Modern alternative:

        
    
          
  • Class-agnostic mask + classification — single mask per RoI।
    • 
          
  • Mask2Former — single output mask per query, class via cross-attention।
    • 
          
  • Decoupling philosophy persists।
    • 
        

        
মূল উপলব্ধি: Multi-task neural network — task structure design matters। Decoupling related but distinct tasks often improves both।

      

    


    

      
        প্র ০২
        RoI Align bilinear interpolation use করে। Trilinear বা cubic চাইলে আরও accurate হত? Trade-off?
      
      

        
Interpolation choice — sampling theory-এর domain।

        
Bilinear interpolation:

        
    
          
  • 4 nearest neighbor weighted average।
    • 
          
  • Linear in each dimension।
    • 
          
  • Fast, simple।
    • 
          
  • Slightly blurry on sharp edge।
    • 
        

        
Bicubic alternative:

        
    
          
  • 16 neighbor (4×4) cubic spline।
    • 
          
  • Sharper edges।
    • 
          
  • 4x more compute।
    • 
          
  • Can overshoot (halo)।
    • 
        

        
Lanczos/sinc-based:

        
    
          
  • Highest theoretical quality।
    • 
          
  • Many neighbor (8x8 to 16x16)।
    • 
          
  • Computational expensive।
    • 
        

        
Empirical Mask R-CNN:

        
    
          
  • Bilinear: 30.3 AP_mask।
    • 
          
  • Bicubic — marginal improvement, not justifying compute।
    • 
          
  • Trilinear (3D feature) — for 3D detection only।
    • 
        

        
Why bilinear sufficient:

        
    
          
  • Feature map already smooth (post-conv)।
    • 
          
  • Subsequent conv re-aggregate।
    • 
          
  • Sharp edge at output layer (final 1×1 conv)।
    • 
        

        
Modern alternative:

        
    
          
  • Deformable convolution — learnable sampling location।
    • 
          
  • Replace static interpolation।
    • 
          
  • Used in DCNv2, Deformable DETR।
    • 
        

        
Hardware acceleration:

        
    
          
  • Bilinear — GPU bilinear sampling primitive।
    • 
          
  • Cubic — more memory access — slow।
    • 
        

        
মূল উপলব্ধি: Bilinear "good enough" for feature map। Sub-pixel accuracy bilinear-এ achieve possible। Diminishing return higher-order।

      

    


    

      
        প্র ০৩
        Mask R-CNN-এর mask 28×28 fixed। Large object (e.g., entire wall)-এ এই resolution insufficient — কী করব?
      
      

        
এটি Mask R-CNN-এর notable limitation। Large object-এ blocky mask।

        
Why 28×28:

        
    
          
  • RoI Align output 14×14 → upsample 28×28।
    • 
          
  • RoI 200×200 image-এ — 28×28 mask = 7x sub-sample।
    • 
          
  • RoI 1000×1000 — 28×28 mask = 36x sub-sample।
    • 
          
  • Large object — coarse mask।
    • 
        

        
Solutions:

        
    
          
  • Mask resolution increase: 56×56 — slower, marginal gain।
    • 
          
  • Refinement post-process: CRF, dense prediction।
    • 
          
  • PointRend (২০২০): uncertain pixel-এ point-wise refine — high-res mask।
    • 
          
  • Cascade Mask R-CNN: multi-stage refine।
    • 
        

        
PointRend specifically:

        
    
          
  • Initial mask coarse (28×28)।
    • 
          
  • Identify "uncertain" pixel (boundary)।
    • 
          
  • Sample those — fine prediction।
    • 
          
  • Original 28×28 + refinement → high-res mask।
    • 
          
  • Effective for large object।
    • 
        

        
Mask2Former approach:

        
    
          
  • Per-pixel mask at full resolution।
    • 
          
  • No RoI cropping/resize।
    • 
          
  • Memory cost বেশি।
    • 
        

        
SAM (২০২৩):

        
    
          
  • Promptable, full-resolution mask।
    • 
          
  • Decoupled from detection।
    • 
          
  • Mask R-CNN বাড়ির মাঝারি instance, SAM single-object high-res।
    • 
        

        
Practical recommendation:

        
    
          
  • Small/medium object: Mask R-CNN OK।
    • 
          
  • Large object boundary critical: PointRend/Mask2Former।
    • 
          
  • Pixel-perfect: SAM with detection prompts।
    • 
        

        
মূল উপলব্ধি: Architecture trade-off — fixed resolution simple but limit। Modern direction — adaptive/multi-resolution।

      

    


    

      
        প্র ০৪
        Bangladesh garment factory defect detection — Mask R-CNN choose কেন? Realtime YOLO + segmentation কোথায় better?
      
      

        
Real production decision। Bangladesh RMG industry — defect detection critical।

        
Garment defect detection profile:

        
    
          
  • Defect type: stain, hole, color, stitch।
    • 
          
  • Resolution: high (4K camera typical)।
    • 
          
  • Speed: 100-1000 garment/min line।
    • 
          
  • Accuracy: critical — miss 1% = customer return।
    • 
        

        
Mask R-CNN strengths:

        
    
          
  • Per-defect instance — count exact।
    • 
          
  • Pixel-precise boundary — defect area measure।
    • 
          
  • Multi-class — defect type categorize।
    • 
          
  • High accuracy on small defects।
    • 
        

        
Mask R-CNN weaknesses:

        
    
          
  • Slow — 10 FPS GPU max।
    • 
          
  • Mobile deploy hard।
    • 
          
  • Heavy GPU requirement।
    • 
        

        
YOLOv8-seg alternative:

        
    
          
  • YOLOv8 — segmentation variant।
    • 
          
  • 30+ FPS।
    • 
          
  • Mobile-deployable।
    • 
          
  • Slightly less accurate।
    • 
        

        
Decision matrix for Bangladesh garment:

        
    
          
  • Primary inspection (high-cam, server): Mask R-CNN — accuracy priority।
    • 
          
  • Mobile inspection (worker handheld): YOLOv8-seg।
    • 
          
  • Realtime line camera: YOLOv8-seg।
    • 
          
  • Final QC desk: Mask R-CNN, slower but accurate।
    • 
        

        
Pipeline approach:

        
    
          
  • Stage 1: YOLO realtime — coarse defect detect।
    • 
          
  • Stage 2: Mask R-CNN — flagged image precise analyze।
    • 
          
  • Hybrid speed + accuracy।
    • 
        

        
Bangladesh implementation:

        
    
          
  • BGMEA — RMG industry body — AI initiative।
    • 
          
  • Local startups (Doer, etc.) — defect detection product।
    • 
          
  • Investment: GPU server $5K-20K।
    • 
          
  • ROI: defect rate 5% → 1% — millions saved।
    • 
        

        
Domain considerations:

        
    
          
  • Fabric color variability — augment heavy।
    • 
          
  • Lighting variability — controlled lightbox।
    • 
          
  • New defect emergence — model continuously update।
    • 
        

        
মূল উপলব্ধি: Production system — single model rare। Hybrid pipeline (fast + accurate) optimal। Bangladesh RMG-এ AI deployment significant business potential।

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Mask R-CNN inference: torchvision-এ pretrained Mask R-CNN load + inference example।
        
    
          
          
    
            
    from torchvision.models.detection import maskrcnn_resnet50_fpn
m = maskrcnn_resnet50_fpn(weights='DEFAULT').eval()
out = m([img_tensor])
masks = (out[0]['masks'] > 0.5).squeeze(1)
    
          
    
        
    
      
    2. 

      
  2. 
        Custom finetune: Mask R-CNN custom 5-class dataset adapt।
        
    
          
          
    
            
    from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor
m = maskrcnn_resnet50_fpn(weights='DEFAULT')
n_class = 6  # 5 + background
in_features = m.roi_heads.box_predictor.cls_score.in_features
m.roi_heads.box_predictor = FastRCNNPredictor(in_features, n_class)
in_features_mask = m.roi_heads.mask_predictor.conv5_mask.in_channels
m.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask, 256, n_class)
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Mask R-CNN vs YOLOv8-seg vs Mask2Former — কখন কোনটা?
        
    
          
          
    
            
    Mask R-CNN — accuracy & flexibility। YOLOv8-seg — realtime speed। Mask2Former — universal segmentation, transformer SOTA, accuracy maximum কিন্তু slow।
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    
    

      কোড রানার কাজ না করলে? Google Colab use করুন।
    

  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ২২ · U-Net