পাঠ ২৩ · ৩৫-এর মধ্যে · মডিউল ৩

Home / AI Courses / Computer Vision / Mask R-CNN

Mask R-CNN

Mask R-CNN — Faster R-CNN with mask head

৭ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

Mask R-CNN — Faster R-CNN-এর extension
RoI Align — quantization-free region pooling
Mask head architecture
PyTorch-এ inference + finetune

১ · Mask R-CNN-এর জন্ম

Kaiming He et al. (২০১৭, Facebook AI Research). Faster R-CNN-এর extension. Conceptually simple, accuracy high.

Goal: instance segmentation — per-object pixel mask. প্রতিটি car-এর আলাদা mask. Object detection + per-instance segmentation.

কেন্দ্রীয় ধারণা

Faster R-CNN-এর প্রতিটি RoI-এ — মাত্র এক additional branch (mask head). Multi-task learning, decoupled prediction. Simple addition, dramatic capability.

২ · Mask R-CNN architecture

Shared components (Faster R-CNN থেকে)

Backbone: ResNet-50/101 + FPN.
RPN: region proposal.
RoI Align: feature extract (RoI Pool → RoI Align).

Three parallel heads

Classification head: RoI → class probability.
Bbox regression head: RoI → refined bbox.
Mask head (new): RoI → 28×28 binary mask per class.

৩ · RoI Align

RoI Pool-এর problem: integer quantization-এ sub-pixel misalignment. Mask precision-এ এটি critical.

RoI Pool: integer round → bin division integer.
RoI Align: continuous coordinate, bilinear interpolation.
4 sample point per bin, average.
Sub-pixel accurate.

Effect on mask AP: 24.5 → 30.3 (+5.8%) — vital improvement.

৪ · Mask head detail

RoI feature 14×14 → 4 conv 3×3 → upsample 28×28 → 1 conv 1×1.
Per-class mask channel: $K \times 28 \times 28$ (where $K$ = num classes).
Inference: only target class-এর mask use.
Output binary — per pixel sigmoid.

৫ · Loss function

Multi-task loss:

$$\mathcal{L} = \mathcal{L}_{\text{cls}} + \mathcal{L}_{\text{bbox}} + \mathcal{L}_{\text{mask}}$$

$\mathcal{L}_{\text{cls}}$: softmax cross-entropy.
$\mathcal{L}_{\text{bbox}}$: smooth L1 (Faster R-CNN).
$\mathcal{L}_{\text{mask}}$: per-pixel binary cross-entropy (only on positive RoI).

Decoupling: mask loss only on ground-truth class — class prediction-এর সাথে competition নেই।

৬ · Why decoupling matters

Standard FCN-based segmentation: mask head per-pixel softmax over all class. Class compete.

Mask R-CNN: per-class binary mask. Classification & mask separate task. Each class own mask binary.

Result: mask quality higher, training stable.

Mask R-CNN = Faster R-CNN + Photoshop "magic wand" per object. Detect first, then trace boundary precisely. Two-stage philosophy ideal for fine-grained mask.

Mask R-CNN — Faster R-CNN-এর সাথে mask head. RoI Align দিয়ে precision.

৭ · PyTorch-এ Mask R-CNN

Python · PyTorch

import torch
from torchvision.models.detection import maskrcnn_resnet50_fpn
from torchvision.models.detection import MaskRCNN_ResNet50_FPN_Weights

w = MaskRCNN_ResNet50_FPN_Weights.COCO_V1
model = maskrcnn_resnet50_fpn(weights=w)
model.eval()

img = torch.rand(3, 800, 800)
with torch.no_grad():
    out = model([img])

det = out[0]
print("Boxes:", det['boxes'].shape)         # (N, 4)
print("Labels:", det['labels'].shape)       # (N,)
print("Scores:", det['scores'].shape)       # (N,)
print("Masks:", det['masks'].shape)         # (N, 1, H, W) — float

# Filter & threshold mask
keep = det['scores'] > 0.7
final_masks = det['masks'][keep]
binary_masks = (final_masks > 0.5).squeeze(1)
print("Confident detections with masks:", binary_masks.shape)

Mask output float (sigmoid) — threshold 0.5-এ binary mask. Per-pixel confidence reveal.

৮ · Mask R-CNN extensions

Cascade Mask R-CNN: multi-stage refinement.
HTC (Hybrid Task Cascade): task interleaving.
PointRend: uncertain pixel refine separately.
Mask2Former: universal segmentation, transformer-based.

৯ · Use cases

Autonomous driving: per-vehicle, per-pedestrian instance.
Robotics: object grasping.
Retail: Amazon Go-style automatic checkout.
Medical: per-cell, per-lesion analysis.
Wildlife: animal census.
AR: virtual try-on, occlusion.
Bangladesh use: garment defect detect, fish counting.

১০ · Detectron2

Facebook AI Research-এর library.
Mask R-CNN, Cascade, PointRend included.
Modular, well-documented.
Production-grade.

Mask R-CNN slow on CPU (~5 sec/image). Realtime mobile-এ MobileNet backbone use. Server inference fine — 10 FPS GPU.

ভাবনার প্রশ্ন

প্র ০১ Mask R-CNN-এ per-class binary mask — vs FCN-style multi-class softmax mask. Decoupling-এর rationale?

এটি instance segmentation-এর fundamental insight.

FCN-style (semantic):

Per pixel — softmax over $C$ class.
Pixel can be only one class.
Class compete at pixel level.

Mask R-CNN per-class binary:

$K$ binary mask, one per class.
Sigmoid (not softmax).
Pixel can be multiple class technically.
RoI-level class decide first.

Why decouple:

Class prediction independent of mask.
Mask training simpler — no inter-class compete.
Per-instance binary easier to learn.
Empirically: AP 30.3 (decoupled) vs 24.7 (coupled).

Mask quality reason:

Coupled: pixel must "compete" — boundary fuzzy.
Decoupled: per-class focused — boundary sharp.

Inference:

RoI classify first → predicted class.
Use that class's mask channel.
Other class's mask discard.

Memory cost:

$K$ mask per RoI — bigger output.
COCO 80 class — 80 × 28 × 28 mask per RoI.
Acceptable trade-off.

Modern alternative:

Class-agnostic mask + classification — single mask per RoI.
Mask2Former — single output mask per query, class via cross-attention.
Decoupling philosophy persists.

মূল উপলব্ধি: Multi-task neural network — task structure design matters. Decoupling related but distinct tasks often improves both.

প্র ০২ RoI Align bilinear interpolation use করে। Trilinear বা cubic চাইলে আরও accurate হত? Trade-off?

Interpolation choice — sampling theory-এর domain.

Bilinear interpolation:

4 nearest neighbor weighted average.
Linear in each dimension.
Fast, simple.
Slightly blurry on sharp edge.

Bicubic alternative:

16 neighbor (4×4) cubic spline.
Sharper edges.
4x more compute.
Can overshoot (halo).

Lanczos/sinc-based:

Highest theoretical quality.
Many neighbor (8x8 to 16x16).
Computational expensive.

Empirical Mask R-CNN:

Bilinear: 30.3 AP_mask.
Bicubic — marginal improvement, not justifying compute.
Trilinear (3D feature) — for 3D detection only.

Why bilinear sufficient:

Feature map already smooth (post-conv).
Subsequent conv re-aggregate.
Sharp edge at output layer (final 1×1 conv).

Modern alternative:

Deformable convolution — learnable sampling location.
Replace static interpolation.
Used in DCNv2, Deformable DETR.

Hardware acceleration:

Bilinear — GPU bilinear sampling primitive.
Cubic — more memory access — slow.

মূল উপলব্ধি: Bilinear "good enough" for feature map. Sub-pixel accuracy bilinear-এ achieve possible. Diminishing return higher-order.

প্র ০৩ Mask R-CNN-এর mask 28×28 fixed. Large object (e.g., entire wall)-এ এই resolution insufficient — কী করব?

এটি Mask R-CNN-এর notable limitation. Large object-এ blocky mask.

Why 28×28:

RoI Align output 14×14 → upsample 28×28.
RoI 200×200 image-এ — 28×28 mask = 7x sub-sample.
RoI 1000×1000 — 28×28 mask = 36x sub-sample.
Large object — coarse mask.

Solutions:

Mask resolution increase: 56×56 — slower, marginal gain.
Refinement post-process: CRF, dense prediction.
PointRend (২০২০): uncertain pixel-এ point-wise refine — high-res mask.
Cascade Mask R-CNN: multi-stage refine.

PointRend specifically:

Initial mask coarse (28×28).
Identify "uncertain" pixel (boundary).
Sample those — fine prediction.
Original 28×28 + refinement → high-res mask.
Effective for large object.

Mask2Former approach:

Per-pixel mask at full resolution.
No RoI cropping/resize.
Memory cost বেশি।

SAM (২০২৩):

Promptable, full-resolution mask.
Decoupled from detection.
Mask R-CNN বাড়ির মাঝারি instance, SAM single-object high-res.

Practical recommendation:

Small/medium object: Mask R-CNN OK.
Large object boundary critical: PointRend/Mask2Former.
Pixel-perfect: SAM with detection prompts.

মূল উপলব্ধি: Architecture trade-off — fixed resolution simple but limit. Modern direction — adaptive/multi-resolution.

প্র ০৪ Bangladesh garment factory defect detection — Mask R-CNN choose কেন? Realtime YOLO + segmentation কোথায় better?

Real production decision. Bangladesh RMG industry — defect detection critical.

Garment defect detection profile:

Defect type: stain, hole, color, stitch.
Resolution: high (4K camera typical).
Speed: 100-1000 garment/min line.
Accuracy: critical — miss 1% = customer return.

Mask R-CNN strengths:

Per-defect instance — count exact.
Pixel-precise boundary — defect area measure.
Multi-class — defect type categorize.
High accuracy on small defects.

Mask R-CNN weaknesses:

Slow — 10 FPS GPU max.
Mobile deploy hard.
Heavy GPU requirement.

YOLOv8-seg alternative:

YOLOv8 — segmentation variant.
30+ FPS.
Mobile-deployable.
Slightly less accurate.

Decision matrix for Bangladesh garment:

Primary inspection (high-cam, server): Mask R-CNN — accuracy priority.
Mobile inspection (worker handheld): YOLOv8-seg.
Realtime line camera: YOLOv8-seg.
Final QC desk: Mask R-CNN, slower but accurate.

Pipeline approach:

Stage 1: YOLO realtime — coarse defect detect.
Stage 2: Mask R-CNN — flagged image precise analyze.
Hybrid speed + accuracy.

Bangladesh implementation:

BGMEA — RMG industry body — AI initiative.
Local startups (Doer, etc.) — defect detection product.
Investment: GPU server $5K-20K.
ROI: defect rate 5% → 1% — millions saved.

Domain considerations:

Fabric color variability — augment heavy.
Lighting variability — controlled lightbox.
New defect emergence — model continuously update.

মূল উপলব্ধি: Production system — single model rare. Hybrid pipeline (fast + accurate) optimal. Bangladesh RMG-এ AI deployment significant business potential.

অনুশীলন

Mask R-CNN inference: torchvision-এ pretrained Mask R-CNN load + inference example.

from torchvision.models.detection import maskrcnn_resnet50_fpn
m = maskrcnn_resnet50_fpn(weights='DEFAULT').eval()
out = m([img_tensor])
masks = (out[0]['masks'] > 0.5).squeeze(1)

Custom finetune: Mask R-CNN custom 5-class dataset adapt.

from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor
m = maskrcnn_resnet50_fpn(weights='DEFAULT')
n_class = 6  # 5 + background
in_features = m.roi_heads.box_predictor.cls_score.in_features
m.roi_heads.box_predictor = FastRCNNPredictor(in_features, n_class)
in_features_mask = m.roi_heads.mask_predictor.conv5_mask.in_channels
m.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask, 256, n_class)

ভাবুন: Mask R-CNN vs YOLOv8-seg vs Mask2Former — কখন কোনটা?

Mask R-CNN — accuracy & flexibility. YOLOv8-seg — realtime speed. Mask2Former — universal segmentation, transformer SOTA, accuracy maximum কিন্তু slow.