পাঠ ১৮ · ৩৫-এর মধ্যে · মডিউল ৩

Home / AI Courses / Computer Vision / R-CNN family

R-CNN ও Faster R-CNN

The R-CNN family — region-based detectors evolution

৮ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

R-CNN evolution — slow → fast → faster
RoI pooling ও RoI Align
Region Proposal Network (RPN)
PyTorch-এ Faster R-CNN inference

১ · R-CNN (২০১৪) — beginning

Ross Girshick et al. — "Rich feature hierarchies for accurate object detection"। AlexNet-এর উপর ভিত্তি করে CNN-এর প্রথম successful detection।

Pipeline:

Selective Search: classical algorithm — ~2000 region proposal generate।
Warp: each region 227×227-এ resize।
CNN feature: each warped region আলাদাভাবে AlexNet-এ feed।
SVM: per-class linear SVM classify।
Bbox regression: linear regressor refine box।

সমস্যা: 2000 region × CNN forward = অসম্ভব slow। Train-এ days, inference 47 sec/image।

কেন্দ্রীয় ধারণা

R-CNN proof — CNN feature SIFT/HOG-এর চেয়ে ভাল detection-এ। কিন্তু compute prohibitive। পরের iteration এই sharing solve করে।

২ · Fast R-CNN (২০১৫)

Girshick alone — "Fast R-CNN"। Key insight: পুরো image-কে একবার CNN-এ পাঠাও, তারপর region propose feature map-এ।

Pipeline:

Image → CNN backbone → feature map (e.g., 14×14×512)।
Selective Search → ~2000 proposal (image space)।
Each proposal → corresponding region in feature map।
RoI Pooling: each region-কে fixed size 7×7-এ pool।
FC layers → classification + bbox regression।

Speedup: CNN forward 1 time per image (not 2000 per region)। 25x train speed, 213x inference speed (still ~2 sec)।

৩ · RoI Pooling

Variable-sized region → fixed-sized output. Algorithm:

Region in feature map (e.g., 30×40)।
Divide into 7×7 grid (target size)।
Each grid cell — max pool corresponding feature map cells।
Output 7×7 → flatten → FC।

Issue: integer coordinates → quantization error → misalignment।

৪ · Faster R-CNN (২০১৫)

Ren et al. — "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks"।

Key innovation: Selective Search-এর বদলে neural Region Proposal Network (RPN)। Fully neural pipeline।

Pipeline:

Image → CNN backbone → feature map।
RPN: per-pixel anchor (multi-scale, multi-ratio) → objectness + box refine।
Top-N proposal (~300)।
RoI Pooling → FC heads: class + bbox।

Speed: ~5 FPS (VGG backbone)। ~17 FPS (ResNet)। Realtime threshold-এ approach।

৫ · Region Proposal Network (RPN)

3×3 conv on feature map → intermediate feature।
Two parallel 1×1 conv head:
- Cls: per anchor — object/background score।
- Reg: per anchor — bbox refinement (dx, dy, dw, dh)।
Anchor: 9 per pixel (3 scale × 3 aspect ratio)।
Trained jointly with main detector।

৬ · Anchor box concept

Pre-defined boxes at each spatial location।
Standard: 3 scales (128, 256, 512 px) × 3 aspect ratios (1:1, 1:2, 2:1) = 9।
Each anchor predict offset from base — easier than absolute box।
Match with ground-truth via IoU > 0.7 (positive), <0.3 (negative)।

R-CNN → Fast → Faster R-CNN — engineering iteration-এর textbook example। Each iteration removed bottleneck: shared compute, learnable proposal, end-to-end training।

R-CNN family — engineering iteration-এর model। প্রতিটি step একটি bottleneck remove।

৭ · Mask R-CNN (২০১৭)

Faster R-CNN-এর extension — instance segmentation। He et al. ICCV 2017 best paper।

RoI Pooling → RoI Align: bilinear interpolation, quantization free।
Mask head: RoI feature → small FCN → per-class binary mask।
Multi-task: classification + bbox + mask।
Mask R-CNN — instance segmentation-এর gold standard।

৮ · PyTorch-এ Faster R-CNN

Python · PyTorch

import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision.models.detection import FasterRCNN_ResNet50_FPN_Weights

w = FasterRCNN_ResNet50_FPN_Weights.COCO_V1
model = fasterrcnn_resnet50_fpn(weights=w)
model.eval()

# Inference
img = torch.rand(3, 800, 800)   # any size, FPN handles
with torch.no_grad():
    out = model([img])

# Output: list of dict per image
det = out[0]
print("Boxes:", det['boxes'].shape)     # (N, 4) — xyxy
print("Labels:", det['labels'].shape)   # (N,) — class index
print("Scores:", det['scores'].shape)   # (N,) — confidence

# Filter confidence > 0.5
keep = det['scores'] > 0.5
print(f"\n{keep.sum()} confident detections")

torchvision Faster R-CNN — COCO 80 class pretrained। Custom dataset finetune-এ — last classifier replace।

৯ · Two-stage variants

Cascade R-CNN (২০১৮): multi-stage classifier with increasing IoU threshold।
Mask R-CNN: + segmentation।
Sparse R-CNN: learnable proposal — DETR-like end-to-end।
HTC (Hybrid Task Cascade): multiple task interleave।

১০ · Two-stage modern uses

Maximum accuracy benchmarks।
Medical imaging — high precision matters।
Custom annotation type (keypoint + mask + bbox)।
Mask R-CNN — instance segmentation production।

Two-stage detector slow but flexible। YOLO-র simplicity-র জন্য realtime মুহূর্তে — কিন্তু high accuracy + custom heads-এ Faster R-CNN family champion।

ভাবনার প্রশ্ন

প্র ০১ RoI Pooling ও RoI Align-এর পার্থক্য কী? Mask R-CNN-এ pooling থেকে align-এ switch কেন?

এটি subtle কিন্তু segmentation accuracy-তে বিরাট impact।

RoI Pooling-এর problem:

Region coordinate continuous (e.g., 18.6)।
Feature map integer index।
Step 1 quantization: round to nearest integer (18.6 → 19)।
Step 2 quantization: bin division integer।
Total — sub-pixel shift accumulate।

Effect:

Classification — minor (pool over region)।
Bbox refine — minor।
Mask prediction — major! Pixel-level accuracy needed।

RoI Align solution:

No integer rounding।
Bin boundary continuous।
Bilinear interpolation — sample at exact continuous coordinate।
4 sample point per bin → average।

Mathematical:

RoI Pooling: $\text{out}[i,j] = \max_{x,y \in \text{quantized bin}} F[\lfloor x \rfloor, \lfloor y \rfloor]$।
RoI Align: $\text{out}[i,j] = \frac{1}{4} \sum \text{bilinear}(F, x_k, y_k)$।

Mask R-CNN paper-এর result:

RoI Pool: AP_mask 24.5।
RoI Align: AP_mask 30.3।
+5.8% — substantial।

Computational cost:

RoI Align — slightly more (4 sample per bin, bilinear)।
Trade-off worth it।

Used elsewhere:

Mask R-CNN, Cascade Mask R-CNN।
Detection-এ Faster R-CNN-ও modern implementation-এ।
Anywhere precise feature extraction — keypoint detection।

মূল উপলব্ধি: "Implementation detail" architecture decision-এর সমান impact। Sub-pixel accuracy — segmentation, fine-grained detection-এর key।

প্র ০২ RPN-এ anchor box-এর choice (3 scale × 3 ratio) — domain-specific tune করা দরকার? Bangla street sign detect-এ কী anchor?

Anchor design — Faster R-CNN-এর underrated tunable। Default ImageNet-COCO-এর জন্য optimal, custom-এর জন্য না।

Default anchor:

Scales: 128, 256, 512 px।
Ratios: 1:1, 1:2, 2:1।
9 anchor per spatial location।
FPN-এ — scale per pyramid level।

Why this matters:

Anchor — "guess" — closest match-এ gradient flow।
Bad anchor → slow convergence, missed object।
Right anchor → fast train, high recall।

Bangla street sign analysis:

Sign typically square (1:1)।
Street view — usually 30-150 px (camera dist)।
Vertical signs (parking restriction) — 1:3।
Horizontal banner — 3:1।

Custom anchor:

Scales: 32, 64, 128 (smaller — sign typically distant)।
Ratios: 1:1, 1:3, 3:1 (vertical/horizontal banner)।
9 anchor still।

K-means anchor (YOLOv2-style):

Training set bbox-এ k-means clustering।
Cluster center = optimal anchor।
Domain-specific automatic।

YOLO modern (v3-v8):

Auto-anchor tool — dataset analyze, suggest anchor।
Training-time auto-tune।

Anchor-free alternatives:

FCOS, CenterNet — no anchor concept।
DETR — set prediction।
Modern trend — anchor-free easier।

Practical recommendation:

Stage 1: COCO default — quick baseline।
Stage 2: dataset analyze — k-means anchor।
Stage 3: anchor-free try (FCOS, RT-DETR)।

মূল উপলব্ধি: Anchor — historical hack, but tunable। Domain-specific design — accuracy bump। Modern anchor-free — simpler design-এর direction।

প্র ০৩ Faster R-CNN-এ FPN (Feature Pyramid Network) integration। Multi-scale feature কেন important? ছোট object detect-এর কী secret?

FPN (Lin et al., 2017) — detection-এর underrated breakthrough। Small object detection-এ revolution।

Problem before FPN:

Faster R-CNN — final feature map (high-level, 32× downsample)।
Small object — final map-এ খুবই ছোট spatial extent।
Large object — captured ভাল।
Small object detection mAP < 25%।

Single-scale problem:

Image pyramid (multi-scale input) — slow।
Feature pyramid (different layer) — early shallow features semantic poor।

FPN insight:

Use different stage feature map (P2, P3, P4, P5)।
Top-down pathway — high-level semantic propagate down।
Lateral connection — combine high & low-level।
Result: each level — semantically rich + spatially precise।

FPN structure:

P5 = lateral(C5)
P4 = lateral(C4) + upsample(P5)
P3 = lateral(C3) + upsample(P4)
P2 = lateral(C2) + upsample(P3)

Detection per level:

Small object → P2 (high resolution)।
Medium → P3, P4।
Large → P5।
Anchor scale per level।

Result on COCO:

Faster R-CNN: 35.0 mAP।
Faster R-CNN + FPN: 39.4 mAP।
Small object AP: 17 → 24 (+41%)।

FPN modern adoption:

Almost all detection backbone।
YOLO v3+ uses FPN-like (PANet)।
RetinaNet — FPN baseline।
Mask R-CNN — FPN essential।

Variants:

PANet: bottom-up addition over FPN।
BiFPN (EfficientDet): bidirectional।
NAS-FPN: architecture searched।
Recursive FPN: multi-pass।

Bangladesh use case:

Crowded street — pedestrian distance variable।
Far pedestrian small in pixel — FPN essential।
Drone aerial photo — small object dominant।
Default detector + FPN strongly recommended।

মূল উপলব্ধি: Single-scale CNN insufficient real-world detection। FPN — neck design-এর foundational pattern। Backbone alone যথেষ্ট না।

প্র ০৪ Faster R-CNN ২০১৫-র। ১১ বছর পর — DETR, YOLOv8 আছে। Faster R-CNN আজও কোথায় কেন ব্যবহার?

Honest assessment — niche কিন্তু important।

Where Faster R-CNN still wins:

Custom heads: Mask R-CNN, keypoint, dense pose।
Small object accuracy: RoI alignment fine-grained।
Custom annotation: non-standard task — adapt easy।
Research baselines: well-understood, tunable।
Production legacy: 2017-2020 systems still running।

Where YOLO/DETR wins:

Realtime: mobile, embedded।
Single-task standard detection: no need for heads।
Modern accuracy: DINO-DETR > Faster R-CNN।
Simplicity: single loss, easier debug।

Specific scenarios:

Mask R-CNN — instance segmentation: still strong choice।
HTC, Cascade Mask R-CNN: highest mAP achievable।
Detectron2 (Meta library): Faster R-CNN central।
Medical (FDA-validated): regulatory acceptance।

Modern alternatives for two-stage role:

Sparse R-CNN: two-stage with learnable proposal।
Cascade DINO: DETR-based cascade।
Mask2Former: universal segmentation।

Bangladesh deployment:

Realtime traffic app: YOLOv8।
Medical research: Mask R-CNN।
Document analysis: Faster R-CNN custom head।
Industrial inspection: Faster R-CNN — accuracy critical।

Skill recommendation:

Master Faster R-CNN concept — foundational।
Use Mask R-CNN — segmentation-এ default।
Try DETR/RT-DETR — future direction।
YOLO — realtime mainstream।

মূল উপলব্ধি: Faster R-CNN — like email (1971) or HTTP (1991) — old, still essential foundation। Modern alternatives flashier, but core concept everywhere।

অনুশীলন

RPN output: 800×600 image, ResNet backbone (32x downsample), 9 anchor — RPN total proposal candidate?

Feature map: $25 \times 19$। Per location 9 anchor → $25 \times 19 \times 9 = 4275$ candidate। Top-N (~300-1000) keep।

Custom dataset: Faster R-CNN-এ COCO 80-class থেকে 5-class (Bangla traffic) shift — কোডটি লিখুন।

from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
m = fasterrcnn_resnet50_fpn(weights='DEFAULT')
in_features = m.roi_heads.box_predictor.cls_score.in_features
m.roi_heads.box_predictor = FastRCNNPredictor(in_features, 5+1)  # +1 background

ভাবুন: Faster R-CNN-এ feature share-এর সবচেয়ে বড় benefit কোথায় — single image-এ inference, না training-এ?

Both। Inference — 2000 region per-region forward এড়ায় (200x speedup)। Training — gradient share করে backbone joint train। R-CNN multi-stage train ছিল।

এই পাঠে যা শিখবেন

১ · R-CNN (২০১৪) — beginning

২ · Fast R-CNN (২০১৫)

৩ · RoI Pooling

৪ · Faster R-CNN (২০১৫)

৫ · Region Proposal Network (RPN)

৬ · Anchor box concept

৭ · Mask R-CNN (২০১৭)

৮ · PyTorch-এ Faster R-CNN

৯ · Two-stage variants

১০ · Two-stage modern uses

ভাবনার প্রশ্ন

অনুশীলন

আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ