পাঠ ১৮ · ৩৫-এর মধ্যে · মডিউল ৩

Home / AI Courses / Computer Vision / R-CNN family

R-CNN ও Faster R-CNN

The R-CNN family — region-based detectors evolution

৮ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

R-CNN evolution — slow → fast → faster
RoI pooling ও RoI Align
Region Proposal Network (RPN)
PyTorch-এ Faster R-CNN inference

১ · R-CNN (২০১৪) — beginning

Ross Girshick et al. — "Rich feature hierarchies for accurate object detection". AlexNet-এর উপর ভিত্তি করে CNN-এর প্রথম successful detection.

Pipeline:

Selective Search: classical algorithm — ~2000 region proposal generate.
Warp: each region 227×227-এ resize.
CNN feature: each warped region আলাদাভাবে AlexNet-এ feed.
SVM: per-class linear SVM classify.
Bbox regression: linear regressor refine box.

সমস্যা: 2000 region × CNN forward = অসম্ভব slow. Train-এ days, inference 47 sec/image.

কেন্দ্রীয় ধারণা

R-CNN proof — CNN feature SIFT/HOG-এর চেয়ে ভাল detection-এ। কিন্তু compute prohibitive. পরের iteration এই sharing solve করে।

২ · Fast R-CNN (২০১৫)

Girshick alone — "Fast R-CNN". Key insight: পুরো image-কে একবার CNN-এ পাঠাও, তারপর region propose feature map-এ।

Pipeline:

Image → CNN backbone → feature map (e.g., 14×14×512).
Selective Search → ~2000 proposal (image space).
Each proposal → corresponding region in feature map.
RoI Pooling: each region-কে fixed size 7×7-এ pool.
FC layers → classification + bbox regression.

Speedup: CNN forward 1 time per image (not 2000 per region). 25x train speed, 213x inference speed (still ~2 sec).

৩ · RoI Pooling

Variable-sized region → fixed-sized output. Algorithm:

Region in feature map (e.g., 30×40).
Divide into 7×7 grid (target size).
Each grid cell — max pool corresponding feature map cells.
Output 7×7 → flatten → FC.

Issue: integer coordinates → quantization error → misalignment.

৪ · Faster R-CNN (২০১৫)

Ren et al. — "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks".

Key innovation: Selective Search-এর বদলে neural Region Proposal Network (RPN). Fully neural pipeline.

Pipeline:

Image → CNN backbone → feature map.
RPN: per-pixel anchor (multi-scale, multi-ratio) → objectness + box refine.
Top-N proposal (~300).
RoI Pooling → FC heads: class + bbox.

Speed: ~5 FPS (VGG backbone). ~17 FPS (ResNet). Realtime threshold-এ approach.

৫ · Region Proposal Network (RPN)

3×3 conv on feature map → intermediate feature.
Two parallel 1×1 conv head:
- Cls: per anchor — object/background score.
- Reg: per anchor — bbox refinement (dx, dy, dw, dh).
Anchor: 9 per pixel (3 scale × 3 aspect ratio).
Trained jointly with main detector.

৬ · Anchor box concept

Pre-defined boxes at each spatial location.
Standard: 3 scales (128, 256, 512 px) × 3 aspect ratios (1:1, 1:2, 2:1) = 9.
Each anchor predict offset from base — easier than absolute box.
Match with ground-truth via IoU > 0.7 (positive), <0.3 (negative).

R-CNN → Fast → Faster R-CNN — engineering iteration-এর textbook example. Each iteration removed bottleneck: shared compute, learnable proposal, end-to-end training.

R-CNN family — engineering iteration-এর model. প্রতিটি step একটি bottleneck remove.

৭ · Mask R-CNN (২০১৭)

Faster R-CNN-এর extension — instance segmentation. He et al. ICCV 2017 best paper.

RoI Pooling → RoI Align: bilinear interpolation, quantization free.
Mask head: RoI feature → small FCN → per-class binary mask.
Multi-task: classification + bbox + mask.
Mask R-CNN — instance segmentation-এর gold standard.

৮ · PyTorch-এ Faster R-CNN

Python · PyTorch

import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision.models.detection import FasterRCNN_ResNet50_FPN_Weights

w = FasterRCNN_ResNet50_FPN_Weights.COCO_V1
model = fasterrcnn_resnet50_fpn(weights=w)
model.eval()

# Inference
img = torch.rand(3, 800, 800)   # any size, FPN handles
with torch.no_grad():
    out = model([img])

# Output: list of dict per image
det = out[0]
print("Boxes:", det['boxes'].shape)     # (N, 4) — xyxy
print("Labels:", det['labels'].shape)   # (N,) — class index
print("Scores:", det['scores'].shape)   # (N,) — confidence

# Filter confidence > 0.5
keep = det['scores'] > 0.5
print(f"\n{keep.sum()} confident detections")

torchvision Faster R-CNN — COCO 80 class pretrained. Custom dataset finetune-এ — last classifier replace.

৯ · Two-stage variants

Cascade R-CNN (২০১৮): multi-stage classifier with increasing IoU threshold.
Mask R-CNN: + segmentation.
Sparse R-CNN: learnable proposal — DETR-like end-to-end.
HTC (Hybrid Task Cascade): multiple task interleave.

১০ · Two-stage modern uses

Maximum accuracy benchmarks.
Medical imaging — high precision matters.
Custom annotation type (keypoint + mask + bbox).
Mask R-CNN — instance segmentation production.

Two-stage detector slow but flexible. YOLO-র simplicity-র জন্য realtime মুহূর্তে — কিন্তু high accuracy + custom heads-এ Faster R-CNN family champion.

ভাবনার প্রশ্ন

প্র ০১ RoI Pooling ও RoI Align-এর পার্থক্য কী? Mask R-CNN-এ pooling থেকে align-এ switch কেন?

এটি subtle কিন্তু segmentation accuracy-তে বিরাট impact.

RoI Pooling-এর problem:

Region coordinate continuous (e.g., 18.6).
Feature map integer index.
Step 1 quantization: round to nearest integer (18.6 → 19).
Step 2 quantization: bin division integer.
Total — sub-pixel shift accumulate.

Effect:

Classification — minor (pool over region).
Bbox refine — minor.
Mask prediction — major! Pixel-level accuracy needed.

RoI Align solution:

No integer rounding.
Bin boundary continuous.
Bilinear interpolation — sample at exact continuous coordinate.
4 sample point per bin → average.

Mathematical:

RoI Pooling: $\text{out}[i,j] = \max_{x,y \in \text{quantized bin}} F[\lfloor x \rfloor, \lfloor y \rfloor]$।
RoI Align: $\text{out}[i,j] = \frac{1}{4} \sum \text{bilinear}(F, x_k, y_k)$।

Mask R-CNN paper-এর result:

RoI Pool: AP_mask 24.5.
RoI Align: AP_mask 30.3.
+5.8% — substantial.

Computational cost:

RoI Align — slightly more (4 sample per bin, bilinear).
Trade-off worth it.

Used elsewhere:

Mask R-CNN, Cascade Mask R-CNN.
Detection-এ Faster R-CNN-ও modern implementation-এ।
Anywhere precise feature extraction — keypoint detection.

মূল উপলব্ধি: "Implementation detail" architecture decision-এর সমান impact. Sub-pixel accuracy — segmentation, fine-grained detection-এর key.

প্র ০২ RPN-এ anchor box-এর choice (3 scale × 3 ratio) — domain-specific tune করা দরকার? Bangla street sign detect-এ কী anchor?

Anchor design — Faster R-CNN-এর underrated tunable. Default ImageNet-COCO-এর জন্য optimal, custom-এর জন্য না।

Default anchor:

Scales: 128, 256, 512 px.
Ratios: 1:1, 1:2, 2:1.
9 anchor per spatial location.
FPN-এ — scale per pyramid level.

Why this matters:

Anchor — "guess" — closest match-এ gradient flow.
Bad anchor → slow convergence, missed object.
Right anchor → fast train, high recall.

Bangla street sign analysis:

Sign typically square (1:1).
Street view — usually 30-150 px (camera dist).
Vertical signs (parking restriction) — 1:3.
Horizontal banner — 3:1.

Custom anchor:

Scales: 32, 64, 128 (smaller — sign typically distant).
Ratios: 1:1, 1:3, 3:1 (vertical/horizontal banner).
9 anchor still.

K-means anchor (YOLOv2-style):

Training set bbox-এ k-means clustering.
Cluster center = optimal anchor.
Domain-specific automatic.

YOLO modern (v3-v8):

Auto-anchor tool — dataset analyze, suggest anchor.
Training-time auto-tune.

Anchor-free alternatives:

FCOS, CenterNet — no anchor concept.
DETR — set prediction.
Modern trend — anchor-free easier.

Practical recommendation:

Stage 1: COCO default — quick baseline.
Stage 2: dataset analyze — k-means anchor.
Stage 3: anchor-free try (FCOS, RT-DETR).

মূল উপলব্ধি: Anchor — historical hack, but tunable. Domain-specific design — accuracy bump. Modern anchor-free — simpler design-এর direction.

প্র ০৩ Faster R-CNN-এ FPN (Feature Pyramid Network) integration. Multi-scale feature কেন important? ছোট object detect-এর কী secret?

FPN (Lin et al., 2017) — detection-এর underrated breakthrough. Small object detection-এ revolution.

Problem before FPN:

Faster R-CNN — final feature map (high-level, 32× downsample).
Small object — final map-এ খুবই ছোট spatial extent.
Large object — captured ভাল।
Small object detection mAP < 25%.

Single-scale problem:

Image pyramid (multi-scale input) — slow.
Feature pyramid (different layer) — early shallow features semantic poor.

FPN insight:

Use different stage feature map (P2, P3, P4, P5).
Top-down pathway — high-level semantic propagate down.
Lateral connection — combine high & low-level.
Result: each level — semantically rich + spatially precise.

FPN structure:

P5 = lateral(C5)
P4 = lateral(C4) + upsample(P5)
P3 = lateral(C3) + upsample(P4)
P2 = lateral(C2) + upsample(P3)

Detection per level:

Small object → P2 (high resolution).
Medium → P3, P4.
Large → P5.
Anchor scale per level.

Result on COCO:

Faster R-CNN: 35.0 mAP.
Faster R-CNN + FPN: 39.4 mAP.
Small object AP: 17 → 24 (+41%).

FPN modern adoption:

Almost all detection backbone.
YOLO v3+ uses FPN-like (PANet).
RetinaNet — FPN baseline.
Mask R-CNN — FPN essential.

Variants:

PANet: bottom-up addition over FPN.
BiFPN (EfficientDet): bidirectional.
NAS-FPN: architecture searched.
Recursive FPN: multi-pass.

Bangladesh use case:

Crowded street — pedestrian distance variable.
Far pedestrian small in pixel — FPN essential.
Drone aerial photo — small object dominant.
Default detector + FPN strongly recommended.

মূল উপলব্ধি: Single-scale CNN insufficient real-world detection. FPN — neck design-এর foundational pattern. Backbone alone যথেষ্ট না।

প্র ০৪ Faster R-CNN ২০১৫-র। ১১ বছর পর — DETR, YOLOv8 আছে। Faster R-CNN আজও কোথায় কেন ব্যবহার?

Honest assessment — niche কিন্তু important.

Where Faster R-CNN still wins:

Custom heads: Mask R-CNN, keypoint, dense pose.
Small object accuracy: RoI alignment fine-grained.
Custom annotation: non-standard task — adapt easy.
Research baselines: well-understood, tunable.
Production legacy: 2017-2020 systems still running.

Where YOLO/DETR wins:

Realtime: mobile, embedded.
Single-task standard detection: no need for heads.
Modern accuracy: DINO-DETR > Faster R-CNN.
Simplicity: single loss, easier debug.

Specific scenarios:

Mask R-CNN — instance segmentation: still strong choice.
HTC, Cascade Mask R-CNN: highest mAP achievable.
Detectron2 (Meta library): Faster R-CNN central.
Medical (FDA-validated): regulatory acceptance.

Modern alternatives for two-stage role:

Sparse R-CNN: two-stage with learnable proposal.
Cascade DINO: DETR-based cascade.
Mask2Former: universal segmentation.

Bangladesh deployment:

Realtime traffic app: YOLOv8.
Medical research: Mask R-CNN.
Document analysis: Faster R-CNN custom head.
Industrial inspection: Faster R-CNN — accuracy critical.

Skill recommendation:

Master Faster R-CNN concept — foundational.
Use Mask R-CNN — segmentation-এ default.
Try DETR/RT-DETR — future direction.
YOLO — realtime mainstream.

মূল উপলব্ধি: Faster R-CNN — like email (1971) or HTTP (1991) — old, still essential foundation. Modern alternatives flashier, but core concept everywhere.

অনুশীলন

RPN output: 800×600 image, ResNet backbone (32x downsample), 9 anchor — RPN total proposal candidate?

Feature map: $25 \times 19$। Per location 9 anchor → $25 \times 19 \times 9 = 4275$ candidate. Top-N (~300-1000) keep.

Custom dataset: Faster R-CNN-এ COCO 80-class থেকে 5-class (Bangla traffic) shift — কোডটি লিখুন।

from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
m = fasterrcnn_resnet50_fpn(weights='DEFAULT')
in_features = m.roi_heads.box_predictor.cls_score.in_features
m.roi_heads.box_predictor = FastRCNNPredictor(in_features, 5+1)  # +1 background

ভাবুন: Faster R-CNN-এ feature share-এর সবচেয়ে বড় benefit কোথায় — single image-এ inference, না training-এ?

Both. Inference — 2000 region per-region forward এড়ায় (200x speedup). Training — gradient share করে backbone joint train. R-CNN multi-stage train ছিল।

এই পাঠে যা শিখবেন

১ · R-CNN (২০১৪) — beginning

২ · Fast R-CNN (২০১৫)

৩ · RoI Pooling

৪ · Faster R-CNN (২০১৫)

৫ · Region Proposal Network (RPN)

৬ · Anchor box concept

৭ · Mask R-CNN (২০১৭)

৮ · PyTorch-এ Faster R-CNN

৯ · Two-stage variants

১০ · Two-stage modern uses

ভাবনার প্রশ্ন

অনুশীলন

আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ