পাঠ ১৩ · ৩৫-এর মধ্যে · মডিউল ২
Home / AI Courses / Computer Vision / Inception

Inception ও GoogLeNet

Inception modules and GoogLeNet — parallel branches in CNN
৭ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

  • Inception module-এর design intuition
  • 1×1 bottleneck-এর role
  • GoogLeNet vs VGG comparison
  • Inception v2, v3, v4 evolution

১ · "We Need to Go Deeper"

GoogLeNet-এর famous Inception meme reference (Christopher Nolan-এর film). ২২ layer — তখনের জন্য deepest practical CNN.

Design challenge:

  • Object scale variability — কখনো ছবিতে cat ছোট, কখনো বড়।
  • Single kernel size একই layer-এ — limit.
  • Solution: একসাথে multiple kernel size — let network choose.
কেন্দ্রীয় ধারণা

Sequential network "depth" — VGG-এর approach. Inception "width" — এক layer-এ multiple parallel transformation. দু'টি একসাথে — best practice.

২ · Naive Inception module

Original (অপ্টিমাইজেশনের আগে):

  • Branch 1: 1×1 conv.
  • Branch 2: 3×3 conv.
  • Branch 3: 5×5 conv.
  • Branch 4: 3×3 max pool.
  • সব output channel-wise concat.

সমস্যা: input 256 channel হলে — 5×5 conv-এ 256 → 128 channel = $5 \times 5 \times 256 \times 128 = 819{,}200$ params. Compute explode.

৩ · Inception v1 (with bottleneck)

Smart fix — costly conv-এর আগে 1×1 conv দিয়ে channel reduce.

  • Branch 1: 1×1 conv → 64 ch.
  • Branch 2: 1×1 → 96 ch → 3×3 conv → 128 ch.
  • Branch 3: 1×1 → 16 ch → 5×5 conv → 32 ch.
  • Branch 4: 3×3 max pool → 1×1 → 32 ch.
  • Concat: 64 + 128 + 32 + 32 = 256 ch.

Total compute drops 10x compared to naive. 1×1 = "pointwise mixing" — channel space-এ projection.

৪ · GoogLeNet architecture

  • Input → Conv 7×7 stem → MaxPool.
  • Conv 1×1 + Conv 3×3 → MaxPool.
  • Inception 3a, 3b → MaxPool.
  • Inception 4a-4e → MaxPool.
  • Inception 5a, 5b.
  • Global avg pool → 1000-D FC → Softmax.

Total: ২২ deep layer (counting each layer in Inception module: ~৭০ conv layers).

Parameters: ~৫M — VGG-এর 138M-এর 27x কম। কিন্তু similar accuracy.

৫ · Auxiliary classifier

২২ layer-এ — vanishing gradient. Solution: middle layer-এ "auxiliary head" — Inception 4a ও 4d-এর পরে FC + softmax.

  • Training-এ — additional gradient signal middle থেকে।
  • Total loss: $\mathcal{L} = \mathcal{L}_{\text{main}} + 0.3 \mathcal{L}_{\text{aux1}} + 0.3 \mathcal{L}_{\text{aux2}}$।
  • Inference-এ — শুধু main classifier.

পরে BatchNorm আবিষ্কারের পর auxiliary classifier-এর প্রয়োজন কম। Inception v3-এ optional.

VGG = "Lego tower — same block, stack high". Inception = "Swiss Army knife block — multiple tools একসাথে"। Both depth, but Inception adds width per stage.
Inception module — চারটি parallel branch Input feature 1×1 conv 64 ch 1×1 → 3×3 96 → 128 ch 1×1 → 5×5 16 → 32 ch 3×3 pool → 1×1 → 32 ch Concatenate channels 256 ch total Different kernel sizes একসাথে — multi-scale feature 1×1 bottleneck — compute কমায়, accuracy বজায়
Inception module — 1×1, 3×3, 5×5 conv + pool parallel-এ চলে। 1×1 bottleneck দিয়ে compute সাশ্রয়।

৬ · PyTorch-এ একটি Inception block

Python · PyTorch 
import torch
import torch.nn as nn

class InceptionBlock(nn.Module):
    def __init__(self, in_ch, ch1, ch3_red, ch3, ch5_red, ch5, pool_proj):
        super().__init__()
        # Branch 1: 1x1
        self.b1 = nn.Conv2d(in_ch, ch1, 1)
        # Branch 2: 1x1 → 3x3
        self.b2 = nn.Sequential(
            nn.Conv2d(in_ch, ch3_red, 1), nn.ReLU(),
            nn.Conv2d(ch3_red, ch3, 3, padding=1)
        )
        # Branch 3: 1x1 → 5x5
        self.b3 = nn.Sequential(
            nn.Conv2d(in_ch, ch5_red, 1), nn.ReLU(),
            nn.Conv2d(ch5_red, ch5, 5, padding=2)
        )
        # Branch 4: 3x3 pool → 1x1
        self.b4 = nn.Sequential(
            nn.MaxPool2d(3, stride=1, padding=1),
            nn.Conv2d(in_ch, pool_proj, 1)
        )

    def forward(self, x):
        return torch.cat([self.b1(x), self.b2(x), self.b3(x), self.b4(x)], dim=1)

# Inception 3a: in=192, out = 64+128+32+32 = 256
block = InceptionBlock(192, 64, 96, 128, 16, 32, 32)
x = torch.randn(1, 192, 28, 28)
y = block(x)
print("Output:", y.shape)               # (1, 256, 28, 28)

# Pretrained
from torchvision.models import googlenet
m = googlenet(weights='DEFAULT')
n = sum(p.numel() for p in m.parameters())
print(f"GoogLeNet params: {n:,}")        # ~6.6M

    


    

      Inception block four branches concat-এ output channel = sum. GoogLeNet ৫M+ parameter-এ VGG-র চেয়ে সমান বা ভালো accuracy.
    


    
৭ · Inception v2, v3, v4 evolution

    
    
      
  • Inception v2 (২০১৫): BatchNorm যুক্ত। 5×5 conv → দু'টি 3×3 (factorization).
    • 
      
  • Inception v3 (২০১৫): 7×7 → 1×7 + 7×1 (asymmetric factor). RMSProp. Label smoothing.
    • 
      
  • Inception v4 / Inception-ResNet (২০১৬): ResNet-এর skip connection মিশ্রিত।
    • 
      
  • Xception (২০১৬, Chollet): "Extreme Inception" — depthwise separable conv.
    • 
    


    
৮ · Inception-এর শিক্ষা

    
    
      
  • Multi-scale processing: single kernel size insufficient.
    • 
      
  • 1×1 conv-এর power: bottleneck, channel mix, dimension control.
    • 
      
  • Auxiliary supervision: deep network train-এর কৌশল।
    • 
      
  • Width matters: "going wider" = "going deeper".
    • 
    


    
৯ · GoogLeNet-এর legacy

    
    
      
  • Inception v3 — TensorFlow-এর "Hello world".
    • 
      
  • Pretrained Inception → transfer learning easy.
    • 
      
  • FID metric (image generation evaluation) — Inception-V3 feature.
    • 
      
  • NASNet, EfficientNet — Inception-inspired blocks search.
    • 
    


    

      Modern era-এ pure Inception rare. কিন্তু "multi-branch + 1×1 bottleneck" pattern — ResNeXt, EfficientNet, ConvNeXt-এ অনুরূপ। Concept eternal.
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      
        প্র ০১
        Naive Inception ও bottleneck Inception-এর computation difference একটি specific example-এ — Inception 3a-এর জন্য?
      
      

        

          এটি 1×1 conv-এর power দেখাবার সবচেয়ে concrete way.
        

        

          Inception 3a parameters:
        

        
    
          
  • Input: 192 ch (28×28).
    • 
          
  • Output: 64+128+32+32 = 256 ch.
    • 
        

        

          Naive Inception (without 1×1 bottleneck):
        

        
    
          
  • Branch 1 (1×1, 64): $192 \times 64 = 12.3K$।
    • 
          
  • Branch 2 (3×3, 128): $192 \times 9 \times 128 = 221K$।
    • 
          
  • Branch 3 (5×5, 32): $192 \times 25 \times 32 = 154K$।
    • 
          
  • Branch 4 (pool + 1×1, 32): $192 \times 32 = 6.1K$।
    • 
          
  • Total: ~393K params.
    • 
        

        

          With 1×1 bottleneck:
        

        
    
          
  • Branch 1 (1×1, 64): 12.3K.
    • 
          
  • Branch 2 (1×1→96, 3×3→128): $192 \times 96 + 96 \times 9 \times 128 = 18.4K + 110.6K = 129K$।
    • 
          
  • Branch 3 (1×1→16, 5×5→32): $192 \times 16 + 16 \times 25 \times 32 = 3K + 12.8K = 15.8K$।
    • 
          
  • Branch 4 (pool + 1×1, 32): 6.1K.
    • 
          
  • Total: ~163K params.
    • 
        

        

          Saving: 393K → 163K, 58% reduction.
        

        

          FLOPs (per spatial location):
        

        
    
          
  • Naive 3×3: $192 \times 9 \times 128 = 221K$ MAC.
    • 
          
  • Bottleneck 3×3: $192 + 96 \times 9 \times 128 = 18.4K + 110.6K$ MAC.
    • 
          
  • 2x reduction.
    • 
        

        

          Accuracy:
        

        
    
          
  • Bottleneck — empirically equivalent বা slightly better.
    • 
          
  • Reason: 1×1 conv channel mixing — beneficial nonlinearity.
    • 
          
  • Information bottleneck — implicit regularization.
    • 
        

        

          Lesson — applicable everywhere:
        

        
    
          
  • ResNet-50 bottleneck block: 1×1 → 3×3 → 1×1.
    • 
          
  • MobileNet inverted bottleneck.
    • 
          
  • Transformer FFN: linear-up → activation → linear-down.
    • 
          
  • Universal pattern: expand → process → compress.
    • 
        

        

          মূল উপলব্ধি: Smart architecture > brute force. 1×1 conv "trivial" মনে হলেও — modern CNN-এর backbone. Compute aware design pre-DL also matters.
        

      

    


    

      
        প্র ০২
        GoogLeNet-এ auxiliary classifier — middle layer থেকে loss. কেন helpful, এবং কেন BatchNorm আসার পর deprecated?
      
      

        

          Auxiliary classifier — vanishing gradient problem-এর pre-BatchNorm solution.
        

        

          Vanishing gradient revisited:
        

        
    
          
  • ২২ layer GoogLeNet — backprop chain rule.
    • 
          
  • Last layer-এর loss → first layer-এ gradient তেমন কিছু না।
    • 
          
  • Early layer training stagnant.
    • 
        

        

          Auxiliary classifier solution:
        

        
    
          
  • Inception 4a ও 4d-এর পরে — small head (FC + softmax).
    • 
          
  • Independent loss compute — local gradient signal.
    • 
          
  • Combined loss: $\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{main}} + 0.3(\mathcal{L}_{\text{aux1}} + \mathcal{L}_{\text{aux2}})$।
    • 
          
  • Middle layer directly supervised.
    • 
        

        

          Why this works:
        

        
    
          
  • Shorter gradient path — middle layer-এ stable signal.
    • 
          
  • Implicit regularization — middle features must be "classify-able".
    • 
          
  • Ensemble effect — three classifiers train.
    • 
        

        

          BatchNorm replaces:
        

        
    
          
  • Ioffe & Szegedy (২০১৫) — BN paper.
    • 
          
  • Per-layer normalization — gradient stable through depth.
    • 
          
  • Deeper networks (Inception v3, ResNet 50+) — direct end-to-end train possible.
    • 
          
  • Auxiliary classifier-এর role redundant.
    • 
        

        

          Inception v3-এ:
        

        
    
          
  • One auxiliary classifier (vs two in v1).
    • 
          
  • Mainly regularizer — minor accuracy gain.
    • 
          
  • BN with auxiliary — no significant boost.
    • 
        

        

          Modern architecture:
        

        
    
          
  • ResNet — skip connection (architectural).
    • 
          
  • BatchNorm/LayerNorm — normalization layer.
    • 
          
  • DenseNet — every layer to every layer connectivity.
    • 
          
  • All — vanishing gradient address.
    • 
        

        

          Modern revival — deep supervision:
        

        
    
          
  • U-Net++, DeepLab — multi-level supervision.
    • 
          
  • Knowledge distillation — teacher's intermediate feature.
    • 
          
  • Auxiliary task head (multi-task learning).
    • 
          
  • Concept lives, just under different names.
    • 
        

        

          মূল উপলব্ধি: Engineering trick — temporary fix till fundamental solution. GoogLeNet-এর auxiliary classifier, BN-এর জন্ম-এর পথ সাফ। Engineering pragmatism + theoretical advance.
        

      

    


    

      
        প্র ০৩
        Inception v3-এ 5×5 → দু'টি 3×3 (factorization), 7×7 → 1×7 + 7×1 (asymmetric). দু'টি কেন আলাদা treatment?
      
      

        

          এটি Szegedy et al.-এর factorization paper-এর key insight. Convolution decomposition theory.
        

        

          Symmetric factorization (5×5 → 3×3 + 3×3):
        

        
    
          
  • একটি 5×5 RF = দু'টি stacked 3×3 RF.
    • 
          
  • Parameter: $25c^2 \to 18c^2$ — 28% reduction.
    • 
          
  • Two non-linearities (extra ReLU).
    • 
          
  • Standard 2D filter — maintain isotropy.
    • 
        

        

          Asymmetric factorization (7×7 → 1×7 + 7×1):
        

        
    
          
  • একটি 7×7 = একটি 1×7 (horizontal) + একটি 7×1 (vertical).
    • 
          
  • Parameter: $49c^2 \to 14c^2$ — 71% reduction!
    • 
          
  • Massive saving for large kernel.
    • 
          
  • কিন্তু — strict 2D pattern lose. Only "separable" filter approximate.
    • 
        

        

          Why asymmetric only for big kernels?
        

        
    
          
  • 3×3 → 1×3 + 3×1 — saving মাত্র 33%. 3×3-ই compute-friendly.
    • 
          
  • 5×5 — borderline.
    • 
          
  • 7×7+ — saving worth approximation cost.
    • 
        

        

          Information theory perspective:
        

        
    
          
  • 2D pattern = full 49-D space (7×7).
    • 
          
  • Separable = rank-1 approximation.
    • 
          
  • Most natural image kernel rank-1-এর কাছাকাছি — Sobel separable.
    • 
          
  • Approximation cheap, useful.
    • 
        

        

          Empirical results (Inception v3):
        

        
    
          
  • 5×5 → 3×3+3×3: equal accuracy, faster.
    • 
          
  • 7×7 → 1×7+7×1: slightly worse, much faster.
    • 
          
  • 3×3 → 1×3+3×1: noticeable accuracy drop. Avoided.
    • 
        

        

          Modern architectures:
        

        
    
          
  • MobileNet: depthwise separable — extreme factorization (per-channel + 1×1).
    • 
          
  • Xception: "depthwise + pointwise" — 7×7-এর extreme version.
    • 
          
  • ConvMixer: 7×7 depthwise + 1×1.
    • 
          
  • Vision Transformer: 16×16 patch embedding — extreme factorization (no spatial conv).
    • 
        

        

          Hardware impact:
        

        
    
          
  • Asymmetric conv — special CUDA kernel.
    • 
          
  • NVIDIA cuDNN optimize 3×3 most.
    • 
          
  • 1×7 conv — slower than 7×1 in some implementations (memory layout).
    • 
        

        

          মূল উপলব্ধি: Factorization — math-driven optimization. 2D → 1D approximation modern efficient architecture-এর backbone.
        

      

    


    

      
        প্র ০৪
        FID (Fréchet Inception Distance) — image generation quality metric. Inception V3 feature কেন এত universal? কী problem এই metric solve করে?
      
      

        

          FID — image generation evaluation-এর de-facto standard. Heusel et al. (২০১৭)।
        

        

          Problem এর before:
        

        
    
          
  • GAN আউটপুট কীভাবে evaluate?
    • 
          
  • Inception Score (২০১৬, Salimans et al.) — Inception V3-এর softmax distribution diversity.
    • 
          
  • সমস্যা: real image distribution-এর সাথে compare করে না।
    • 
        

        

          FID definition:
        

        

          $$\text{FID} = \| \mu_r - \mu_g \|^2 + \text{Tr}(\Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2})$$
        

        
    
          
  • $\mu, \Sigma$ — Inception V3-এর pool3 feature-এর mean & covariance.
    • 
          
  • $r$ = real images, $g$ = generated images.
    • 
          
  • Lower FID = generated distribution real-এর কাছাকাছি।
    • 
        

        

          Why Inception V3 specifically?
        

        
    
          
  • Pretrained on ImageNet: diverse natural image features.
    • 
          
  • Feature dimensionality: 2048-D pool3 — rich representation.
    • 
          
  • Standardization: all researchers same network = comparable scores.
    • 
          
  • Historic accident: ২০১৭-এ V3 SOTA. যা set হয়ে গেল।
    • 
        

        

          FID-এর strengths:
        

        
    
          
  • Real vs generated distribution direct compare.
    • 
          
  • Mode collapse detect (low diversity → high FID).
    • 
          
  • Visual quality + diversity দু'টোই capture.
    • 
          
  • Human perception-এর সাথে correlation ভাল।
    • 
        

        

          FID-এর limitations:
        

        
    
          
  • ImageNet-bias — non-natural image (medical, satellite) inappropriate.
    • 
          
  • Inception V3 feature — perfect representation না।
    • 
          
  • Sample size sensitive (50K typical).
    • 
          
  • Outliers can game.
    • 
          
  • Resolution-dependent.
    • 
        

        

          Modern alternatives:
        

        
    
          
  • CLIP-FID: CLIP feature use — multimodal aware.
    • 
          
  • KID (Kernel Inception Distance): unbiased estimator.
    • 
          
  • Precision/Recall: fidelity vs diversity decompose.
    • 
          
  • Improved Precision/Recall: nearest-neighbor based.
    • 
        

        

          Bangladeshi context:
        

        
    
          
  • Bangla art generation (paintings, calligraphy).
    • 
          
  • FID-এর Inception V3 — ImageNet bias problem.
    • 
          
  • Domain-specific feature extractor → better metric.
    • 
        

        

          Stable Diffusion era:
        

        
    
          
  • FID still main metric, but criticized.
    • 
          
  • Human evaluation — gold standard.
    • 
          
  • CLIP score (text-image alignment) for text-to-image.
    • 
        

        

          মূল উপলব্ধি: Metric design ML-এর সবচেয়ে underrated skill. Inception V3 — accidental backbone of generation evaluation. Standard সর্বদা challenge করুন।
        

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Inception 3a output channels: 64 + 128 + 32 + 32 কেন এই specific number? Sum = 256.
        
    
          
          
    
            
    Hand-tuned by GoogLeNet authors empirically. 1×1 ও pool branch কম, 3×3 dominant — natural feature 3×3-এ best capture. Modern NAS (Neural Architecture Search) এই split auto-discover.
    
          
    
        
    
      
    2. 

      
  2. 
        Pretrained Inception: torchvision থেকে Inception V3 load করে inference example লিখুন। Note: 299×299 input!
        
    
          
          
    
            
    from torchvision.models import inception_v3, Inception_V3_Weights
w = Inception_V3_Weights.IMAGENET1K_V1
m = inception_v3(weights=w).eval()
preprocess = w.transforms()  # 299×299, normalize
x = preprocess(img).unsqueeze(0)
out = m(x)
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Inception module-এ pool branch — কেন? Pure conv-only Inception কী missing?
        
    
          
          
    
            
    Pool branch = "no learning" baseline — local invariance. Different operation type ensemble effect. Modern ResNet skip connection-এ similar role — identity baseline.
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    

    

      কোড রানার কাজ না করলে? ব্রাউজারে কাজ না করলে
      Google Colab
      ব্যবহার করুন — Google-এর ফ্রি অনলাইন Python পরিবেশ, শুধু Gmail অ্যাকাউন্ট লাগে।
    

  







  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১২ · VGG