পাঠ ১৩ · ৩৫-এর মধ্যে · মডিউল ২
Home / AI Courses / Computer Vision / Inception

Inception ও GoogLeNet

Inception modules and GoogLeNet — parallel branches in CNN
৭ মিনিট পড়া মাঝারি · Intermediate PyTorch কোডসহ

এই পাঠে যা শিখবেন

  • Inception module-এর design intuition
  • 1×1 bottleneck-এর role
  • GoogLeNet vs VGG comparison
  • Inception v2, v3, v4 evolution

১ · "We Need to Go Deeper"

GoogLeNet-এর famous Inception meme reference (Christopher Nolan-এর film)। ২২ layer — তখনের জন্য deepest practical CNN।

Design challenge:

  • Object scale variability — কখনো ছবিতে cat ছোট, কখনো বড়।
  • Single kernel size একই layer-এ — limit।
  • Solution: একসাথে multiple kernel size — let network choose।
কেন্দ্রীয় ধারণা

Sequential network "depth" — VGG-এর approach। Inception "width" — এক layer-এ multiple parallel transformation। দু'টি একসাথে — best practice।

২ · Naive Inception module

Original (অপ্টিমাইজেশনের আগে):

  • Branch 1: 1×1 conv।
  • Branch 2: 3×3 conv।
  • Branch 3: 5×5 conv।
  • Branch 4: 3×3 max pool।
  • সব output channel-wise concat।

সমস্যা: input 256 channel হলে — 5×5 conv-এ 256 → 128 channel = $5 \times 5 \times 256 \times 128 = 819{,}200$ params। Compute explode।

৩ · Inception v1 (with bottleneck)

Smart fix — costly conv-এর আগে 1×1 conv দিয়ে channel reduce।

  • Branch 1: 1×1 conv → 64 ch।
  • Branch 2: 1×1 → 96 ch → 3×3 conv → 128 ch।
  • Branch 3: 1×1 → 16 ch → 5×5 conv → 32 ch।
  • Branch 4: 3×3 max pool → 1×1 → 32 ch।
  • Concat: 64 + 128 + 32 + 32 = 256 ch।

Total compute drops 10x compared to naive। 1×1 = "pointwise mixing" — channel space-এ projection।

৪ · GoogLeNet architecture

  • Input → Conv 7×7 stem → MaxPool।
  • Conv 1×1 + Conv 3×3 → MaxPool।
  • Inception 3a, 3b → MaxPool।
  • Inception 4a-4e → MaxPool।
  • Inception 5a, 5b।
  • Global avg pool → 1000-D FC → Softmax।

Total: ২২ deep layer (counting each layer in Inception module: ~৭০ conv layers).

Parameters: ~৫M — VGG-এর 138M-এর 27x কম। কিন্তু similar accuracy।

৫ · Auxiliary classifier

২২ layer-এ — vanishing gradient। Solution: middle layer-এ "auxiliary head" — Inception 4a ও 4d-এর পরে FC + softmax।

  • Training-এ — additional gradient signal middle থেকে।
  • Total loss: $\mathcal{L} = \mathcal{L}_{\text{main}} + 0.3 \mathcal{L}_{\text{aux1}} + 0.3 \mathcal{L}_{\text{aux2}}$।
  • Inference-এ — শুধু main classifier।

পরে BatchNorm আবিষ্কারের পর auxiliary classifier-এর প্রয়োজন কম। Inception v3-এ optional।

VGG = "Lego tower — same block, stack high"। Inception = "Swiss Army knife block — multiple tools একসাথে"। Both depth, but Inception adds width per stage।
Inception module — চারটি parallel branch Input feature 1×1 conv 64 ch 1×1 → 3×3 96 → 128 ch 1×1 → 5×5 16 → 32 ch 3×3 pool → 1×1 → 32 ch Concatenate channels 256 ch total Different kernel sizes একসাথে — multi-scale feature 1×1 bottleneck — compute কমায়, accuracy বজায়
Inception module — 1×1, 3×3, 5×5 conv + pool parallel-এ চলে। 1×1 bottleneck দিয়ে compute সাশ্রয়।

৬ · PyTorch-এ একটি Inception block

Python · PyTorch 
import torch
import torch.nn as nn

class InceptionBlock(nn.Module):
    def __init__(self, in_ch, ch1, ch3_red, ch3, ch5_red, ch5, pool_proj):
        super().__init__()
        # Branch 1: 1x1
        self.b1 = nn.Conv2d(in_ch, ch1, 1)
        # Branch 2: 1x1 → 3x3
        self.b2 = nn.Sequential(
            nn.Conv2d(in_ch, ch3_red, 1), nn.ReLU(),
            nn.Conv2d(ch3_red, ch3, 3, padding=1)
        )
        # Branch 3: 1x1 → 5x5
        self.b3 = nn.Sequential(
            nn.Conv2d(in_ch, ch5_red, 1), nn.ReLU(),
            nn.Conv2d(ch5_red, ch5, 5, padding=2)
        )
        # Branch 4: 3x3 pool → 1x1
        self.b4 = nn.Sequential(
            nn.MaxPool2d(3, stride=1, padding=1),
            nn.Conv2d(in_ch, pool_proj, 1)
        )

    def forward(self, x):
        return torch.cat([self.b1(x), self.b2(x), self.b3(x), self.b4(x)], dim=1)

# Inception 3a: in=192, out = 64+128+32+32 = 256
block = InceptionBlock(192, 64, 96, 128, 16, 32, 32)
x = torch.randn(1, 192, 28, 28)
y = block(x)
print("Output:", y.shape)               # (1, 256, 28, 28)

# Pretrained
from torchvision.models import googlenet
m = googlenet(weights='DEFAULT')
n = sum(p.numel() for p in m.parameters())
print(f"GoogLeNet params: {n:,}")        # ~6.6M

    


    

      Inception block four branches concat-এ output channel = sum। GoogLeNet ৫M+ parameter-এ VGG-র চেয়ে সমান বা ভালো accuracy।
    


    
৭ · Inception v2, v3, v4 evolution

    
    
      
  • Inception v2 (২০১৫): BatchNorm যুক্ত। 5×5 conv → দু'টি 3×3 (factorization)।
    • 
      
  • Inception v3 (২০১৫): 7×7 → 1×7 + 7×1 (asymmetric factor)। RMSProp। Label smoothing।
    • 
      
  • Inception v4 / Inception-ResNet (২০১৬): ResNet-এর skip connection মিশ্রিত।
    • 
      
  • Xception (২০১৬, Chollet): "Extreme Inception" — depthwise separable conv।
    • 
    


    
৮ · Inception-এর শিক্ষা

    
    
      
  • Multi-scale processing: single kernel size insufficient।
    • 
      
  • 1×1 conv-এর power: bottleneck, channel mix, dimension control।
    • 
      
  • Auxiliary supervision: deep network train-এর কৌশল।
    • 
      
  • Width matters: "going wider" = "going deeper"।
    • 
    


    
৯ · GoogLeNet-এর legacy

    
    
      
  • Inception v3 — TensorFlow-এর "Hello world"।
    • 
      
  • Pretrained Inception → transfer learning easy।
    • 
      
  • FID metric (image generation evaluation) — Inception-V3 feature।
    • 
      
  • NASNet, EfficientNet — Inception-inspired blocks search।
    • 
    


    

      Modern era-এ pure Inception rare। কিন্তু "multi-branch + 1×1 bottleneck" pattern — ResNeXt, EfficientNet, ConvNeXt-এ অনুরূপ। Concept eternal।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      
        প্র ০১
        Naive Inception ও bottleneck Inception-এর computation difference একটি specific example-এ — Inception 3a-এর জন্য?
      
      

        

          এটি 1×1 conv-এর power দেখাবার সবচেয়ে concrete way।
        

        

          Inception 3a parameters:
        

        
    
          
  • Input: 192 ch (28×28)।
    • 
          
  • Output: 64+128+32+32 = 256 ch।
    • 
        

        

          Naive Inception (without 1×1 bottleneck):
        

        
    
          
  • Branch 1 (1×1, 64): $192 \times 64 = 12.3K$।
    • 
          
  • Branch 2 (3×3, 128): $192 \times 9 \times 128 = 221K$।
    • 
          
  • Branch 3 (5×5, 32): $192 \times 25 \times 32 = 154K$।
    • 
          
  • Branch 4 (pool + 1×1, 32): $192 \times 32 = 6.1K$।
    • 
          
  • Total: ~393K params।
    • 
        

        

          With 1×1 bottleneck:
        

        
    
          
  • Branch 1 (1×1, 64): 12.3K।
    • 
          
  • Branch 2 (1×1→96, 3×3→128): $192 \times 96 + 96 \times 9 \times 128 = 18.4K + 110.6K = 129K$।
    • 
          
  • Branch 3 (1×1→16, 5×5→32): $192 \times 16 + 16 \times 25 \times 32 = 3K + 12.8K = 15.8K$।
    • 
          
  • Branch 4 (pool + 1×1, 32): 6.1K।
    • 
          
  • Total: ~163K params।
    • 
        

        

          Saving: 393K → 163K, 58% reduction।
        

        

          FLOPs (per spatial location):
        

        
    
          
  • Naive 3×3: $192 \times 9 \times 128 = 221K$ MAC।
    • 
          
  • Bottleneck 3×3: $192 + 96 \times 9 \times 128 = 18.4K + 110.6K$ MAC।
    • 
          
  • 2x reduction।
    • 
        

        

          Accuracy:
        

        
    
          
  • Bottleneck — empirically equivalent বা slightly better।
    • 
          
  • Reason: 1×1 conv channel mixing — beneficial nonlinearity।
    • 
          
  • Information bottleneck — implicit regularization।
    • 
        

        

          Lesson — applicable everywhere:
        

        
    
          
  • ResNet-50 bottleneck block: 1×1 → 3×3 → 1×1।
    • 
          
  • MobileNet inverted bottleneck।
    • 
          
  • Transformer FFN: linear-up → activation → linear-down।
    • 
          
  • Universal pattern: expand → process → compress।
    • 
        

        

          মূল উপলব্ধি: Smart architecture > brute force। 1×1 conv "trivial" মনে হলেও — modern CNN-এর backbone। Compute aware design pre-DL also matters।
        

      

    


    

      
        প্র ০২
        GoogLeNet-এ auxiliary classifier — middle layer থেকে loss। কেন helpful, এবং কেন BatchNorm আসার পর deprecated?
      
      

        

          Auxiliary classifier — vanishing gradient problem-এর pre-BatchNorm solution।
        

        

          Vanishing gradient revisited:
        

        
    
          
  • ২২ layer GoogLeNet — backprop chain rule।
    • 
          
  • Last layer-এর loss → first layer-এ gradient তেমন কিছু না।
    • 
          
  • Early layer training stagnant।
    • 
        

        

          Auxiliary classifier solution:
        

        
    
          
  • Inception 4a ও 4d-এর পরে — small head (FC + softmax)।
    • 
          
  • Independent loss compute — local gradient signal।
    • 
          
  • Combined loss: $\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{main}} + 0.3(\mathcal{L}_{\text{aux1}} + \mathcal{L}_{\text{aux2}})$।
    • 
          
  • Middle layer directly supervised।
    • 
        

        

          Why this works:
        

        
    
          
  • Shorter gradient path — middle layer-এ stable signal।
    • 
          
  • Implicit regularization — middle features must be "classify-able"।
    • 
          
  • Ensemble effect — three classifiers train।
    • 
        

        

          BatchNorm replaces:
        

        
    
          
  • Ioffe & Szegedy (২০১৫) — BN paper।
    • 
          
  • Per-layer normalization — gradient stable through depth।
    • 
          
  • Deeper networks (Inception v3, ResNet 50+) — direct end-to-end train possible।
    • 
          
  • Auxiliary classifier-এর role redundant।
    • 
        

        

          Inception v3-এ:
        

        
    
          
  • One auxiliary classifier (vs two in v1)।
    • 
          
  • Mainly regularizer — minor accuracy gain।
    • 
          
  • BN with auxiliary — no significant boost।
    • 
        

        

          Modern architecture:
        

        
    
          
  • ResNet — skip connection (architectural)।
    • 
          
  • BatchNorm/LayerNorm — normalization layer।
    • 
          
  • DenseNet — every layer to every layer connectivity।
    • 
          
  • All — vanishing gradient address।
    • 
        

        

          Modern revival — deep supervision:
        

        
    
          
  • U-Net++, DeepLab — multi-level supervision।
    • 
          
  • Knowledge distillation — teacher's intermediate feature।
    • 
          
  • Auxiliary task head (multi-task learning)।
    • 
          
  • Concept lives, just under different names।
    • 
        

        

          মূল উপলব্ধি: Engineering trick — temporary fix till fundamental solution। GoogLeNet-এর auxiliary classifier, BN-এর জন্ম-এর পথ সাফ। Engineering pragmatism + theoretical advance।
        

      

    


    

      
        প্র ০৩
        Inception v3-এ 5×5 → দু'টি 3×3 (factorization), 7×7 → 1×7 + 7×1 (asymmetric)। দু'টি কেন আলাদা treatment?
      
      

        

          এটি Szegedy et al.-এর factorization paper-এর key insight। Convolution decomposition theory।
        

        

          Symmetric factorization (5×5 → 3×3 + 3×3):
        

        
    
          
  • একটি 5×5 RF = দু'টি stacked 3×3 RF।
    • 
          
  • Parameter: $25c^2 \to 18c^2$ — 28% reduction।
    • 
          
  • Two non-linearities (extra ReLU)।
    • 
          
  • Standard 2D filter — maintain isotropy।
    • 
        

        

          Asymmetric factorization (7×7 → 1×7 + 7×1):
        

        
    
          
  • একটি 7×7 = একটি 1×7 (horizontal) + একটি 7×1 (vertical)।
    • 
          
  • Parameter: $49c^2 \to 14c^2$ — 71% reduction!
    • 
          
  • Massive saving for large kernel।
    • 
          
  • কিন্তু — strict 2D pattern lose। Only "separable" filter approximate।
    • 
        

        

          Why asymmetric only for big kernels?
        

        
    
          
  • 3×3 → 1×3 + 3×1 — saving মাত্র 33%। 3×3-ই compute-friendly।
    • 
          
  • 5×5 — borderline।
    • 
          
  • 7×7+ — saving worth approximation cost।
    • 
        

        

          Information theory perspective:
        

        
    
          
  • 2D pattern = full 49-D space (7×7)।
    • 
          
  • Separable = rank-1 approximation।
    • 
          
  • Most natural image kernel rank-1-এর কাছাকাছি — Sobel separable।
    • 
          
  • Approximation cheap, useful।
    • 
        

        

          Empirical results (Inception v3):
        

        
    
          
  • 5×5 → 3×3+3×3: equal accuracy, faster।
    • 
          
  • 7×7 → 1×7+7×1: slightly worse, much faster।
    • 
          
  • 3×3 → 1×3+3×1: noticeable accuracy drop। Avoided।
    • 
        

        

          Modern architectures:
        

        
    
          
  • MobileNet: depthwise separable — extreme factorization (per-channel + 1×1)।
    • 
          
  • Xception: "depthwise + pointwise" — 7×7-এর extreme version।
    • 
          
  • ConvMixer: 7×7 depthwise + 1×1।
    • 
          
  • Vision Transformer: 16×16 patch embedding — extreme factorization (no spatial conv)।
    • 
        

        

          Hardware impact:
        

        
    
          
  • Asymmetric conv — special CUDA kernel।
    • 
          
  • NVIDIA cuDNN optimize 3×3 most।
    • 
          
  • 1×7 conv — slower than 7×1 in some implementations (memory layout)।
    • 
        

        

          মূল উপলব্ধি: Factorization — math-driven optimization। 2D → 1D approximation modern efficient architecture-এর backbone।
        

      

    


    

      
        প্র ০৪
        FID (Fréchet Inception Distance) — image generation quality metric। Inception V3 feature কেন এত universal? কী problem এই metric solve করে?
      
      

        

          FID — image generation evaluation-এর de-facto standard। Heusel et al. (২০১৭)।
        

        

          Problem এর before:
        

        
    
          
  • GAN আউটপুট কীভাবে evaluate?
    • 
          
  • Inception Score (২০১৬, Salimans et al.) — Inception V3-এর softmax distribution diversity।
    • 
          
  • সমস্যা: real image distribution-এর সাথে compare করে না।
    • 
        

        

          FID definition:
        

        

          $$\text{FID} = \| \mu_r - \mu_g \|^2 + \text{Tr}(\Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2})$$
        

        
    
          
  • $\mu, \Sigma$ — Inception V3-এর pool3 feature-এর mean & covariance।
    • 
          
  • $r$ = real images, $g$ = generated images।
    • 
          
  • Lower FID = generated distribution real-এর কাছাকাছি।
    • 
        

        

          Why Inception V3 specifically?
        

        
    
          
  • Pretrained on ImageNet: diverse natural image features।
    • 
          
  • Feature dimensionality: 2048-D pool3 — rich representation।
    • 
          
  • Standardization: all researchers same network = comparable scores।
    • 
          
  • Historic accident: ২০১৭-এ V3 SOTA। যা set হয়ে গেল।
    • 
        

        

          FID-এর strengths:
        

        
    
          
  • Real vs generated distribution direct compare।
    • 
          
  • Mode collapse detect (low diversity → high FID)।
    • 
          
  • Visual quality + diversity দু'টোই capture।
    • 
          
  • Human perception-এর সাথে correlation ভাল।
    • 
        

        

          FID-এর limitations:
        

        
    
          
  • ImageNet-bias — non-natural image (medical, satellite) inappropriate।
    • 
          
  • Inception V3 feature — perfect representation না।
    • 
          
  • Sample size sensitive (50K typical)।
    • 
          
  • Outliers can game।
    • 
          
  • Resolution-dependent।
    • 
        

        

          Modern alternatives:
        

        
    
          
  • CLIP-FID: CLIP feature use — multimodal aware।
    • 
          
  • KID (Kernel Inception Distance): unbiased estimator।
    • 
          
  • Precision/Recall: fidelity vs diversity decompose।
    • 
          
  • Improved Precision/Recall: nearest-neighbor based।
    • 
        

        

          Bangladeshi context:
        

        
    
          
  • Bangla art generation (paintings, calligraphy)।
    • 
          
  • FID-এর Inception V3 — ImageNet bias problem।
    • 
          
  • Domain-specific feature extractor → better metric।
    • 
        

        

          Stable Diffusion era:
        

        
    
          
  • FID still main metric, but criticized।
    • 
          
  • Human evaluation — gold standard।
    • 
          
  • CLIP score (text-image alignment) for text-to-image।
    • 
        

        

          মূল উপলব্ধি: Metric design ML-এর সবচেয়ে underrated skill। Inception V3 — accidental backbone of generation evaluation। Standard সর্বদা challenge করুন।
        

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Inception 3a output channels: 64 + 128 + 32 + 32 কেন এই specific number? Sum = 256।
        
    
          
          
    
            
    Hand-tuned by GoogLeNet authors empirically। 1×1 ও pool branch কম, 3×3 dominant — natural feature 3×3-এ best capture। Modern NAS (Neural Architecture Search) এই split auto-discover।
    
          
    
        
    
      
    2. 

      
  2. 
        Pretrained Inception: torchvision থেকে Inception V3 load করে inference example লিখুন। Note: 299×299 input!
        
    
          
          
    
            
    from torchvision.models import inception_v3, Inception_V3_Weights
w = Inception_V3_Weights.IMAGENET1K_V1
m = inception_v3(weights=w).eval()
preprocess = w.transforms()  # 299×299, normalize
x = preprocess(img).unsqueeze(0)
out = m(x)
    
          
    
        
    
      
    3. 

      
  3. 
        ভাবুন: Inception module-এ pool branch — কেন? Pure conv-only Inception কী missing?
        
    
          
          
    
            
    Pool branch = "no learning" baseline — local invariance। Different operation type ensemble effect। Modern ResNet skip connection-এ similar role — identity baseline।
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন · ABCL TECH-এ আপনার পরবর্তী পদক্ষেপ

    

    

      কোড রানার কাজ না করলে? ব্রাউজারে কাজ না করলে
      Google Colab
      ব্যবহার করুন — Google-এর ফ্রি অনলাইন Python পরিবেশ, শুধু Gmail অ্যাকাউন্ট লাগে।
    

  







  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১২ · VGG