পাঠ ১৫ · ৩৩-এর মধ্যে · মডিউল ৩
Home / AI Courses / MLOps / Deployment patterns

Deployment patterns

Patterns — REST, gRPC, batch, streaming, edge
৭ মিনিট পড়া মধ্য · Intermediate Concept

এই পাঠে যা শিখবেন

  • প্রতিটি deployment pattern — fit ও trade-off
  • Latency vs throughput-এর comparison
  • কখন serverless overkill, কখন essential
  • Bangladesh production scenarios — সঠিক pattern

১ · কেন pattern matter

"Model deploy" বললে অনেকেই FastAPI REST এর কথা ভাবে। কিন্তু production reality অনেক বেশি diverse:

  • Daraz রাতে সব user-এর recommendation pre-compute করে — batch.
  • Pathao surge prediction প্রতি সেকেন্ডে stream-এ চলে — streaming.
  • bKash fraud check 50ms-এ চাই — online API.
  • মোবাইল app-এ image enhancement — edge (on-device).

২ · ৭টি pattern

  • REST API: HTTP+JSON synchronous. Universal, easy. Latency ১০-১০০ms typical.
  • gRPC: binary, faster than REST, streaming support. Microservice-এর মধ্যে।
  • Batch: periodic large prediction generate; result DB-তে। Hours OK.
  • Streaming: Kafka/Pulsar message → process → output stream. Continuous.
  • Edge / On-device: mobile/IoT-এ inference; no network.
  • Serverless: Lambda/Cloud Run — pay-per-request; cold start risk.
  • Embedded: server-side process within larger app — model loaded inline.

৩ · Latency budget

$$ \text{Total latency} = \text{network} + \text{queue} + \text{preprocess} + \text{inference} + \text{postprocess} + \text{response} $$

Bangladesh fintech p99 budget example:

  • Total: 100 ms.
  • Network (within data-center): 5 ms.
  • Queue + serialization: 10 ms.
  • Preprocess + feature lookup: 15 ms.
  • Model inference: 50 ms.
  • Postprocess: 5 ms.
  • Response: 10 ms.
  • Buffer: 5 ms.

৪ · Decision framework

Pattern picker

১) Real-time per-request? → REST/gRPC online.
২) "All users at once" pre-compute fine? → Batch.
৩) Continuous event stream? → Streaming.
৪) No network OR privacy + low compute? → Edge.
৫) Sporadic + low traffic? → Serverless.
৬) Already in larger app, low scale? → Embedded.

৫ · Batch vs online detail

(Lesson 19-এ deeper). Quick comparison:

  • Batch latency: hours/days; online: ms.
  • Batch cost: $/1M predictions cheap; online: always-on infra.
  • Batch freshness: day-old data; online: instant.
  • Batch infrastructure: Spark/Beam; online: FastAPI/Triton.

৬ · Edge deployment

Bangla mobile keyboard auto-correct, on-device image enhancement.

  • Frameworks: TensorFlow Lite, Core ML (iOS), ONNX Runtime mobile.
  • Model size: usually < 50 MB. Quantize aggressively.
  • No backend cost — but no central monitoring.
  • Update via app store — slow.

৭ · Serverless

  • AWS Lambda, GCP Cloud Run, Azure Functions.
  • Pros: zero idle cost, auto-scale.
  • Cons: cold start (1-10s), CPU/RAM limit, no GPU usually.
  • Use case: low-volume model, sporadic traffic, dev/test environment.
  • Cold start mitigation: provisioned concurrency (cost), container snapshot.
Deployment patterns — latency vs throughput map throughput → ↑ latency tolerance REST / gRPC low-latency online Streaming continuous events Batch large-scale offline Serverless cold start risk Edge on-device, no net
Pattern selection — latency tolerance + throughput requirement-এর function. Production-এ একাধিক pattern coexist.

৮ · Sample REST endpoint

Python · Minimal REST 
from fastapi import FastAPI
from pydantic import BaseModel
import joblib

model = joblib.load("model.joblib")
app = FastAPI()

class ReviewIn(BaseModel):
    text: str

class PredictionOut(BaseModel):
    label: str
    confidence: float

@app.post("/predict", response_model=PredictionOut)
def predict(req: ReviewIn):
    proba = model.predict_proba([req.text])[0]
    label_idx = proba.argmax()
    return PredictionOut(
        label=model.classes_[label_idx],
        confidence=float(proba[label_idx]),
    )

    


    

      Lesson 16-এ FastAPI complete production-grade পদ্ধতি বিস্তারিত। এখানে শুধু pattern overview.
    


    

      Pattern decision-এ "৮০% production = REST online" — কিন্তু বাকি ২০% common cases-এ ভিন্ন pattern এর ROI বিশাল।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      প্র ০১"Bangladesh fintech (bKash style) — কোন patterns মিশ্রণে?"
      

        
একটি bKash-style fintech-এ multiple ML use case — প্রতিটিতে ভিন্ন pattern.

        
    
          
  • Fraud check (real-time): REST/gRPC online — 50ms p99. Per-transaction.
    • 
          
  • Credit scoring: nightly batch; result DB-তে cache; serving অনুরোধে DB lookup.
    • 
          
  • Customer segmentation: weekly batch; CRM-এ feed.
    • 
          
  • Chatbot intent classification: REST online + streaming for log analysis.
    • 
          
  • Anomaly detection: Kafka stream; alert-driven.
    • 
        

        
Each model right-tooled — saves cost, simplifies SLA.

        
Anti-pattern: "Everything REST" — fraud (50ms) আর segmentation (24h OK) same infrastructure-এ — over-engineered.

        
মূল উপলব্ধি: Pattern selection per-model, latency budget driven. Mixed deployment normal — actually preferred.

      

    


    

      প্র ০২"Cold start serverless — kibhabe practical?"
      

        
Cold start = idle container start time. ML model serverless-এ painful.

        
Cold start sources:

        
    
          
  • Container image pull: GB-scale image — slow.
    • 
          
  • Library import: PyTorch ~5s.
    • 
          
  • Model load: serialized model unpickle.
    • 
          
  • GPU init: extra second.
    • 
        

        
Mitigations:

        
    
          
  • Provisioned concurrency (Lambda) — ৫-১০ instance always warm; cost more.
    • 
          
  • SnapStart (Lambda) — checkpoint runtime state.
    • 
          
  • Keep model loaded between invocations — module-level load (works for "warm" only).
    • 
          
  • Smaller model image — slim base, ONNX runtime instead of PyTorch.
    • 
          
  • Cloud Run min-instance — keep warm pool.
    • 
        

        
"Use serverless" decision:

        
    
          
  • Burst traffic, mostly idle — yes (with mitigation).
    • 
          
  • Constant traffic — over-engineered; just K8s.
    • 
          
  • P99 strict SLA — risky; better K8s with autoscaler.
    • 
        

        
মূল উপলব্ধি: Serverless ML — sporadic traffic, cost-conscious, P99 lenient. Don't fight cold start with mitigations expensive — sometimes K8s simpler.

      

    


    

      প্র ০৩"Edge deployment — Bangladesh-এ challenges?"
      

        
Edge ML attractive — privacy, latency, cost — কিন্তু operational hard.

        
Bangladesh device landscape:

        
    
          
  • Mobile: budget Android dominant; iOS ~5%.
    • 
          
  • RAM 2-4 GB common; high-end 8-12 GB.
    • 
          
  • CPU low-end Snapdragon 4xx — not powerful.
    • 
          
  • Battery sensitivity high.
    • 
        

        
Constraints:

        
    
          
  • Model size — < 50 MB.
    • 
          
  • Inference time < 200ms — UX usable.
    • 
          
  • Battery — heavy model = drain.
    • 
          
  • RAM — large model OOM-kill.
    • 
        

        
Tooling:

        
    
          
  • TF Lite — most popular; converter from TF.
    • 
          
  • ONNX Runtime mobile — flexible source models.
    • 
          
  • MediaPipe — Google's mobile ML kit.
    • 
          
  • Core ML iOS.
    • 
        

        
Challenges:

        
    
          
  • Update cycle — model in app, app store deploy.
    • 
          
  • Telemetry — usage/accuracy tough monitor.
    • 
          
  • Drift detection — central server-এ aggregate signal upload.
    • 
          
  • Device fragmentation — model behavior varies.
    • 
        

        
BD use cases successful:

        
    
          
  • Mobile keyboard predictive (Bangla).
    • 
          
  • Photo gallery enhancement.
    • 
          
  • OCR (NID scan, document).
    • 
          
  • Voice command (offline).
    • 
        

        
Hybrid pattern:

        
    
          
  • Edge basic; backend escalate complex.
    • 
          
  • Edge "first guess", server "second opinion".
    • 
        

        
মূল উপলব্ধি: Edge ML privacy + UX-এ powerful, কিন্তু operational complex. Quantization + distillation essential. Bangladesh budget-device matters most.

      

    


    

      প্র ০৪"Pattern migration — REST থেকে batch-এ যাওয়া কোন signals দেখলে?"
      

        
Pattern migration cost আছে — wrong pattern indication clear হলে move worth.

        
REST → batch signals:

        
    
          
  • Cost analysis — REST infrastructure idle 70%+ time.
    • 
          
  • Predictable user need — "all users-এর daily score" type query.
    • 
          
  • Latency tolerance found relaxed — "actually 24h cached fine".
    • 
          
  • Volume bursting — single batch cheaper than always-on.
    • 
        

        
Batch → online signals:

        
    
          
  • Stale data complaint.
    • 
          
  • "Refresh now" features needed.
    • 
          
  • Per-user customization growing.
    • 
          
  • Real-time signal (e.g., clickstream) underused.
    • 
        

        
Online → streaming signals:

        
    
          
  • Polling pattern emerging.
    • 
          
  • Event-driven architecture maturing.
    • 
          
  • Per-event processing required.
    • 
        

        
Migration cost:

        
    
          
  • Rewrite serving logic.
    • 
          
  • Different infrastructure (Spark vs FastAPI).
    • 
          
  • Different observability.
    • 
          
  • Team skill shift.
    • 
        

        
Strategy:

        
    
          
  • Don't migrate all-at-once. Hybrid period.
    • 
          
  • New use case — try better pattern first.
    • 
          
  • Existing well-running — don't touch.
    • 
        

        
মূল উপলব্ধি: Pattern migration triggered by clear cost/latency signals. Premature migration — sunk cost; late migration — accumulated waste. Yearly architecture review healthy practice.

      

    

  


  

    
 অনুশীলন

    
    
      
  1. Pattern matrix: Daraz-এর ৫টি ML use case list করুন; প্রতিটির জন্য সঠিক pattern + reasoning.
        
    
        
    
          
      
            
    • Recommendation list — daily batch + serving cache.
      • 
            
    • Search ranking — REST online (per-query).
      • 
            
    • Fraud detection — REST online + streaming alert.
      • 
            
    • Demand forecasting — batch (planning).
      • 
            
    • Image search (visual similar) — REST + edge fallback.
      • 
          
    
        
    
      
    2. 
      
  2. Latency budget: একটি 100ms budget-এ কোন কোন component কত % নেবে — table তৈরি করুন।
        
    
        
    উপরের section ৩-এর breakdown follow করুন; নিজের project-এ adapt.
    
      
    3. 
      
  3. চিন্তা: Pathao surge prediction — pattern combination কেন? কীভাবে integrate?
        
    
        
    
          
      
            
    • Streaming for live event ingestion (rides, demand pings).
      • 
            
    • Online API per-zone prediction.
      • 
            
    • Batch nightly model retrain.
      • 
            
    • Edge — driver mobile app local cache, cold-start শ্রেষ্ঠ approximation.
      • 
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    
  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৪ · Kubeflow Pipelines