পাঠ ১৫ · ৩৩-এর মধ্যে · মডিউল ৩
Home / AI Courses / MLOps / Deployment patterns

Deployment patterns

Patterns — REST, gRPC, batch, streaming, edge
৭ মিনিট পড়া মধ্য · Intermediate Concept

এই পাঠে যা শিখবেন

  • প্রতিটি deployment pattern — fit ও trade-off
  • Latency vs throughput-এর comparison
  • কখন serverless overkill, কখন essential
  • Bangladesh production scenarios — সঠিক pattern

১ · কেন pattern matter

"Model deploy" বললে অনেকেই FastAPI REST এর কথা ভাবে। কিন্তু production reality অনেক বেশি diverse:

  • Daraz রাতে সব user-এর recommendation pre-compute করে — batch।
  • Pathao surge prediction প্রতি সেকেন্ডে stream-এ চলে — streaming।
  • bKash fraud check 50ms-এ চাই — online API।
  • মোবাইল app-এ image enhancement — edge (on-device)।

২ · ৭টি pattern

  • REST API: HTTP+JSON synchronous। Universal, easy। Latency ১০-১০০ms typical।
  • gRPC: binary, faster than REST, streaming support। Microservice-এর মধ্যে।
  • Batch: periodic large prediction generate; result DB-তে। Hours OK।
  • Streaming: Kafka/Pulsar message → process → output stream। Continuous।
  • Edge / On-device: mobile/IoT-এ inference; no network।
  • Serverless: Lambda/Cloud Run — pay-per-request; cold start risk।
  • Embedded: server-side process within larger app — model loaded inline।

৩ · Latency budget

$$ \text{Total latency} = \text{network} + \text{queue} + \text{preprocess} + \text{inference} + \text{postprocess} + \text{response} $$

Bangladesh fintech p99 budget example:

  • Total: 100 ms।
  • Network (within data-center): 5 ms।
  • Queue + serialization: 10 ms।
  • Preprocess + feature lookup: 15 ms।
  • Model inference: 50 ms।
  • Postprocess: 5 ms।
  • Response: 10 ms।
  • Buffer: 5 ms।

৪ · Decision framework

Pattern picker

১) Real-time per-request? → REST/gRPC online।
২) "All users at once" pre-compute fine? → Batch।
৩) Continuous event stream? → Streaming।
৪) No network OR privacy + low compute? → Edge।
৫) Sporadic + low traffic? → Serverless।
৬) Already in larger app, low scale? → Embedded।

৫ · Batch vs online detail

(Lesson 19-এ deeper). Quick comparison:

  • Batch latency: hours/days; online: ms।
  • Batch cost: $/1M predictions cheap; online: always-on infra।
  • Batch freshness: day-old data; online: instant।
  • Batch infrastructure: Spark/Beam; online: FastAPI/Triton।

৬ · Edge deployment

Bangla mobile keyboard auto-correct, on-device image enhancement।

  • Frameworks: TensorFlow Lite, Core ML (iOS), ONNX Runtime mobile।
  • Model size: usually < 50 MB। Quantize aggressively।
  • No backend cost — but no central monitoring।
  • Update via app store — slow।

৭ · Serverless

  • AWS Lambda, GCP Cloud Run, Azure Functions।
  • Pros: zero idle cost, auto-scale।
  • Cons: cold start (1-10s), CPU/RAM limit, no GPU usually।
  • Use case: low-volume model, sporadic traffic, dev/test environment।
  • Cold start mitigation: provisioned concurrency (cost), container snapshot।
Deployment patterns — latency vs throughput map throughput → ↑ latency tolerance REST / gRPC low-latency online Streaming continuous events Batch large-scale offline Serverless cold start risk Edge on-device, no net
Pattern selection — latency tolerance + throughput requirement-এর function। Production-এ একাধিক pattern coexist।

৮ · Sample REST endpoint

Python · Minimal REST 
from fastapi import FastAPI
from pydantic import BaseModel
import joblib

model = joblib.load("model.joblib")
app = FastAPI()

class ReviewIn(BaseModel):
    text: str

class PredictionOut(BaseModel):
    label: str
    confidence: float

@app.post("/predict", response_model=PredictionOut)
def predict(req: ReviewIn):
    proba = model.predict_proba([req.text])[0]
    label_idx = proba.argmax()
    return PredictionOut(
        label=model.classes_[label_idx],
        confidence=float(proba[label_idx]),
    )

    


    

      Lesson 16-এ FastAPI complete production-grade পদ্ধতি বিস্তারিত। এখানে শুধু pattern overview।
    


    

      Pattern decision-এ "৮০% production = REST online" — কিন্তু বাকি ২০% common cases-এ ভিন্ন pattern এর ROI বিশাল।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      প্র ০১"Bangladesh fintech (bKash style) — কোন patterns মিশ্রণে?"
      

        
একটি bKash-style fintech-এ multiple ML use case — প্রতিটিতে ভিন্ন pattern।

        
    
          
  • Fraud check (real-time): REST/gRPC online — 50ms p99। Per-transaction।
    • 
          
  • Credit scoring: nightly batch; result DB-তে cache; serving অনুরোধে DB lookup।
    • 
          
  • Customer segmentation: weekly batch; CRM-এ feed।
    • 
          
  • Chatbot intent classification: REST online + streaming for log analysis।
    • 
          
  • Anomaly detection: Kafka stream; alert-driven।
    • 
        

        
Each model right-tooled — saves cost, simplifies SLA।

        
Anti-pattern: "Everything REST" — fraud (50ms) আর segmentation (24h OK) same infrastructure-এ — over-engineered।

        
মূল উপলব্ধি: Pattern selection per-model, latency budget driven। Mixed deployment normal — actually preferred।

      

    


    

      প্র ০২"Cold start serverless — kibhabe practical?"
      

        
Cold start = idle container start time। ML model serverless-এ painful।

        
Cold start sources:

        
    
          
  • Container image pull: GB-scale image — slow।
    • 
          
  • Library import: PyTorch ~5s।
    • 
          
  • Model load: serialized model unpickle।
    • 
          
  • GPU init: extra second।
    • 
        

        
Mitigations:

        
    
          
  • Provisioned concurrency (Lambda) — ৫-১০ instance always warm; cost more।
    • 
          
  • SnapStart (Lambda) — checkpoint runtime state।
    • 
          
  • Keep model loaded between invocations — module-level load (works for "warm" only)।
    • 
          
  • Smaller model image — slim base, ONNX runtime instead of PyTorch।
    • 
          
  • Cloud Run min-instance — keep warm pool।
    • 
        

        
"Use serverless" decision:

        
    
          
  • Burst traffic, mostly idle — yes (with mitigation)।
    • 
          
  • Constant traffic — over-engineered; just K8s।
    • 
          
  • P99 strict SLA — risky; better K8s with autoscaler।
    • 
        

        
মূল উপলব্ধি: Serverless ML — sporadic traffic, cost-conscious, P99 lenient। Don't fight cold start with mitigations expensive — sometimes K8s simpler।

      

    


    

      প্র ০৩"Edge deployment — Bangladesh-এ challenges?"
      

        
Edge ML attractive — privacy, latency, cost — কিন্তু operational hard।

        
Bangladesh device landscape:

        
    
          
  • Mobile: budget Android dominant; iOS ~5%।
    • 
          
  • RAM 2-4 GB common; high-end 8-12 GB।
    • 
          
  • CPU low-end Snapdragon 4xx — not powerful।
    • 
          
  • Battery sensitivity high।
    • 
        

        
Constraints:

        
    
          
  • Model size — < 50 MB।
    • 
          
  • Inference time < 200ms — UX usable।
    • 
          
  • Battery — heavy model = drain।
    • 
          
  • RAM — large model OOM-kill।
    • 
        

        
Tooling:

        
    
          
  • TF Lite — most popular; converter from TF।
    • 
          
  • ONNX Runtime mobile — flexible source models।
    • 
          
  • MediaPipe — Google's mobile ML kit।
    • 
          
  • Core ML iOS।
    • 
        

        
Challenges:

        
    
          
  • Update cycle — model in app, app store deploy।
    • 
          
  • Telemetry — usage/accuracy tough monitor।
    • 
          
  • Drift detection — central server-এ aggregate signal upload।
    • 
          
  • Device fragmentation — model behavior varies।
    • 
        

        
BD use cases successful:

        
    
          
  • Mobile keyboard predictive (Bangla)।
    • 
          
  • Photo gallery enhancement।
    • 
          
  • OCR (NID scan, document)।
    • 
          
  • Voice command (offline)।
    • 
        

        
Hybrid pattern:

        
    
          
  • Edge basic; backend escalate complex।
    • 
          
  • Edge "first guess", server "second opinion"।
    • 
        

        
মূল উপলব্ধি: Edge ML privacy + UX-এ powerful, কিন্তু operational complex। Quantization + distillation essential। Bangladesh budget-device matters most।

      

    


    

      প্র ০৪"Pattern migration — REST থেকে batch-এ যাওয়া কোন signals দেখলে?"
      

        
Pattern migration cost আছে — wrong pattern indication clear হলে move worth।

        
REST → batch signals:

        
    
          
  • Cost analysis — REST infrastructure idle 70%+ time।
    • 
          
  • Predictable user need — "all users-এর daily score" type query।
    • 
          
  • Latency tolerance found relaxed — "actually 24h cached fine"।
    • 
          
  • Volume bursting — single batch cheaper than always-on।
    • 
        

        
Batch → online signals:

        
    
          
  • Stale data complaint।
    • 
          
  • "Refresh now" features needed।
    • 
          
  • Per-user customization growing।
    • 
          
  • Real-time signal (e.g., clickstream) underused।
    • 
        

        
Online → streaming signals:

        
    
          
  • Polling pattern emerging।
    • 
          
  • Event-driven architecture maturing।
    • 
          
  • Per-event processing required।
    • 
        

        
Migration cost:

        
    
          
  • Rewrite serving logic।
    • 
          
  • Different infrastructure (Spark vs FastAPI)।
    • 
          
  • Different observability।
    • 
          
  • Team skill shift।
    • 
        

        
Strategy:

        
    
          
  • Don't migrate all-at-once। Hybrid period।
    • 
          
  • New use case — try better pattern first।
    • 
          
  • Existing well-running — don't touch।
    • 
        

        
মূল উপলব্ধি: Pattern migration triggered by clear cost/latency signals। Premature migration — sunk cost; late migration — accumulated waste। Yearly architecture review healthy practice।

      

    

  


  

    
 অনুশীলন

    
    
      
  1. Pattern matrix: Daraz-এর ৫টি ML use case list করুন; প্রতিটির জন্য সঠিক pattern + reasoning।
        
    
        
    
          
      
            
    • Recommendation list — daily batch + serving cache।
      • 
            
    • Search ranking — REST online (per-query)।
      • 
            
    • Fraud detection — REST online + streaming alert।
      • 
            
    • Demand forecasting — batch (planning)।
      • 
            
    • Image search (visual similar) — REST + edge fallback।
      • 
          
    
        
    
      
    2. 
      
  2. Latency budget: একটি 100ms budget-এ কোন কোন component কত % নেবে — table তৈরি করুন।
        
    
        
    উপরের section ৩-এর breakdown follow করুন; নিজের project-এ adapt।
    
      
    3. 
      
  3. চিন্তা: Pathao surge prediction — pattern combination কেন? কীভাবে integrate?
        
    
        
    
          
      
            
    • Streaming for live event ingestion (rides, demand pings)।
      • 
            
    • Online API per-zone prediction।
      • 
            
    • Batch nightly model retrain।
      • 
            
    • Edge — driver mobile app local cache, cold-start শ্রেষ্ঠ approximation।
      • 
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    
  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৪ · Kubeflow Pipelines