পাঠ ২৮ · ৩৩-এর মধ্যে · মডিউল ৪
Home / AI Courses / MLOps / LLMOps essentials

LLMOps — কী আলাদা

LLMOps — what's different from classical MLOps
৮ মিনিট পড়া উচ্চ · Advanced Concept

এই পাঠে যা শিখবেন

  • LLMOps vs classical MLOps — key differences
  • Foundation model vs fine-tune vs RAG decision
  • Vector DB landscape
  • Bangla LLM-specific challenges

১ · কেন আলাদা

Classical ML: train custom model, deploy, monitor। LLM-এ অনেক assumption ভাঙে।

২ · ৭টি core difference

LLMOps differences
  1. Non-determinism: temperature, top-p — same prompt different output।
  2. Prompt as artifact: code-like, version controlled, tested।
  3. Foundation model dependency: rarely train from scratch; fine-tune or use API।
  4. RAG infrastructure: vector DB, retrieval, context management।
  5. Token cost: input + output tokens, model-tier pricing।
  6. Hallucinations: confidently wrong; classical metric inadequate।
  7. Streaming: token-by-token return; latency definition shifted।

৩ · Build vs buy

  • API (OpenAI, Anthropic, Cohere):
    • Pros: zero infra, latest model, fast।
    • Cons: per-token cost, data privacy, vendor lock।
  • Self-host open model (LLaMA, Mistral, Qwen):
    • Pros: data control, customizable, eventually cheaper at scale।
    • Cons: GPU cost, ops complexity, behind frontier।
  • Fine-tune:
    • Pros: domain-specific quality, smaller model possible।
    • Cons: training cost, drift management।

৪ · RAG (Retrieval Augmented Generation)

LLM-এর knowledge cut-off + hallucinations issue। RAG = retrieve relevant context → LLM answer।

  • Document ingestion → chunk → embed → vector DB।
  • Query: embed query → vector search → top-k chunks।
  • Prompt: "Given context: [...], answer: ..."।

৫ · Vector DB landscape

  • FAISS (Meta): in-memory library; fast; no persistence built-in।
  • pgvector (Postgres): embedded SQL extension; familiar; scalable।
  • Qdrant: Rust-built, fast, REST API।
  • Weaviate: GraphQL, schema-aware, hybrid search।
  • Pinecone: SaaS, easiest, pricey।
  • Chroma: simple, dev-friendly।
  • Milvus: Cloud-native, K8s-friendly, large scale।

৬ · Simple RAG example

Python · RAG with pgvector 
import openai
import psycopg2
from psycopg2.extras import execute_values

# 1. Ingest documents
def ingest(docs: list[str], conn):
    for doc in docs:
        emb = openai.embeddings.create(
            input=doc, model="text-embedding-3-small"
        ).data[0].embedding
        conn.execute(
            "INSERT INTO docs (text, embedding) VALUES (%s, %s)",
            (doc, emb),
        )

# 2. Retrieve top-k
def retrieve(query: str, k: int = 5, conn) -> list[str]:
    emb = openai.embeddings.create(
        input=query, model="text-embedding-3-small"
    ).data[0].embedding
    rows = conn.execute(
        """SELECT text FROM docs
           ORDER BY embedding <=> %s::vector
           LIMIT %s""",
        (emb, k),
    ).fetchall()
    return [r[0] for r in rows]

# 3. Generate
def answer(query: str, conn) -> str:
    chunks = retrieve(query, conn=conn)
    context = "\n".join(chunks)
    prompt = f"""Context:
{context}

Question: {query}
Answer:"""
    resp = openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content

    


    

      Minimal RAG: embed → store → retrieve → generate। Production-এ chunking strategy, hybrid search, reranker — সবই add।
    


    
৭ · Hallucinations


    

      LLM confidently wrong answer। Classical metric adequately capture না।
    

    
    
      
  • Detection: factuality check (knowledge graph), self-consistency, retrieval grounding।
    • 
      
  • Mitigation: RAG, low temperature, instruct model "say I don't know"।
    • 
      
  • Eval: human review, LLM-as-judge (Lesson 30)।
    • 
    


    
৮ · Bangla LLM specifics


    
    
      
  • Tokenization — Bangla character-rich; many model token-inefficient (300% more tokens)।
    • 
      
  • Models: BanglaT5, BanglaGPT, BanglaBERT, mBART, BanglaLLaMA।
    • 
      
  • Eval scarce — Bangla benchmarks limited।
    • 
      
  • Cultural context — Western model often miss।
    • 
    


    

      
        
        LLMOps stack — RAG architecture

        
        User query

        
        Embed query
        embedding model

        
        Vector DB
        retrieve top-k

        
        Reranker
        optional

        
        Prompt template
        versioned

        
        LLM (API/local)
        streaming

        
        Guardrails
        safety, citations

        
        User

        
        
        
        
        
        
        

        Each component own monitoring + cost + version। Classical ML stack-এর সাথে quite different।
      
      
Production RAG architecture — embed/retrieve/rerank/prompt/LLM/guardrails — many moving parts each independently versioned।

    


    

      Bangladesh-এ LLMOps adoption fast — chatbot, customer support, knowledge base। Bangla LLM eval ও cost-control critical।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      প্র ০১"Foundation model vs fine-tune vs RAG — কখন কোনটা?"
      

        
Choice depends task + data + budget।

        
Foundation model (out-of-box):

        
    
          
  • Pros: zero data needed, fast iterate।
    • 
          
  • Cons: generic; specialized task moderate quality।
    • 
          
  • Use: prototyping, simple tasks (summarize, translate)।
    • 
        

        
RAG:

        
    
          
  • Pros: knowledge grounding, no training, easy update knowledge।
    • 
          
  • Cons: retrieval quality dependent, latency added।
    • 
          
  • Use: question-answering with proprietary documents।
    • 
        

        
Fine-tune:

        
    
          
  • Pros: domain-specific quality, faster inference (smaller model)।
    • 
          
  • Cons: training data needed, retrain cost।
    • 
          
  • Use: classification, structured output, style adaptation।
    • 
        

        
Decision tree:

        
    
          
  • "Knowledge facts (changing)" → RAG।
    • 
          
  • "Style/format adaptation" → fine-tune।
    • 
          
  • "Generic task" → foundation।
    • 
          
  • "Complex domain" → fine-tune + RAG।
    • 
        

        
BD example — Bangla customer support:

        
    
          
  • Knowledge: company FAQ → RAG।
    • 
          
  • Tone: friendly Bangla → fine-tune।
    • 
          
  • Both combined: best result।
    • 
        

        
মূল উপলব্ধি: RAG-first usually। Fine-tune for style/format। Foundation for prototype। Combined production stack-এ different layers।

      

    


    

      প্র ০২"In-house LLM hosting vs API — Bangladesh fintech-এ?"
      

        
Strategic decision — privacy, cost, latency।

        
API (OpenAI, Anthropic):

        
    
          
  • Pros: best quality, no infra, fast iterate।
    • 
          
  • Cons: data privacy (financial), per-token cost, latency to US।
    • 
        

        
Self-host:

        
    
          
  • Pros: data control, fixed cost at scale, on-prem possible।
    • 
          
  • Cons: GPU expensive, ops, lower model quality।
    • 
        

        
Hybrid:

        
    
          
  • Sensitive — self-host (anonymized data API)।
    • 
          
  • Non-sensitive — API।
    • 
        

        
Bangladesh Bank ICT:

        
    
          
  • Customer data abroad — restrictive।
    • 
          
  • Self-host or anonymize before API।
    • 
        

        
Cost crossover:

        
    
          
  • OpenAI GPT-4o: $5/1M input + $15/1M output tokens।
    • 
          
  • 1M conversations/day × 1K tokens = 1B tokens/day = $5K-15K/day।
    • 
          
  • Self-host: ~$5K/month cluster fixed।
    • 
          
  • Crossover at ~50K conversations/day।
    • 
        

        
BD reality:

        
    
          
  • Most BD startups API (cost manageable at low volume)।
    • 
          
  • Banks/fintech moving self-host for compliance + scale।
    • 
          
  • Hybrid common compromise।
    • 
        

        
মূল উপলব্ধি: Volume + privacy → self-host; agility + low-volume → API। Hybrid practical for most production।

      

    


    

      প্র ০৩"Bangla LLM tokenization inefficiency কতটা painful?"
      

        
Bangla character-rich; many models trained primarily on English।

        
Tokenization tax:

        
    
          
  • English: ~0.75 tokens/word।
    • 
          
  • Bangla on English-tuned tokenizer: ~3-5 tokens/word।
    • 
          
  • Translation: 1 Bangla word = 4× cost vs English।
    • 
        

        
Models with better Bangla:

        
    
          
  • GPT-4o — improved multilingual; still 2-3× English।
    • 
          
  • Claude 3 — comparable।
    • 
          
  • Gemini — Google's Bangla focus।
    • 
          
  • BanglaGPT, BanglaT5 — Bangla-native tokenizer; 1× rate।
    • 
        

        
Cost impact:

        
    
          
  • $1000/month English → $3000-5000/month Bangla।
    • 
          
  • Significant for scale।
    • 
        

        
Mitigation:

        
    
          
  • Translate to English (lossy)।
    • 
          
  • Bangla-native model (lower quality)।
    • 
          
  • Custom tokenizer + fine-tune।
    • 
          
  • Hybrid: structured fields English; user input Bangla।
    • 
        

        
BD platform reality:

        
    
          
  • Customer support chatbot — Bangla-heavy, cost surprise often।
    • 
          
  • Budget projection English-rate → 3× actual।
    • 
        

        
মূল উপলব্ধি: Bangla token cost asymmetry critical for BD ops planning। Realistic estimate at start। Bangla-native model evaluate seriously।

      

    


    

      প্র ০৪"Hallucination — production-grade mitigation strategy?"
      

        
Hallucination single biggest LLM production risk।

        
Sources:

        
    
          
  • Knowledge gap — model doesn't know, fabricates।
    • 
          
  • Reasoning failure — confident wrong logic।
    • 
          
  • Prompt ambiguity — model assumes।
    • 
        

        
Mitigation layers:

        
(১) Retrieval grounding:

        
    
          
  • "Answer from context only"।
    • 
          
  • Citations forced — answer cite source।
    • 
          
  • Mostly RAG-based।
    • 
        

        
(২) Low temperature:

        
    
          
  • Temperature 0.0-0.2 for fact-based।
    • 
          
  • Reduces creativity, increases consistency।
    • 
        

        
(৩) Self-verification:

        
    
          
  • "Check your answer for accuracy" follow-up prompt।
    • 
          
  • "I don't know" allowed in instruction।
    • 
        

        
(৪) Output guardrails:

        
    
          
  • NeMo Guardrails, LlamaGuard।
    • 
          
  • Citation verification — generated facts vs source।
    • 
        

        
(৫) Confidence scores:

        
    
          
  • Multi-sample agreement = confidence।
    • 
          
  • Low confidence → escalate human।
    • 
        

        
(৬) Human-in-loop:

        
    
          
  • Critical decision — review queue।
    • 
          
  • Spot-check sample।
    • 
        

        
BD context — financial advice:

        
    
          
  • Hallucinated rates / regulations dangerous।
    • 
          
  • Strict RAG grounding।
    • 
          
  • Disclaimers।
    • 
          
  • Low-confidence → human।
    • 
        

        
মূল উপলব্ধি: Hallucination = layered defense। RAG + temperature + guardrails + human review। Single technique insufficient। Risk-tier task accordingly।

      

    

  


  

    
 অনুশীলন

    
    
      
  1. Mini RAG: Local pgvector + OpenAI embedding + GPT-4o-mini — 10 docs ingest, query।
        
    Postgres pgvector enable। Sample text ingest। Query similar retrieve।
    
      
    2. 
      
  2. Compare: Same prompt 5 times — output variation observe।
        
    Temperature 0.7 visible variation; 0.0 stable. Determinism trade-off।
    
      
    3. 
      
  3. চিন্তা: Bangla customer support chatbot architecture design — vendor + RAG + guardrails।
        
    
          
      
            
    • OpenAI GPT-4o + custom Bangla prompt template।
      • 
            
    • RAG: Bangla FAQ pgvector।
      • 
            
    • Guardrails: PII filter, max length, profanity check।
      • 
            
    • Confidence scoring → human escalation।
      • 
            
    • Logging: full conversation analytics।
      • 
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    
  







  
   এই পাঠে ভুল পেয়েছেন? জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ২৭ · Prometheus & Grafana