পাঠ ১৪ · ৩৩-এর মধ্যে · মডিউল ২
Home / AI Courses / MLOps / Kubeflow Pipelines

Kubeflow Pipelines

KFP — K8s-native ML orchestrator
৮ মিনিট পড়া উচ্চ · Advanced Python DSL

এই পাঠে যা শিখবেন

  • KFP-এর core concept — component, pipeline, artifact
  • Python DSL দিয়ে pipeline define
  • Caching ও artifact lineage
  • Airflow vs KFP — কখন কোনটা

১ · কেন KFP, কেন আলাদা

Airflow general-purpose orchestrator। KFP ML-specific — K8s-native, container-per-step, artifact tracking built-in।

  • প্রতিটি step একটি container — strict isolation।
  • Artifact (data, model) automatic typed pass between components।
  • Caching automatic — same input → cached output।
  • UI-তে DAG + lineage graph visualization।

২ · Component

Component = containerized step। Three styles:

  • Lightweight Python component: function-based, KFP wraps in container।
  • Container component: existing Docker image-এ command + args।
  • Reusable component: YAML spec, share across pipelines।

৩ · Python DSL pipeline

Python · KFP v2 pipeline 
from kfp import dsl, compiler

@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["pandas", "scikit-learn"],
)
def fetch_data(date: str, output_csv: dsl.Output[dsl.Dataset]):
    import pandas as pd
    # download / generate data
    df = pd.DataFrame(...)
    df.to_csv(output_csv.path, index=False)


@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["pandas", "scikit-learn", "joblib"],
)
def train_model(
    input_csv: dsl.Input[dsl.Dataset],
    output_model: dsl.Output[dsl.Model],
    n_estimators: int = 100,
):
    import pandas as pd, joblib
    from sklearn.ensemble import RandomForestClassifier
    df = pd.read_csv(input_csv.path)
    X, y = df.drop(columns=["label"]), df["label"]
    model = RandomForestClassifier(n_estimators=n_estimators, random_state=42)
    model.fit(X, y)
    joblib.dump(model, output_model.path)


@dsl.component(base_image="python:3.11-slim", packages_to_install=["scikit-learn", "joblib"])
def evaluate(
    input_model: dsl.Input[dsl.Model],
    input_csv: dsl.Input[dsl.Dataset],
    metrics: dsl.Output[dsl.Metrics],
):
    import pandas as pd, joblib
    from sklearn.metrics import roc_auc_score
    model = joblib.load(input_model.path)
    df = pd.read_csv(input_csv.path)
    auc = roc_auc_score(df["label"], model.predict_proba(df.drop(columns=["label"]))[:, 1])
    metrics.log_metric("auc", auc)


@dsl.pipeline(name="bangla-sentiment-train")
def training_pipeline(date: str = "2025-01-15", n_estimators: int = 100):
    data_op = fetch_data(date=date)
    train_op = train_model(input_csv=data_op.outputs["output_csv"], n_estimators=n_estimators)
    eval_op = evaluate(
        input_model=train_op.outputs["output_model"],
        input_csv=data_op.outputs["output_csv"],
    )


# Compile + submit
compiler.Compiler().compile(training_pipeline, "pipeline.yaml")
# UI দিয়ে upload, OR programmatically:
# client = kfp.Client(host="...")
# client.create_run_from_pipeline_func(training_pipeline, arguments={...})

    


    

      Compiled pipeline.yaml → KFP UI বা Vertex Pipelines-এ submit। Artifacts (Dataset, Model, Metrics) — typed, automatically tracked।
    


    
৪ · Caching


    

      KFP automatically caches component results। Same component + same inputs → reuse previous output।
    

    
    
      
  • Cache key: component spec hash + input hash।
    • 
      
  • Cache hit → no rerun, instant।
    • 
      
  • Disable per-component: set_caching_options(enable_caching=False)।
    • 
      
  • Useful when developing — change last component only, earlier cached।
    • 
    


    
৫ · GPU support


    

      

        
        Python · GPU component
      

      
from kfp import dsl

@dsl.pipeline(name="gpu-train")
def gpu_pipeline():
    train_op = train_model(...)
    train_op.set_accelerator_type("nvidia.com/gpu")
    train_op.set_accelerator_limit(1)
    train_op.set_memory_limit("16Gi")
    train_op.set_cpu_limit("4")

      

        
        
      

      

    


    

      KFP K8s GPU node-এ schedule করে। Resource requests + limits ML-friendly।
    


    
৬ · Vertex Pipelines compatibility


    

      Same KFP DSL pipeline.yaml — Vertex AI Pipelines (GCP)-এও submit করা যায়। Portable: dev local KFP, prod Vertex Pipelines।
    


    
৭ · Airflow vs KFP


    
    
      
  • Airflow:
        
      
          
    • General-purpose, broader operator library।
      • 
          
    • Dynamic DAG limited।
      • 
          
    • Heavier control plane।
      • 
        
    
      
    • 
      
  • KFP:
        
      
          
    • ML-specific, container-per-step।
      • 
          
    • Type-checked artifact passing।
      • 
          
    • K8s-native — needs K8s।
      • 
        
    
      
    • 
      
  • Common pattern:
        
      
          
    • Airflow for ETL + scheduled ML retraining trigger।
      • 
          
    • KFP for actual ML pipeline execution।
      • 
          
    • Airflow trigger-করে KFP run।
      • 
        
    
      
    • 
    


    

      
        
          
            
          
        

        Kubeflow Pipelines — container-per-step on K8s
        Typed artifacts auto-passed

        
        Component 1
        fetch_data
        → Dataset

        
        Component 2
        train_model
        GPU pod
        → Model

        
        Component 3
        evaluate
        → Metrics

        
        

        
        
        Artifact store (S3/GCS) — typed, lineage tracked
        Dataset, Model, Metrics — auto persisted, auto mounted

        
        
        
      
      
প্রতিটি component আলাদা K8s pod-এ চালু। Artifact (Dataset, Model, Metrics) auto-persist artifact store-এ; পরের component automatic mount।

    


    

      Bangladesh-এ KFP adoption Airflow-এর চেয়ে কম, কিন্তু GPU-heavy ML team-এ growing। Vertex Pipelines (GCP Mumbai) — managed, no K8s ops।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      প্র ০১"KFP vs Airflow — ML-এর জন্য কোনটা better?"
      

        
"Better" depends — both excel different aspects।

        
KFP advantages:

        
    
          
  • Container-per-step — strict isolation, no dependency hell।
    • 
          
  • Typed artifacts — Dataset, Model, Metrics built-in।
    • 
          
  • Auto caching — dev iteration fast।
    • 
          
  • Lineage UI — built-in।
    • 
          
  • GPU resource declarative।
    • 
          
  • Vertex Pipelines portable।
    • 
        

        
Airflow advantages:

        
    
          
  • Mature, battle-tested।
    • 
          
  • Operator library vast (Snowflake, Slack, Hadoop)।
    • 
          
  • Smaller K8s requirement।
    • 
          
  • Wider community।
    • 
          
  • Non-ML use cases-ও cover।
    • 
        

        
Decision matrix:

        
    
          
  • K8s shop, ML-only → KFP।
    • 
          
  • Mixed ETL+ML, broader ops → Airflow।
    • 
          
  • GCP shop → Vertex Pipelines (KFP under the hood)।
    • 
          
  • Both possible — Airflow scheduler triggers KFP runs।
    • 
        

        
Bangladesh context:

        
    
          
  • Most BD ML teams Airflow — already use for ETL।
    • 
          
  • Pure-ML platform team adopting KFP — clean separation।
    • 
          
  • Cloud-native (GCP) shops Vertex Pipelines fast adoption।
    • 
        

        
মূল উপলব্ধি: Airflow general; KFP ML-specific। K8s-and-ML-only → KFP cleaner। Otherwise Airflow। Both can coexist।

      

    


    

      প্র ০২"Caching when valuable, when harmful?"
      

        
Caching speed boost; কিন্তু wrong-cache silent bug।

        
When valuable:

        
    
          
  • Idempotent components, deterministic input।
    • 
          
  • Dev iteration — change last component, earlier cached।
    • 
          
  • Expensive components (large data preprocess)।
    • 
        

        
When harmful:

        
    
          
  • External state-dependent component (e.g., reads "today's S3 data")।
    • 
          
  • Random behavior in component — cached randomness misleading।
    • 
          
  • Bug fix — cached result still old version।
    • 
        

        
Cache key design:

        
    
          
  • Include execution date as parameter — daily refresh।
    • 
          
  • Include image SHA — image rebuild → cache invalidate।
    • 
          
  • Include code version (git SHA via env var)।
    • 
        

        
Practice:

        
    
          
  • Default enabled in dev।
    • 
          
  • Disable in production — explicit decision।
    • 
          
  • Or: enable in production with strict cache key (date + image SHA)।
    • 
        

        
মূল উপলব্ধি: Caching = dev productivity tool, production correctness risk। Discipline cache key carefully। Production-এ caching default-off safer।

      

    


    

      প্র ০৩"Component reuse — কীভাবে effectively?"
      

        
Component reuse — KFP-এর strength, কিন্তু practical challenge।

        
Patterns:

        
    
          
  • Generic components (e.g., "evaluate-classification") — reusable across projects।
    • 
          
  • Versioned component library — internal git repo।
    • 
          
  • Parametrize: same train code, different model class।
    • 
        

        
Anti-patterns:

        
    
          
  • "God component" doing everything — not reusable।
    • 
          
  • Hardcoded paths — non-portable।
    • 
          
  • Ambiguous schema — hard to plug in।
    • 
        

        
Versioning strategy:

        
    
          
  • Component image SHA tag।
    • 
          
  • Pipeline pin specific component version।
    • 
          
  • Backwards-compatible interface।
    • 
        

        
মূল উপলব্ধি: Reuse — design discipline + version management। Without — chaos quickly।

      

    


    

      প্র ০৪"Failed pipeline run debug — KFP-এর সাথে workflow?"
      

        
KFP debug — UI + kubectl combination।

        
UI level:

        
    
          
  • Pipeline graph — failed component highlighted।
    • 
          
  • Click → logs।
    • 
          
  • Inputs/outputs visible।
    • 
        

        
kubectl level:

        
    
          
  • kubectl describe pod — events।
    • 
          
  • kubectl logs — full output।
    • 
          
  • kubectl exec — interactive (if pod still running)।
    • 
        

        
Common failures:

        
    
          
  • OOM kill — increase memory_limit।
    • 
          
  • Image pull error — registry auth।
    • 
          
  • GPU unavailable — node selector mismatch।
    • 
          
  • Volume mount — artifact path issue।
    • 
        

        
Re-run from failed:

        
    
          
  • UI "rerun" — full pipeline।
    • 
          
  • "Retry from failed" — only failed onwards (caching helps)।
    • 
        

        
মূল উপলব্ধি: KFP debug = pipeline UI + K8s tools। Caching speeds re-run iteration। Logs structured retain।

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Compile: উপরের pipeline compile করুন। YAML inspect করুন।
        
    
          
          
    
            
    compiler.Compiler().compile(training_pipeline, "pipeline.yaml")। YAML-এ Argo Workflow spec দেখা যাবে — KFP underlying Argo এ compile।
    
          
    
        
    
      
    2. 
      
  2. 
        Local KFP: Minikube/kind-এ KFP install। Pipeline upload + run।
        
    
          
          
    
            
    Kubeflow standalone install instructions follow। UI port-forward। YAML upload, run-এ আধা ঘণ্টা।
    
          
    
        
    
      
    3. 
      
  3. 
        চিন্তা: ৪-component KFP pipeline — Bangla NLP-এর জন্য। Component reuse-এ কী potential?
        
    
          
          
    
            
      
              
    • tokenize-bangla — reusable across NLP projects।
      • 
              
    • train-classifier — generic, parameterize model class।
      • 
              
    • evaluate-multi-class — fairness gates parameterized।
      • 
              
    • register-mlflow — generic registry push।
      • 
            
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    
  







  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৩ · Training orchestration