পাঠ ১৪ · ৩৩-এর মধ্যে · মডিউল ২
Home / AI Courses / MLOps / Kubeflow Pipelines

Kubeflow Pipelines

KFP — K8s-native ML orchestrator
৮ মিনিট পড়া উচ্চ · Advanced Python DSL

এই পাঠে যা শিখবেন

  • KFP-এর core concept — component, pipeline, artifact
  • Python DSL দিয়ে pipeline define
  • Caching ও artifact lineage
  • Airflow vs KFP — কখন কোনটা

১ · কেন KFP, কেন আলাদা

Airflow general-purpose orchestrator. KFP ML-specific — K8s-native, container-per-step, artifact tracking built-in.

  • প্রতিটি step একটি container — strict isolation.
  • Artifact (data, model) automatic typed pass between components.
  • Caching automatic — same input → cached output.
  • UI-তে DAG + lineage graph visualization.

২ · Component

Component = containerized step. Three styles:

  • Lightweight Python component: function-based, KFP wraps in container.
  • Container component: existing Docker image-এ command + args.
  • Reusable component: YAML spec, share across pipelines.

৩ · Python DSL pipeline

Python · KFP v2 pipeline 
from kfp import dsl, compiler

@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["pandas", "scikit-learn"],
)
def fetch_data(date: str, output_csv: dsl.Output[dsl.Dataset]):
    import pandas as pd
    # download / generate data
    df = pd.DataFrame(...)
    df.to_csv(output_csv.path, index=False)


@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["pandas", "scikit-learn", "joblib"],
)
def train_model(
    input_csv: dsl.Input[dsl.Dataset],
    output_model: dsl.Output[dsl.Model],
    n_estimators: int = 100,
):
    import pandas as pd, joblib
    from sklearn.ensemble import RandomForestClassifier
    df = pd.read_csv(input_csv.path)
    X, y = df.drop(columns=["label"]), df["label"]
    model = RandomForestClassifier(n_estimators=n_estimators, random_state=42)
    model.fit(X, y)
    joblib.dump(model, output_model.path)


@dsl.component(base_image="python:3.11-slim", packages_to_install=["scikit-learn", "joblib"])
def evaluate(
    input_model: dsl.Input[dsl.Model],
    input_csv: dsl.Input[dsl.Dataset],
    metrics: dsl.Output[dsl.Metrics],
):
    import pandas as pd, joblib
    from sklearn.metrics import roc_auc_score
    model = joblib.load(input_model.path)
    df = pd.read_csv(input_csv.path)
    auc = roc_auc_score(df["label"], model.predict_proba(df.drop(columns=["label"]))[:, 1])
    metrics.log_metric("auc", auc)


@dsl.pipeline(name="bangla-sentiment-train")
def training_pipeline(date: str = "2025-01-15", n_estimators: int = 100):
    data_op = fetch_data(date=date)
    train_op = train_model(input_csv=data_op.outputs["output_csv"], n_estimators=n_estimators)
    eval_op = evaluate(
        input_model=train_op.outputs["output_model"],
        input_csv=data_op.outputs["output_csv"],
    )


# Compile + submit
compiler.Compiler().compile(training_pipeline, "pipeline.yaml")
# UI দিয়ে upload, OR programmatically:
# client = kfp.Client(host="...")
# client.create_run_from_pipeline_func(training_pipeline, arguments={...})

    


    

      Compiled pipeline.yaml → KFP UI বা Vertex Pipelines-এ submit. Artifacts (Dataset, Model, Metrics) — typed, automatically tracked.
    


    
৪ · Caching


    

      KFP automatically caches component results. Same component + same inputs → reuse previous output.
    

    
    
      
  • Cache key: component spec hash + input hash.
    • 
      
  • Cache hit → no rerun, instant.
    • 
      
  • Disable per-component: set_caching_options(enable_caching=False).
    • 
      
  • Useful when developing — change last component only, earlier cached.
    • 
    


    
৫ · GPU support


    

      

        
        Python · GPU component
      

      
from kfp import dsl

@dsl.pipeline(name="gpu-train")
def gpu_pipeline():
    train_op = train_model(...)
    train_op.set_accelerator_type("nvidia.com/gpu")
    train_op.set_accelerator_limit(1)
    train_op.set_memory_limit("16Gi")
    train_op.set_cpu_limit("4")

      

        
        
      

      

    


    

      KFP K8s GPU node-এ schedule করে। Resource requests + limits ML-friendly.
    


    
৬ · Vertex Pipelines compatibility


    

      Same KFP DSL pipeline.yaml — Vertex AI Pipelines (GCP)-এও submit করা যায়। Portable: dev local KFP, prod Vertex Pipelines.
    


    
৭ · Airflow vs KFP


    
    
      
  • Airflow:
        
      
          
    • General-purpose, broader operator library.
      • 
          
    • Dynamic DAG limited.
      • 
          
    • Heavier control plane.
      • 
        
    
      
    • 
      
  • KFP:
        
      
          
    • ML-specific, container-per-step.
      • 
          
    • Type-checked artifact passing.
      • 
          
    • K8s-native — needs K8s.
      • 
        
    
      
    • 
      
  • Common pattern:
        
      
          
    • Airflow for ETL + scheduled ML retraining trigger.
      • 
          
    • KFP for actual ML pipeline execution.
      • 
          
    • Airflow trigger-করে KFP run.
      • 
        
    
      
    • 
    


    

      
        
          
            
          
        

        Kubeflow Pipelines — container-per-step on K8s
        Typed artifacts auto-passed

        
        Component 1
        fetch_data
        → Dataset

        
        Component 2
        train_model
        GPU pod
        → Model

        
        Component 3
        evaluate
        → Metrics

        
        

        
        
        Artifact store (S3/GCS) — typed, lineage tracked
        Dataset, Model, Metrics — auto persisted, auto mounted

        
        
        
      
      
প্রতিটি component আলাদা K8s pod-এ চালু। Artifact (Dataset, Model, Metrics) auto-persist artifact store-এ; পরের component automatic mount.

    


    

      Bangladesh-এ KFP adoption Airflow-এর চেয়ে কম, কিন্তু GPU-heavy ML team-এ growing. Vertex Pipelines (GCP Mumbai) — managed, no K8s ops.
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন


    

      প্র ০১"KFP vs Airflow — ML-এর জন্য কোনটা better?"
      

        
"Better" depends — both excel different aspects.

        
KFP advantages:

        
    
          
  • Container-per-step — strict isolation, no dependency hell.
    • 
          
  • Typed artifacts — Dataset, Model, Metrics built-in.
    • 
          
  • Auto caching — dev iteration fast.
    • 
          
  • Lineage UI — built-in.
    • 
          
  • GPU resource declarative.
    • 
          
  • Vertex Pipelines portable.
    • 
        

        
Airflow advantages:

        
    
          
  • Mature, battle-tested.
    • 
          
  • Operator library vast (Snowflake, Slack, Hadoop).
    • 
          
  • Smaller K8s requirement.
    • 
          
  • Wider community.
    • 
          
  • Non-ML use cases-ও cover.
    • 
        

        
Decision matrix:

        
    
          
  • K8s shop, ML-only → KFP.
    • 
          
  • Mixed ETL+ML, broader ops → Airflow.
    • 
          
  • GCP shop → Vertex Pipelines (KFP under the hood).
    • 
          
  • Both possible — Airflow scheduler triggers KFP runs.
    • 
        

        
Bangladesh context:

        
    
          
  • Most BD ML teams Airflow — already use for ETL.
    • 
          
  • Pure-ML platform team adopting KFP — clean separation.
    • 
          
  • Cloud-native (GCP) shops Vertex Pipelines fast adoption.
    • 
        

        
মূল উপলব্ধি: Airflow general; KFP ML-specific. K8s-and-ML-only → KFP cleaner. Otherwise Airflow. Both can coexist.

      

    


    

      প্র ০২"Caching when valuable, when harmful?"
      

        
Caching speed boost; কিন্তু wrong-cache silent bug.

        
When valuable:

        
    
          
  • Idempotent components, deterministic input.
    • 
          
  • Dev iteration — change last component, earlier cached.
    • 
          
  • Expensive components (large data preprocess).
    • 
        

        
When harmful:

        
    
          
  • External state-dependent component (e.g., reads "today's S3 data").
    • 
          
  • Random behavior in component — cached randomness misleading.
    • 
          
  • Bug fix — cached result still old version.
    • 
        

        
Cache key design:

        
    
          
  • Include execution date as parameter — daily refresh.
    • 
          
  • Include image SHA — image rebuild → cache invalidate.
    • 
          
  • Include code version (git SHA via env var).
    • 
        

        
Practice:

        
    
          
  • Default enabled in dev.
    • 
          
  • Disable in production — explicit decision.
    • 
          
  • Or: enable in production with strict cache key (date + image SHA).
    • 
        

        
মূল উপলব্ধি: Caching = dev productivity tool, production correctness risk. Discipline cache key carefully. Production-এ caching default-off safer.

      

    


    

      প্র ০৩"Component reuse — কীভাবে effectively?"
      

        
Component reuse — KFP-এর strength, কিন্তু practical challenge.

        
Patterns:

        
    
          
  • Generic components (e.g., "evaluate-classification") — reusable across projects.
    • 
          
  • Versioned component library — internal git repo.
    • 
          
  • Parametrize: same train code, different model class.
    • 
        

        
Anti-patterns:

        
    
          
  • "God component" doing everything — not reusable.
    • 
          
  • Hardcoded paths — non-portable.
    • 
          
  • Ambiguous schema — hard to plug in.
    • 
        

        
Versioning strategy:

        
    
          
  • Component image SHA tag.
    • 
          
  • Pipeline pin specific component version.
    • 
          
  • Backwards-compatible interface.
    • 
        

        
মূল উপলব্ধি: Reuse — design discipline + version management. Without — chaos quickly.

      

    


    

      প্র ০৪"Failed pipeline run debug — KFP-এর সাথে workflow?"
      

        
KFP debug — UI + kubectl combination.

        
UI level:

        
    
          
  • Pipeline graph — failed component highlighted.
    • 
          
  • Click → logs.
    • 
          
  • Inputs/outputs visible.
    • 
        

        
kubectl level:

        
    
          
  • kubectl describe pod — events.
    • 
          
  • kubectl logs — full output.
    • 
          
  • kubectl exec — interactive (if pod still running).
    • 
        

        
Common failures:

        
    
          
  • OOM kill — increase memory_limit.
    • 
          
  • Image pull error — registry auth.
    • 
          
  • GPU unavailable — node selector mismatch.
    • 
          
  • Volume mount — artifact path issue.
    • 
        

        
Re-run from failed:

        
    
          
  • UI "rerun" — full pipeline.
    • 
          
  • "Retry from failed" — only failed onwards (caching helps).
    • 
        

        
মূল উপলব্ধি: KFP debug = pipeline UI + K8s tools. Caching speeds re-run iteration. Logs structured retain.

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        Compile: উপরের pipeline compile করুন। YAML inspect করুন।
        
    
          
          
    
            
    compiler.Compiler().compile(training_pipeline, "pipeline.yaml"). YAML-এ Argo Workflow spec দেখা যাবে — KFP underlying Argo এ compile.
    
          
    
        
    
      
    2. 
      
  2. 
        Local KFP: Minikube/kind-এ KFP install. Pipeline upload + run.
        
    
          
          
    
            
    Kubeflow standalone install instructions follow. UI port-forward. YAML upload, run-এ আধা ঘণ্টা।
    
          
    
        
    
      
    3. 
      
  3. 
        চিন্তা: ৪-component KFP pipeline — Bangla NLP-এর জন্য। Component reuse-এ কী potential?
        
    
          
          
    
            
      
              
    • tokenize-bangla — reusable across NLP projects.
      • 
              
    • train-classifier — generic, parameterize model class.
      • 
              
    • evaluate-multi-class — fairness gates parameterized.
      • 
              
    • register-mlflow — generic registry push.
      • 
            
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    
  







  
  
     এই পাঠে ভুল পেয়েছেন? জানান
  





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৩ · Training orchestration