পাঠ ৯ · ৩৩-এর মধ্যে · মডিউল ২

Home / AI Courses / MLOps / Weights & Biases

Weights & Biases

W&B — tracking, sweeps, artifacts, reports

৭ মিনিট পড়া মধ্য · Intermediate Python

এই পাঠে যা শিখবেন

W&B-এর core API — wandb.init, wandb.log, wandb.config
Sweeps — হাজার hyperparameter combination automate
Artifacts — data ও model lineage
MLflow-এর সাথে practical comparison

১ · কেন W&B

Weights & Biases (W&B) ২০১৭-এ launched. MLflow-এর প্রায় সব functionality + একটু polished UX + integrated hyperparameter search. বেশিরভাগ ML researcher (especially academic + DL) W&B-এর adopt করেছেন।

২ · Core API

Python · W&B basic

import wandb

# ১. init — run শুরু
wandb.init(
    project="bangla-sentiment",
    name="rf-baseline",
    config={
        "n_estimators": 100,
        "max_depth": 8,
        "data_version": "v3",
    },
    tags=["nlp", "baseline"],
)

config = wandb.config  # auto-managed config object

# ২. training (any framework)
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(
    n_estimators=config.n_estimators,
    max_depth=config.max_depth,
    random_state=42,
)
model.fit(X_train, y_train)

# ৩. log metrics — incremental, real-time
auc = roc_auc_score(y_test, model.predict_proba(X_test)[:, 1])
wandb.log({"auc": auc, "n_train": len(X_train)})

# ৪. log model as artifact
import joblib
joblib.dump(model, "model.joblib")
artifact = wandb.Artifact("rf-model", type="model")
artifact.add_file("model.joblib")
wandb.log_artifact(artifact)

# ৫. finish
wandb.finish()

W&B run-এর URL terminal-এ দেখা যাবে। Browser-এ live chart update — even training চলাকালীন।

৩ · Sweeps — hyperparameter search

"১০০ hyperparameter combination test করতে হবে" — manually কঠিন। W&B Sweeps coordinator হিসেবে কাজ করে — agent-গুলো যা যা compute করতে পারে।

YAML + Python · Sweep

# sweep.yaml
program: train.py
method: bayes              # bayes | grid | random
metric:
  name: val_auc
  goal: maximize
parameters:
  n_estimators:
    values: [50, 100, 200, 500]
  max_depth:
    distribution: int_uniform
    min: 3
    max: 16
  learning_rate:
    distribution: log_uniform_values
    min: 0.001
    max: 0.3

# launch
# $ wandb sweep sweep.yaml
# $ wandb agent {sweep-id}   # run on each machine

# train.py-এ
import wandb
wandb.init()
config = wandb.config
# ...train with config.n_estimators etc...
wandb.log({"val_auc": auc})

method: bayes — agent best param explore করে। ১০০-৫০০ runs-এ optimal find করা typical. Multiple machine-এ একসাথে agent চালান — parallel exploration.

৪ · Artifacts — data ও model lineage

W&B Artifact = versioned file collection (dataset, preprocessed features, model). Run-এর input/output হিসেবে track.

Dataset artifact: raw data version-controlled.
Preprocessed: raw → preprocessed pipeline traced.
Model: training run-এর output, deployment-এর input.
Lineage graph: auto-built — visual data flow.

৫ · Reports — collaborative documents

Markdown + live W&B charts in one document.
"Q1 model improvements" report — Slack-এ share, team review.
Stakeholder-friendly — non-DS-ও পড়তে পারে।
MLflow-এ direct equivalent নেই (Notebook-এ embedded chart approximation).

৬ · Self-hosted W&B (on-prem)

W&B SaaS default. কিন্তু enterprise plan-এ on-prem option:

Bangladesh banking compliance — data on-prem mandate. On-prem W&B costly ($enterprise license).
Most BD startups → SaaS sufficient.
Hybrid possible: dev SaaS, production MLflow on-prem.

৭ · MLflow-এর সাথে comparison table

UI polish: W&B better UX (clean, real-time).
Self-hosted: MLflow easy free; W&B enterprise-only.
Hyperparameter sweeps: W&B integrated; MLflow + Optuna combo.
Reports: W&B unique.
Model registry: MLflow first-class; W&B has it but less central.
Cost: MLflow $0 OSS; W&B free for individual, paid for team.
Framework support: দুটো-ই broad — sklearn, PyTorch, TF, JAX, etc.।

Sweep coordinator (cloud) Bayesian optimizer চালায়; multiple agent (laptop, GPU node, spot instance) parallel-এ trial-গুলো execute করে। Result coordinator-এ ফেরত যায়, পরের params smart-এ choose হয়।

Bangladesh research team-এর জন্য (academic, DL-heavy) — W&B-এর free tier (individual) অসাধারণ value. Group plan paid কিন্তু feature ROI usually positive.

ভাবনার প্রশ্ন

প্র ০১ "W&B-এর free tier (individual) — student/research-এর জন্য কতটা যথেষ্ট?"

W&B free tier — 100 GB storage, individual + public projects unlimited (private projects limited). Bangladesh student/researcher-এর জন্য excellent.

Free-এ available:

Unlimited public project + tracking.
Sweeps, artifacts, reports — সব functionally.
৩ collaborator-এর সাথে private project.
Basic API + Python SDK.

Free-তে limit:

100 GB cumulative storage — large checkpoints quick পরিশেষ।
SSO, audit log, compliance certifications — paid only.
Service-level support নেই।

Storage tip — minimize:

Large checkpoints — best-only save.
Old artifacts cleanup periodic.
Image/video artifacts — sample save, full না।

Bangladesh academic context:

BUET, NSU, IUT, BRAC University-এর research lab — W&B-এ many runs.
Paper publication-এর সাথে public reports — credibility +.
Free tier sufficient for thesis/research.

Educational discounts:

W&B-এর "Academic" plan — free for verified university users (full team features).
Apply with .edu email or institutional verification.
Research project public — extra benefit.

Practical workflow:

Personal project: free individual tier.
Lab project: academic team plan (free for verified).
Industry: paid team plan needed.

Privacy consideration:

Public project = anyone can see. Sensitive research data caution.
Private project under academic plan: ভাল compromise.
Industrial/proprietary work — paid private only.

মূল উপলব্ধি: W&B-এর free tier overall extremely generous. Bangladesh student-এর জন্য — sign up ASAP, learning দ্রুত accelerate. Academic plan apply করলে full features.

প্র ০২ "Sweep — grid vs random vs Bayes — কোনটা কখন?"

Hyperparameter search method-এর choice — search space size + budget-এর function.

Grid search:

সব combination exhaustive.
Pros: deterministic, complete, simple.
Cons: dimensions বাড়লে exponential blowup.
Use case: ৩-৪ params, ৩-৫ values each, < ১০০ combinations.

Random search:

Random sample N times.
Pros: Bergstra & Bengio (২০১২) showed — usually beats grid in same budget. কারণ — useless params-এ grid waste; random spread.
Cons: no learning from past trials.
Use case: ৫-১০ params, fixed budget (১০০-১০০০ trials).

Bayesian optimization:

Past trials থেকে শেখে; promising regions explore.
Pros: best-of-3, especially mid-budget (50-500 trials).
Cons: serial-by-default (next trial depends on previous); parallelization tricky (W&B handles). Cold start poor.
Use case: expensive trials (training expensive), reasonable budget.

Other methods (W&B supports some):

Hyperband / BOHB: early stopping + Bayesian — more efficient.
Population-based training (PBT): evolve hyperparameters during training.
Optuna: separate library; W&B integration possible.

Decision tree:

< ২০ combinations, fast trial → grid.
< ১০০ combinations, fast trial → random.
Trial expensive (> ১৫ মিনিট), budget ৫০-৫০০ → Bayes.
Very expensive trials (large LM), small budget → Hyperband.
Online, evolving → PBT.

Common mistake — too-broad search space:

"learning_rate: 0.0001 to 1.0" — মূলত $৪$ orders of magnitude. Bayes এমন wide space-এ poor.
Solution: log-uniform with sensible bounds.
"Two-stage" search: wide random first → narrow Bayes.

Bangladesh context cost note:

GPU instance expensive. Random search 50 trials ≈ Bayes 30 trials, often comparable result.
"Always Bayes" anti-pattern — overhead worth wide searches small budget-এ না।

মূল উপলব্ধি: Method-এর importance অনেকে exaggerate. Best ROI usually — better search space definition + sensible bounds + medium budget. Grid for tiny, random for moderate, Bayes for expensive — practical rule of thumb.

প্র ০৩ "W&B-এ data privacy concern" — Bangladesh financial/health sector-এ W&B SaaS use করা যায়?

Privacy + regulatory compliance critical Bangladesh's banking, fintech, healthcare-এ। W&B SaaS এ careful caveat থাকে।

SaaS data flow:

Training script → wandb.log → W&B servers (US default, EU optional).
Metrics, params, model artifacts — সব cloud-এ store.
Training data sample if logged (e.g., wandb.Image) — cloud-এ।

What's typically OK:

Aggregate metrics (accuracy, loss) — usually no privacy concern.
Hyperparameters — no PII.
Anonymized training summaries.

What's risky:

Raw training data samples (NID images, customer records) — DON'T log.
Model artifact-এ training data leak (e.g., embedding-এ memorized).
Confidence-related data (predictions on real users).

Bangladesh regulatory considerations:

Bangladesh Bank ICT guideline — financial data domestic.
Personal Data Protection Act (২০২৩ draft) — cross-border data transfer restrictions.
Health data — Bangladesh Health Information Act considerations.

Practical patterns for sensitive sectors:

Pattern ১ — Local MLflow only:

সব tracking on-prem MLflow.
No SaaS dependency.
Slower UX, but compliance clean.

Pattern ২ — W&B for non-sensitive only:

Public/synthetic data experiments → W&B SaaS.
Real customer data → on-prem MLflow.
Data classification at training time.

Pattern ৩ — W&B Server (self-hosted):

Enterprise license — on-prem deploy.
Full UX, but cost ($enterprise + ops).
Bangladesh-এ rare due to cost.

Pattern ৪ — Sanitization before logging:

Custom logger that strips PII, hashes IDs.
Aggregate-level metrics only.
Code review-এ enforced.

Audit trail consideration:

Regulator audit-এ "tracking system কোথায়, data কোথায়" প্রশ্ন।
SaaS = additional audit complexity.
On-prem = simpler narrative.

মূল উপলব্ধি: Sensitive Bangladesh sector-এ — W&B SaaS-এ raw training data DON'T log. Aggregate metrics OK usually. On-prem MLflow safer default; W&B SaaS for less-sensitive R&D. Combination most practical.

প্র ০৪ "Reports — কখন worth, কখন paperwork?"

Reports — W&B's unique feature. Powerful কিন্তু overhead-ও আছে।

Worth when:

Stakeholder communication: business team-কে result explain — chart + narrative.
Onboarding new team member: "এই project-এর গত ৬ মাসের journey".
Decision documentation: "এই hyperparameter choose কেন" — future reference.
Cross-team review: ML team's work outside-এর জন্য digestible.
Compliance audit: "এই model এই decision-গুলো এই data-এ" — regulator-friendly.

Paperwork when:

Solo developer: writing for self — overhead high, value low.
Rapid iteration phase: daily change — report obsolete in days.
Internal-only metric tracking: dashboard sufficient.
Trivial experiments: "tried lr=0.01 instead of 0.001" — too small.

Sweet spot use cases:

Quarterly summary "what we shipped this quarter, with metrics".
Key decision rationale "Why we chose model X over Y".
Public-facing research blog (companies' tech blog ←)।
Customer-facing model documentation.

Practical structure:

Goal section — "what problem".
Method — "what we tried".
Results — embedded charts (live, not screenshot).
Decision — "what we'll do next".
References — Run links, dataset versions.

"Living document" anti-pattern:

Report keep updating forever — context drift.
Better: snapshot at decision points, link forward.

MLflow alternative — Notebook:

Jupyter notebook embedded MLflow charts — similar effect.
Clunkier UX but git-trackable.

Bangladesh team practical advice:

Start with quarterly retrospective report — habit-form.
Internal-only first; external (blog) later.
Bangla-language summary helps non-tech stakeholder.

মূল উপলব্ধি: Reports — small communicate burden, large knowledge dividend if used at right inflection points. "Every experiment a report" — paperwork. "Quarterly summary, key decisions" — gold. Right cadence-এ written reports — team's institutional memory.

অনুশীলন

Setup: W&B account তৈরি করুন (free). একটি simple sklearn run track করুন।

pip install wandb → wandb login → wandb.init(project="hello") → log a few metrics → finish. Browser-এ run দেখুন।
Sweep: উপরের sweep.yaml দিয়ে একটি ৩০-trial Bayes search চালান। Best run কোনটি?

wandb sweep sweep.yaml → sweep id পাবেন। wandb agent <sweep-id> run করুন। ৩০ trial-এর পর UI-এ "Sort by val_auc DESC" — top run দেখুন। Parallel coordinates plot impressive.
চিন্তা: আপনি একটি Bangladeshi fintech-এ MLOps lead. MLflow আছে, কিন্তু DS team W&B-এ যেতে চায়। কী decision criteria + transition plan?
- Criteria: data privacy (financial), cost, current MLflow ROI, sweep need.
- Transition: hybrid — research W&B (synthetic data), production registry MLflow.
- Cost-benefit: W&B paid plan vs MLflow infra burden compare.
- Compliance review with InfoSec team.