Weights & Biases
এই পাঠে যা শিখবেন
- W&B-এর core API —
wandb.init,wandb.log,wandb.config - Sweeps — হাজার hyperparameter combination automate
- Artifacts — data ও model lineage
- MLflow-এর সাথে practical comparison
১ · কেন W&B
Weights & Biases (W&B) ২০১৭-এ launched। MLflow-এর প্রায় সব functionality + একটু polished UX + integrated hyperparameter search। বেশিরভাগ ML researcher (especially academic + DL) W&B-এর adopt করেছেন।
২ · Core API
import wandb
# ১. init — run শুরু
wandb.init(
project="bangla-sentiment",
name="rf-baseline",
config={
"n_estimators": 100,
"max_depth": 8,
"data_version": "v3",
},
tags=["nlp", "baseline"],
)
config = wandb.config # auto-managed config object
# ২. training (any framework)
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(
n_estimators=config.n_estimators,
max_depth=config.max_depth,
random_state=42,
)
model.fit(X_train, y_train)
# ৩. log metrics — incremental, real-time
auc = roc_auc_score(y_test, model.predict_proba(X_test)[:, 1])
wandb.log({"auc": auc, "n_train": len(X_train)})
# ৪. log model as artifact
import joblib
joblib.dump(model, "model.joblib")
artifact = wandb.Artifact("rf-model", type="model")
artifact.add_file("model.joblib")
wandb.log_artifact(artifact)
# ৫. finish
wandb.finish()৩ · Sweeps — hyperparameter search
"১০০ hyperparameter combination test করতে হবে" — manually কঠিন। W&B Sweeps coordinator হিসেবে কাজ করে — agent-গুলো যা যা compute করতে পারে।
# sweep.yaml
program: train.py
method: bayes # bayes | grid | random
metric:
name: val_auc
goal: maximize
parameters:
n_estimators:
values: [50, 100, 200, 500]
max_depth:
distribution: int_uniform
min: 3
max: 16
learning_rate:
distribution: log_uniform_values
min: 0.001
max: 0.3
# launch
# $ wandb sweep sweep.yaml
# $ wandb agent {sweep-id} # run on each machine
# train.py-এ
import wandb
wandb.init()
config = wandb.config
# ...train with config.n_estimators etc...
wandb.log({"val_auc": auc})method: bayes — agent best param explore করে। ১০০-৫০০ runs-এ optimal find করা typical। Multiple machine-এ একসাথে agent চালান — parallel exploration।
৪ · Artifacts — data ও model lineage
W&B Artifact = versioned file collection (dataset, preprocessed features, model)। Run-এর input/output হিসেবে track।
- Dataset artifact: raw data version-controlled।
- Preprocessed: raw → preprocessed pipeline traced।
- Model: training run-এর output, deployment-এর input।
- Lineage graph: auto-built — visual data flow।
৫ · Reports — collaborative documents
- Markdown + live W&B charts in one document।
- "Q1 model improvements" report — Slack-এ share, team review।
- Stakeholder-friendly — non-DS-ও পড়তে পারে।
- MLflow-এ direct equivalent নেই (Notebook-এ embedded chart approximation)।
৬ · Self-hosted W&B (on-prem)
W&B SaaS default। কিন্তু enterprise plan-এ on-prem option:
- Bangladesh banking compliance — data on-prem mandate। On-prem W&B costly ($enterprise license)।
- Most BD startups → SaaS sufficient।
- Hybrid possible: dev SaaS, production MLflow on-prem।
৭ · MLflow-এর সাথে comparison table
- UI polish: W&B better UX (clean, real-time)।
- Self-hosted: MLflow easy free; W&B enterprise-only।
- Hyperparameter sweeps: W&B integrated; MLflow + Optuna combo।
- Reports: W&B unique।
- Model registry: MLflow first-class; W&B has it but less central।
- Cost: MLflow $0 OSS; W&B free for individual, paid for team।
- Framework support: দুটো-ই broad — sklearn, PyTorch, TF, JAX, etc.।
ভাবনার প্রশ্ন
প্র ০১ "W&B-এর free tier (individual) — student/research-এর জন্য কতটা যথেষ্ট?"
W&B free tier — 100 GB storage, individual + public projects unlimited (private projects limited). Bangladesh student/researcher-এর জন্য excellent।
Free-এ available:
- Unlimited public project + tracking।
- Sweeps, artifacts, reports — সব functionally।
- ৩ collaborator-এর সাথে private project।
- Basic API + Python SDK।
Free-তে limit:
- 100 GB cumulative storage — large checkpoints quick পরিশেষ।
- SSO, audit log, compliance certifications — paid only।
- Service-level support নেই।
Storage tip — minimize:
- Large checkpoints — best-only save।
- Old artifacts cleanup periodic।
- Image/video artifacts — sample save, full না।
Bangladesh academic context:
- BUET, NSU, IUT, BRAC University-এর research lab — W&B-এ many runs।
- Paper publication-এর সাথে public reports — credibility +।
- Free tier sufficient for thesis/research।
Educational discounts:
- W&B-এর "Academic" plan — free for verified university users (full team features)।
- Apply with .edu email or institutional verification।
- Research project public — extra benefit।
Practical workflow:
- Personal project: free individual tier।
- Lab project: academic team plan (free for verified)।
- Industry: paid team plan needed।
Privacy consideration:
- Public project = anyone can see। Sensitive research data caution।
- Private project under academic plan: ভাল compromise।
- Industrial/proprietary work — paid private only।
মূল উপলব্ধি: W&B-এর free tier overall extremely generous। Bangladesh student-এর জন্য — sign up ASAP, learning দ্রুত accelerate। Academic plan apply করলে full features।
প্র ০২ "Sweep — grid vs random vs Bayes — কোনটা কখন?"
Hyperparameter search method-এর choice — search space size + budget-এর function।
Grid search:
- সব combination exhaustive।
- Pros: deterministic, complete, simple।
- Cons: dimensions বাড়লে exponential blowup।
- Use case: ৩-৪ params, ৩-৫ values each, < ১০০ combinations।
Random search:
- Random sample N times।
- Pros: Bergstra & Bengio (২০১২) showed — usually beats grid in same budget। কারণ — useless params-এ grid waste; random spread।
- Cons: no learning from past trials।
- Use case: ৫-১০ params, fixed budget (১০০-১০০০ trials)।
Bayesian optimization:
- Past trials থেকে শেখে; promising regions explore।
- Pros: best-of-3, especially mid-budget (50-500 trials)।
- Cons: serial-by-default (next trial depends on previous); parallelization tricky (W&B handles)। Cold start poor।
- Use case: expensive trials (training expensive), reasonable budget।
Other methods (W&B supports some):
- Hyperband / BOHB: early stopping + Bayesian — more efficient।
- Population-based training (PBT): evolve hyperparameters during training।
- Optuna: separate library; W&B integration possible।
Decision tree:
- < ২০ combinations, fast trial → grid।
- < ১০০ combinations, fast trial → random।
- Trial expensive (> ১৫ মিনিট), budget ৫০-৫০০ → Bayes।
- Very expensive trials (large LM), small budget → Hyperband।
- Online, evolving → PBT।
Common mistake — too-broad search space:
- "learning_rate: 0.0001 to 1.0" — মূলত $৪$ orders of magnitude। Bayes এমন wide space-এ poor।
- Solution: log-uniform with sensible bounds।
- "Two-stage" search: wide random first → narrow Bayes।
Bangladesh context cost note:
- GPU instance expensive। Random search 50 trials ≈ Bayes 30 trials, often comparable result।
- "Always Bayes" anti-pattern — overhead worth wide searches small budget-এ না।
মূল উপলব্ধি: Method-এর importance অনেকে exaggerate। Best ROI usually — better search space definition + sensible bounds + medium budget। Grid for tiny, random for moderate, Bayes for expensive — practical rule of thumb।
প্র ০৩ "W&B-এ data privacy concern" — Bangladesh financial/health sector-এ W&B SaaS use করা যায়?
Privacy + regulatory compliance critical Bangladesh's banking, fintech, healthcare-এ। W&B SaaS এ careful caveat থাকে।
SaaS data flow:
- Training script → wandb.log → W&B servers (US default, EU optional)।
- Metrics, params, model artifacts — সব cloud-এ store।
- Training data sample if logged (e.g., wandb.Image) — cloud-এ।
What's typically OK:
- Aggregate metrics (accuracy, loss) — usually no privacy concern।
- Hyperparameters — no PII।
- Anonymized training summaries।
What's risky:
- Raw training data samples (NID images, customer records) — DON'T log।
- Model artifact-এ training data leak (e.g., embedding-এ memorized)।
- Confidence-related data (predictions on real users)।
Bangladesh regulatory considerations:
- Bangladesh Bank ICT guideline — financial data domestic।
- Personal Data Protection Act (২০২৩ draft) — cross-border data transfer restrictions।
- Health data — Bangladesh Health Information Act considerations।
Practical patterns for sensitive sectors:
Pattern ১ — Local MLflow only:
- সব tracking on-prem MLflow।
- No SaaS dependency।
- Slower UX, but compliance clean।
Pattern ২ — W&B for non-sensitive only:
- Public/synthetic data experiments → W&B SaaS।
- Real customer data → on-prem MLflow।
- Data classification at training time।
Pattern ৩ — W&B Server (self-hosted):
- Enterprise license — on-prem deploy।
- Full UX, but cost ($enterprise + ops)।
- Bangladesh-এ rare due to cost।
Pattern ৪ — Sanitization before logging:
- Custom logger that strips PII, hashes IDs।
- Aggregate-level metrics only।
- Code review-এ enforced।
Audit trail consideration:
- Regulator audit-এ "tracking system কোথায়, data কোথায়" প্রশ্ন।
- SaaS = additional audit complexity।
- On-prem = simpler narrative।
মূল উপলব্ধি: Sensitive Bangladesh sector-এ — W&B SaaS-এ raw training data DON'T log। Aggregate metrics OK usually। On-prem MLflow safer default; W&B SaaS for less-sensitive R&D। Combination most practical।
প্র ০৪ "Reports — কখন worth, কখন paperwork?"
Reports — W&B's unique feature। Powerful কিন্তু overhead-ও আছে।
Worth when:
- Stakeholder communication: business team-কে result explain — chart + narrative।
- Onboarding new team member: "এই project-এর গত ৬ মাসের journey"।
- Decision documentation: "এই hyperparameter choose কেন" — future reference।
- Cross-team review: ML team's work outside-এর জন্য digestible।
- Compliance audit: "এই model এই decision-গুলো এই data-এ" — regulator-friendly।
Paperwork when:
- Solo developer: writing for self — overhead high, value low।
- Rapid iteration phase: daily change — report obsolete in days।
- Internal-only metric tracking: dashboard sufficient।
- Trivial experiments: "tried lr=0.01 instead of 0.001" — too small।
Sweet spot use cases:
- Quarterly summary "what we shipped this quarter, with metrics"।
- Key decision rationale "Why we chose model X over Y"।
- Public-facing research blog (companies' tech blog ←)।
- Customer-facing model documentation।
Practical structure:
- Goal section — "what problem"।
- Method — "what we tried"।
- Results — embedded charts (live, not screenshot)।
- Decision — "what we'll do next"।
- References — Run links, dataset versions।
"Living document" anti-pattern:
- Report keep updating forever — context drift।
- Better: snapshot at decision points, link forward।
MLflow alternative — Notebook:
- Jupyter notebook embedded MLflow charts — similar effect।
- Clunkier UX but git-trackable।
Bangladesh team practical advice:
- Start with quarterly retrospective report — habit-form।
- Internal-only first; external (blog) later।
- Bangla-language summary helps non-tech stakeholder।
মূল উপলব্ধি: Reports — small communicate burden, large knowledge dividend if used at right inflection points। "Every experiment a report" — paperwork। "Quarterly summary, key decisions" — gold। Right cadence-এ written reports — team's institutional memory।
অনুশীলন
-
Setup: W&B account তৈরি করুন (free)। একটি simple sklearn run track করুন।
pip install wandb→wandb login→wandb.init(project="hello")→ log a few metrics → finish। Browser-এ run দেখুন। -
Sweep: উপরের sweep.yaml দিয়ে একটি ৩০-trial Bayes search চালান। Best run কোনটি?
wandb sweep sweep.yaml→ sweep id পাবেন।wandb agent <sweep-id>run করুন। ৩০ trial-এর পর UI-এ "Sort by val_auc DESC" — top run দেখুন। Parallel coordinates plot impressive। -
চিন্তা: আপনি একটি Bangladeshi fintech-এ MLOps lead। MLflow আছে, কিন্তু DS team W&B-এ যেতে চায়। কী decision criteria + transition plan?
- Criteria: data privacy (financial), cost, current MLflow ROI, sweep need।
- Transition: hybrid — research W&B (synthetic data), production registry MLflow।
- Cost-benefit: W&B paid plan vs MLflow infra burden compare।
- Compliance review with InfoSec team।