পাঠ ১২ · ৩৩-এর মধ্যে · মডিউল ২

Home / AI Courses / MLOps / Feature store

Feature store — Feast

Feature store — single source for features

৮ মিনিট পড়া উচ্চ · Advanced Concept + Python

এই পাঠে যা শিখবেন

Feature store-এর core concepts — entity, feature view, online/offline
Point-in-time correctness কী ও কেন critical
Feast-এ feature definition + materialize
কখন feature store ROI positive, কখন overkill

১ · কোন সমস্যা solve করে

Bangladesh-এর একটি e-commerce team দু'টো model বানায় — recommendation, fraud. দু'টোতেই একই feature: "user-এর গত ৩০ দিনের avg order value".

Recommendation team pandas-এ compute করেছে।
Fraud team Spark-এ compute করেছে।
৩ মাস পরে — recommendation training accuracy fine, production accuracy poor. Reason — production-এ feature subtly different.
Fraud team-এর code review-এ ভুল implementation পাওয়া গেল — যা months-এর data drift মাস্ক করেছিল।

Feature storeFeature Storecentralized service — feature definition + computation + serving. Training ও serving উভয়ের জন্য single source of truth. এই সমস্যা solve করে — feature definition একবার, ব্যবহার সবখানে।

২ · Architecture

Two-store architecture

Offline store = historical features for training (BigQuery, Snowflake, S3 Parquet). Throughput optimized.
Online store = latest features for serving (Redis, DynamoDB, Cassandra). Latency optimized (< 10ms).
Feature definition = single source — both stores generate from same logic.

৩ · Core concepts (Feast)

Entity: "user_id", "merchant_id" — what feature describes.
Data source: raw table — BigQuery, Parquet, Kafka.
Feature view: entity + features + source + freshness window.
Feature service: a bundle of feature views — application-specific.
Materialize: offline → online sync.

৪ · Feast feature definition

Python · Feast feature view

from feast import Entity, FeatureView, Field, FileSource
from feast.types import Float32, Int64
from datetime import timedelta

# ১. Entity define
user = Entity(name="user_id", join_keys=["user_id"])

# ২. Source — historical data
order_history_source = FileSource(
    path="s3://my-bucket/features/order_history.parquet",
    timestamp_field="event_timestamp",
)

# ৩. Feature view — definition
user_order_features = FeatureView(
    name="user_order_features",
    entities=[user],
    ttl=timedelta(days=30),
    schema=[
        Field(name="orders_30d", dtype=Int64),
        Field(name="avg_order_value_30d", dtype=Float32),
        Field(name="cart_abandon_rate_30d", dtype=Float32),
        Field(name="days_since_last_order", dtype=Int64),
    ],
    source=order_history_source,
    online=True,
)

Single feature view — both training pipeline (offline) ও serving API (online) এই same definition থেকে। Schema mismatch impossible.

৫ · Training: get historical features

Python · Training data

from feast import FeatureStore
import pandas as pd

store = FeatureStore(repo_path=".")

# Entity rows — labels with timestamps
entity_df = pd.DataFrame({
    "user_id": [101, 102, 103, ...],
    "event_timestamp": [
        "2025-01-01 10:30:00",
        "2025-01-02 14:00:00",
        ...,
    ],
    "label": [1, 0, 1, ...],  # purchased or not
})

# ⭐ Point-in-time join — "as of" feature value
training_df = store.get_historical_features(
    entity_df=entity_df,
    features=[
        "user_order_features:orders_30d",
        "user_order_features:avg_order_value_30d",
        "user_order_features:cart_abandon_rate_30d",
    ],
).to_df()

# Now training_df-এ each row-এর label-এর সময়ের feature value
X = training_df[feature_cols]
y = training_df["label"]
# train model...

⭐ Point-in-time join — Feast ensures user-এর "feature value as of 2025-01-01 10:30" — future leakage avoid. Manually এটা compute করা ভুলপ্রবণ।

৬ · Serving: get online features

Python · Serving inference

# Daily materialize — offline → online sync
# (এই command Airflow-এ scheduled)
$ feast materialize-incremental 2025-01-15T00:00:00

# In FastAPI serving:
from feast import FeatureStore
store = FeatureStore(repo_path=".")

@app.post("/predict")
def predict(req: Request):
    # Online lookup — milliseconds
    features = store.get_online_features(
        features=[
            "user_order_features:orders_30d",
            "user_order_features:avg_order_value_30d",
            "user_order_features:cart_abandon_rate_30d",
        ],
        entity_rows=[{"user_id": req.user_id}],
    ).to_dict()

    # model predict (same features as training)
    score = model.predict_proba([list(features.values())])[0, 1]
    return {"score": score}

Production serving Redis থেকে instant lookup. Training pipeline-এর সাথে identical feature definitions — skew impossible.

৭ · Point-in-time correctness

Concept: training-এ row-এর label time "as of"-এর feature value চাই; future feature data leak করতে দেবেন না।

Example: "User_101 কি 2025-01-15 10:00-এ purchase করল?" — এই decision-এ label time-এর আগে user-এর order history use. 11:00-এর data leak (যেমন purchase event itself) ভুল।

Manual SQL-এ এটি কঠিন; Feast এই join automatically করে।

৮ · কখন feature store ROI positive

৩+ ML team common features share করছে।
Same feature multiple model-এ — duplication problematic.
Online serving low-latency (<50ms) requirement.
Training-serving skew bug কয়েকবার ঘটেছে।
Feature engineering team আছে।

৯ · কখন overkill

একটি model, একটি team, simple features.
Batch-only inference.
Stack-এ already ETL exists, no skew problem yet.

Single feature definition — offline store training-এ, online store serving-এ। Materialize step দু'টোকে sync রাখে। Skew impossible by construction.

Bangladesh-এ feature store adoption এখনো early stage. bKash, Pathao-র মত top company-তে exists; smaller startups simple inhouse feature pipeline সাধারণত যথেষ্ট।

ভাবনার প্রশ্ন

প্র ০১ "Feature store ROI calculation" — Bangladesh team-এর জন্য concrete number.

Feature store substantial investment. Concrete ROI-এর জন্য numbers দেখি।

Cost (annualized for 5-engineer team):

Feast OSS — free; infrastructure (Redis, DB) — $২০০-৫০০/month = ~৩-৬ লাখ BDT/year.
Setup time — 2-3 engineer-month = ~৬-৯ লাখ BDT one-time.
Ongoing maintenance — 0.2 FTE = ~৫ লাখ BDT/year.
Year 1 total: ~১৫-২০ লাখ BDT.

Benefit (annualized):

Skew bug prevention: 2-3 incidents/year × ~৫ লাখ each (debugging + lost revenue) = ১০-১৫ লাখ saved.
Faster feature reuse: 5 features × 3 models × 1 week each = 15 engineer-week saved = ~৭.৫ লাখ।
Latency improvement: Redis 5ms vs custom 80ms — UX gain in conversion (hard to quantify but real).
Year 1 benefit: ~১৭-২৫ লাখ BDT.

Break-even: Year 1 marginal positive. Year 2+ benefits dominate as setup amortized.

Threshold rule of thumb:

৩+ models sharing features → likely positive ROI.
1 model only → don't invest.
Skew incidents already happening → strong signal to invest.

BD cost-saving alternatives:

Inhouse "lightweight feature store" — single Postgres + Python helper library.
Shared Spark transform module — at least same code.
SaaS (Tecton) — 5-10x more expensive but turn-key.

"Feature store regret" stories:

Premature investment — small team, complex setup, slow to use.
Better — reach pain point first, then invest.

মূল উপলব্ধি: Feature store — pain-driven, not platform-driven invest. Real skew bugs, real feature duplication — then go. Otherwise simpler patterns sufficient.

প্র ০২ "Online store latency budget — Redis vs DynamoDB vs Cassandra."

Online store choice — latency, cost, availability trade-off.

Redis:

In-memory, sub-millisecond reads.
Self-hosted easy, managed (Elasticache, MemoryStore) easier.
Memory-bound — large feature sets expensive.
Cluster mode complex but scalable.

DynamoDB:

AWS managed, ~10ms reads (typical).
Auto-scaling, pay-per-request.
Wide adoption, integrated with Feast.
Cost based on RCU/WCU.

Cassandra:

Distributed, eventual-consistent, ~5-15ms.
Self-hosted heavy ops; managed (Astra) available.
Best for very large-scale, multi-region.

Latency budget allocation example:

Total p99 budget: 100ms.
Network from caller: 10ms.
Feature lookup: 5-15ms (target).
Model inference: 30-50ms.
Response back: 5ms.
Buffer: 20ms.

BD-context choice:

Small/medium ML team — Redis (managed via DigitalOcean, AWS Elasticache Mumbai). Simple, fast.
AWS-native team — DynamoDB seamless.
On-prem mandate — self-hosted Redis cluster + Sentinel for HA.

Cost comparison (rough, monthly for moderate workload):

Redis Elasticache (cache.r6g.large × 2): ~$৩০০।
DynamoDB (10K RCU + storage): ~$২০০-৫০০ depending traffic.
Self-hosted Redis (3-node K8s cluster on existing VM): ~$৫০ + ops.

Common mistake:

Wrong eviction policy — features evicted before next prediction needs.
TTL too aggressive — frequent miss → fallback to offline → SLA breach.
Hot keys (popular user features) — cluster shard hotspot.

মূল উপলব্ধি: Latency target match → backend choose. 99% production use case Redis fits. DynamoDB AWS-only convenience. Cassandra rare-need (multi-region, very high scale). Start Redis, migrate if needed.

প্র ০৩ "Feature freshness vs cost trade-off" — daily refresh vs streaming.

Feature freshness — model accuracy ও infrastructure cost-এর tension.

Daily batch materialize:

Offline → online sync once/day (e.g., 2 AM).
Pros: simple, cheap, predictable.
Cons: features up to 24 hours stale.
Use case: features that change slowly (user profile, preferences).

Hourly batch:

24x more compute cost than daily.
Use case: medium-velocity features (session aggregates).

Streaming materialize:

Kafka → Spark Streaming/Flink → online store.
Pros: seconds-fresh.
Cons: complex, expensive, debugging painful.
Use case: real-time fraud, ad bidding.

Bangladesh use case examples:

Pathao ride pricing: demand changes minute-by-minute → streaming.
Daraz user profile: daily preferences sufficient → batch.
bKash fraud: session-level streaming + user-level batch hybrid.

Hybrid pattern (most common):

User-level features: daily batch.
Session-level features: streaming (last N events).
Real-time signal: in-request compute (cheap, last X minutes).

Cost saving — selective freshness:

Not all features need same freshness.
Feature view-এ ttl set differently per feature group.
"Feature value" 알려진 — invest accordingly.

Ablation study suggestion:

Remove fresh features (use stale only) → measure accuracy drop.
If 0.1% drop only — daily batch sufficient.
If 5%+ drop — streaming worth.

মূল উপলব্ধি: Freshness — investment-worth carefully measured. Default daily batch; promote to streaming only with measurable benefit. Bangladesh fintech-এ session-level streaming common for fraud; e-commerce mostly batch sufficient.

প্র ০৪ "Feast vs Tecton vs Hopsworks vs build-it-yourself."

Feature store landscape ২০২৫-এ — broader but stratified.

Feast (open-source):

Free, self-hostable, Pythonic.
Active community, Linux Foundation governed.
Integrations: BigQuery, Snowflake, Redis, etc.।
"DIY" infrastructure — you operate.
Best: cost-sensitive, control-wanting team.

Tecton (SaaS):

Founded by ex-Uber Michelangelo team.
Managed, polished UI, batch + streaming.
Very expensive (enterprise-priced).
Best: well-funded mid/large team.

Hopsworks:

Open-source feature store + ML platform.
Includes feature engineering, model registry, serving.
Self-host or managed.
Best: comprehensive ML platform need.

Cloud provider native:

SageMaker Feature Store (AWS).
Vertex AI Feature Store (GCP).
Azure ML Managed Feature Store.
Tied to cloud, integrated with their ML services.

Build-it-yourself:

Simple Postgres + helper Python lib.
Pros: minimal, totally controlled.
Cons: maintenance, missing point-in-time logic.
Best: very small team, simple use case.

BD-context decision:

Most BD ML teams — Feast.
SageMaker shop — SageMaker FS.
Build-yourself only with single model, single team, simple features.
Tecton/Hopsworks rare in BD due to cost.

Decision criteria:

Budget — Feast for free, others paid.
Team expertise — Feast needs Python+infra, SaaS needs none.
Scale — Feast handles billion-feature; SaaS may be cheaper at huge scale.
Compliance — on-prem Feast for sensitive data.

Future trend:

"DAG-based feature engineering" (Hamilton, etc.) — orthogonal to FS.
"Online ML platforms" — feature store + model + serving bundled (Feature Store 2.0).
LLM-era — embedding feature store new sub-category.

মূল উপলব্ধি: Feast = sweet spot for most BD teams. Open-source, flexible, well-supported. Cloud-native if AWS/GCP-locked-in. Tecton enterprise rare-justified. DIY sometimes okay but rarely optimal.

অনুশীলন

Setup: Local Feast project create করুন। একটি simple feature view define করুন।

feast init my_project → cd my_project → feast apply. UI not built-in but CLI দিয়ে exploration.
Materialize: Sample data দিয়ে materialize করুন; online lookup test করুন।

feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S") → Python from FeatureStore.get_online_features()। Sub-ms response.
চিন্তা: Pathao-র driver matching model. ৫টি feature design করুন। কোনগুলো batch, কোনগুলো streaming?
- driver_acceptance_rate_30d — batch.
- driver_completion_rate_30d — batch.
- driver_current_lat_lng — streaming.
- driver_active_minutes_today — hourly batch.
- recent_5_min_demand_in_area — streaming.