পাঠ ১৭ · ৪৫-এর মধ্যে · মডিউল ২
Home / AI Courses / Machine Learning / k-Nearest Neighbors

k-Nearest Neighbors

k-NN algorithm
৭ মিনিট পড়া মাঝারি · Intermediate NumPy + sklearn

এই পাঠে যা শিখবেন

  • k-NN — algorithm, distance metrics, voting
  • $k$-এর effect — bias-variance dial
  • Scaling-এর critical role
  • Curse of dimensionality — high-D-তে k-NN fail কেন

১ · Algorithm — সরল কিন্তু শক্তিশালী

k-NNk-Nearest NeighborsInstance-based learning — training-এ কিছু শেখে না, predict-time-এ closest training samples ভোট দেয়। ১৯৫১-এ Fix & Hodges প্রস্তাব। algorithm:

  1. সব training points store।
  2. New test point $\mathbf{x}^*$ আসলে — সব training points-এর সাথে distance compute।
  3. Smallest distance-এর $k$ neighbors বাছ।
  4. Classification: majority vote। Regression: mean।
"No training" model

Linear/logistic regression — explicit parameters শেখে। k-NN training-এ শুধু data মুখস্থ। সব কাজ inference-এ। তাই — "lazy learner"।

২ · Distance metrics

"কাছাকাছি" মাপতে — distance function লাগে।

  • Euclidean (L2): $d = \sqrt{\sum (x_i - y_i)^2}$ — সবচেয়ে common।
  • Manhattan (L1): $d = \sum |x_i - y_i|$ — block-distance।
  • Minkowski: $d = (\sum |x_i - y_i|^p)^{1/p}$ — generalization।
  • Cosine: $d = 1 - \frac{\mathbf{x} \cdot \mathbf{y}}{\|\mathbf{x}\| \|\mathbf{y}\|}$ — direction-based।
  • Hamming: Discrete features — mismatch count।
ভাবুন ঢাকা শহরে দু'টি বাড়ি। Euclidean — straight-line aerial distance। Manhattan — actual road distance (বাঁক করতে হয়)। Cosine — দিকের মিল (একই road-এ থাকলে close)। সব context-এ ভিন্ন meaningful।

৩ · $k$-এর role

  • $k = 1$: Closest point-এর label use। Highly variable, noise-sensitive।
  • $k$ small: Overfit — local noise capture।
  • $k$ large: Underfit — class boundary smooth, detail miss।
  • $k = N$: সব training point use — majority class always predict (useless)।

Bias-variance lens-এ — $k$ explicit dial।

k-NN — How Voting Works ? Query k = 5 neighborhood ৫ closest neighbors ● Class A: 3 ■ Class B: 2 → Majority vote Predicted: A confidence: 60% k vary করলে — boundary বদলায়।
Query point-এর ৫ closest neighbors → majority vote → predicted class।

৪ · NumPy দিয়ে — শূন্য থেকে

Python · NumPy 
import numpy as np

def knn_predict(X_train, y_train, x_test, k=5):
    """Single test point-এর জন্য prediction."""
    # Euclidean distances
    dists = np.sqrt(np.sum((X_train - x_test) ** 2, axis=1))
    # Top-k indices
    nearest = np.argsort(dists)[:k]
    nearest_labels = y_train[nearest]
    # Majority vote
    vals, counts = np.unique(nearest_labels, return_counts=True)
    return vals[np.argmax(counts)]

# Toy data
np.random.seed(0)
N = 100
X = np.random.randn(N, 2)
y = (X[:, 0] + X[:, 1] > 0).astype(int)

# Test query
x_q = np.array([0.3, 0.4])
for k in [1, 3, 5, 11]:
    pred = knn_predict(X, y, x_q, k=k)
    print(f"k={k:2d}: predicted class = {pred}")

    


    

      Different $k$ — sometimes same prediction, sometimes ভিন্ন। Boundary-এর কাছাকাছি query-এ $k$-এর effect বেশি।
    


    
৫ · scikit-learn — production-grade


    

      

        
        Python · scikit-learn
      

      
from sklearn.datasets import load_iris
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split, cross_val_score
import numpy as np

X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)

scaler = StandardScaler()
X_tr = scaler.fit_transform(X_tr)
X_te = scaler.transform(X_te)

# k vary
for k in [1, 3, 5, 7, 15, 30]:
    model = KNeighborsClassifier(n_neighbors=k)
    cv_score = cross_val_score(model, X_tr, y_tr, cv=5).mean()
    model.fit(X_tr, y_tr)
    test = model.score(X_te, y_te)
    print(f"k={k:2d}: cv_acc={cv_score:.4f}, test_acc={test:.4f}")

      

        
        
      

      

    


    

      $k = 5$-৭ — sweet spot for Iris। $k = 1$ — overfit hint। $k = 30$ — too smooth।
    


    
৬ · Scaling — অপরিহার্য


    

      Distance metric scale-sensitive। "Income" range ০-১০M ও "age" ০-১০০ — income dominate। Solution: StandardScaler বা MinMaxScaler always।
    


    

      

        
        Python · scikit-learn
      

      
import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import StandardScaler

np.random.seed(0)
N = 500
# age (0-100) ও income (0-100k) ভিন্ন scale-এ
age = np.random.uniform(20, 70, N)
income = np.random.uniform(20000, 200000, N)
X = np.column_stack([age, income])
y = (income > 80000).astype(int)

knn = KNeighborsClassifier(n_neighbors=5)

# Without scaling
cv1 = cross_val_score(knn, X, y, cv=5).mean()
# With scaling
X_s = StandardScaler().fit_transform(X)
cv2 = cross_val_score(knn, X_s, y, cv=5).mean()

print(f"Without scaling: cv accuracy = {cv1:.4f}")
print(f"With scaling   : cv accuracy = {cv2:.4f}")

      

        
        
      

      

    


    

      Without scaling — age প্রায় ignored (income-dominated distance)। Scaled — both features fairly contribute। Accuracy markedly improve।
    


    
৭ · Computational considerations


    
    
      
  • Naive search: $O(N \cdot d)$ per query। Big data-তে slow।
    • 
      
  • KD-Tree: Low-D ($d < 20$) — $O(\log N)$ average।
    • 
      
  • Ball Tree: Higher-D, non-Euclidean।
    • 
      
  • Approximate (FAISS, Annoy): Massive scale, slight accuracy trade।
    • 
    


    
৮ · কোথায় k-NN rules


    
    
      
  • Recommendation systems: "Similar users" — instance-based natural fit।
    • 
      
  • Image retrieval: "Find similar images" — embedding-এ k-NN।
    • 
      
  • Anomaly detection: Isolated points — far from neighbors।
    • 
      
  • Imputation: Missing values — k nearest-এর mean।
    • 
      
  • RAG (Retrieval-Augmented Generation): Modern AI-তে massive k-NN over embeddings।
    • 
    


    
৯ · কোথায় fail


    
    
      
  • High dimensions: Curse of dimensionality — সব points "equally far"।
    • 
      
  • Imbalanced classes: Majority class neighbor-এ dominate।
    • 
      
  • Noisy features: Irrelevant features distance dominate।
    • 
      
  • Big data: Memory + inference slow।
    • 
    


    

      k-NN — pedagogically beautiful, production-এ specific niche। Modern AI vector DBs (Pinecone, Weaviate) — internally k-NN। তাই concept master করা long-term valuable।
    


  


  

    
    
  


  

    
 ভাবনার প্রশ্ন

    
প্রতিটি প্রশ্ন নিজে কিছুক্ষণ ভাবুন — তারপর "→ উত্তর" চাপুন।


    

      
        প্র ০১
        "Curse of dimensionality" k-NN-কে কীভাবে fail করে? সব points "equidistant" — গাণিতিকভাবে কেন?
      
      

        

          High-D-তে distance-based methods structurally fail। Theoretical foundation deep।
        

        

          Empirical observation:
        

        
    
          
  • Random points $N$ in $d$-D unit cube।
    • 
          
  • Pairwise distances compute।
    • 
          
  • $d$ small: clear nearest/farthest distinction।
    • 
          
  • $d$ large: nearest ≈ farthest।
    • 
        

        

          Concentration of measure:
        

        
    
          
  • Beyer et al. (1999) theorem।
    • 
          
  • $\frac{\text{maxdist} - \text{mindist}}{\text{mindist}} \to 0$ as $d \to \infty$।
    • 
          
  • "Meaningfulness of nearest neighbor" disappears।
    • 
        

        

          Mathematical intuition:
        

        
    
          
  • $d$ independent random coordinates।
    • 
          
  • Each pair sum $d$ random variables।
    • 
          
  • Variance grows with $d$।
    • 
          
  • But mean grows faster — relative variance shrinks।
    • 
          
  • "All distances cluster around mean"।
    • 
        

        

          Geometric intuition:
        

        
    
          
  • $d$-D cube — most volume in corners।
    • 
          
  • Center increasingly empty।
    • 
          
  • Random points near boundary।
    • 
          
  • Far apart from "expected center"।
    • 
        

        

          k-NN consequences:
        

        
    
          
  • "k-nearest" indistinguishable from far।
    • 
          
  • Vote unreliable।
    • 
          
  • Decision boundary noisy।
    • 
          
  • Performance degrades with $d$।
    • 
        

        

          Sample complexity:
        

        
    
          
  • To cover $d$-D unit cube uniformly — $N \sim \epsilon^{-d}$ samples।
    • 
          
  • $d = 10, \epsilon = 0.1$: $10^{10}$ samples।
    • 
          
  • Real datasets — far smaller।
    • 
          
  • "Holes" everywhere।
    • 
        

        

          Affected algorithms:
        

        
    
          
  • k-NN — primary victim।
    • 
          
  • Kernel methods — bandwidth choice ill-defined।
    • 
          
  • Density estimation — underestimate everywhere।
    • 
          
  • Clustering — meaningless distances।
    • 
        

        

          Solutions:
        

        

          (১) Dimensionality reduction:
        

        
    
          
  • PCA — linear projection।
    • 
          
  • t-SNE — non-linear (visualization)।
    • 
          
  • UMAP — modern, scalable।
    • 
          
  • Pre-process before k-NN।
    • 
        

        

          (২) Feature selection:
        

        
    
          
  • Domain knowledge — important features।
    • 
          
  • Filter methods — correlation।
    • 
          
  • Wrapper methods — model-based।
    • 
        

        

          (৩) Manifold hypothesis:
        

        
    
          
  • Real data — low-D manifold in high-D space।
    • 
          
  • Distances on manifold meaningful।
    • 
          
  • Geodesic distances — Isomap।
    • 
        

        

          (৪) Learned distances:
        

        
    
          
  • Metric learning — task-specific।
    • 
          
  • Embedding-based — neural networks।
    • 
          
  • Contrastive learning — modern।
    • 
        

        

          (৫) Approximate methods:
        

        
    
          
  • LSH (Locality Sensitive Hashing)।
    • 
          
  • Probabilistic guarantees।
    • 
          
  • Trade accuracy for speed।
    • 
        

        

          Modern AI mitigations:
        

        
    
          
  • Pre-trained embeddings — meaningful low-D representations।
    • 
          
  • BERT, GPT, CLIP — embedding spaces well-structured।
    • 
          
  • Vector DBs (Pinecone, FAISS) — efficient k-NN in embedding space।
    • 
        

        

          When dimensionality not curse:
        

        
    
          
  • Sparse data (text TF-IDF) — most distances meaningful।
    • 
          
  • Low intrinsic dimensionality — ambient $d$ misleading।
    • 
          
  • Highly structured data।
    • 
        

        

          Practical guidelines:
        

        
    
          
  • $d < 20$: k-NN often viable।
    • 
          
  • $d \in [20, 100]$: dimensionality reduction first।
    • 
          
  • $d > 100$: avoid raw k-NN।
    • 
        

        

          মূল উপলব্ধি: Curse of dimensionality — k-NN's existential threat। Modern AI sidesteps via learned embeddings। "Naive distance" দীর্ঘকাল আগেই মৃত — "learned distance" alive।
        

      

    


    

      
        প্র ০২
        "Lazy learner" বলা হয় k-NN-কে। Memory + inference cost। কখন এটা production-এ acceptable, কখন না?
      
      

        

          Eager vs lazy learning — fundamental ML architecture choice।
        

        

          Eager learning (most ML):
        

        
    
          
  • Train phase — model parameters extract।
    • 
          
  • Inference phase — compact, fast।
    • 
          
  • Linear regression, NN, decision trees।
    • 
        

        

          Lazy learning:
        

        
    
          
  • Train — store data।
    • 
          
  • Inference — compute against entire dataset।
    • 
          
  • k-NN, kernel methods।
    • 
        

        

          k-NN tradeoffs:
        

        

          (১) Memory:
        

        
    
          
  • Entire training set in RAM।
    • 
          
  • ৪M images × ১০২৪-D = ১৬GB।
    • 
          
  • Production prohibitive often।
    • 
        

        

          (২) Inference latency:
        

        
    
          
  • Naive — $O(Nd)$ per query।
    • 
          
  • Real-time impossible at scale।
    • 
          
  • Indexing helps — KD-tree, ball tree।
    • 
        

        

          (৩) No model file:
        

        
    
          
  • Deployment — entire dataset।
    • 
          
  • Privacy concerns — training data exposed।
    • 
          
  • Versioning awkward।
    • 
        

        

          Acceptable scenarios:
        

        

          (১) Small data:
        

        
    
          
  • $N < 10^4$ — memory fine।
    • 
          
  • Inference fast enough।
    • 
          
  • Simple deployment।
    • 
        

        

          (২) Frequent retraining:
        

        
    
          
  • "Add data, no retraining"।
    • 
          
  • Online learning natural fit।
    • 
          
  • Recommendation systems।
    • 
        

        

          (৩) Interpretability:
        

        
    
          
  • "Found ৫ similar cases" — clear explanation।
    • 
          
  • Medical diagnosis — case-based reasoning।
    • 
          
  • Legal precedents।
    • 
        

        

          (৪) Custom distance:
        

        
    
          
  • Domain expertise — special metric।
    • 
          
  • Sequence alignment, edit distance।
    • 
          
  • Hard to encode in NN।
    • 
        

        

          (৫) Local patterns:
        

        
    
          
  • Heterogeneous data — global model fails।
    • 
          
  • Local structure capture।
    • 
          
  • Minority groups well-served।
    • 
        

        

          Unacceptable scenarios:
        

        

          (১) Big data:
        

        
    
          
  • $N > 10^6$ — slow।
    • 
          
  • Memory blow-up।
    • 
          
  • Use eager methods।
    • 
        

        

          (২) Real-time strict:
        

        
    
          
  • <1ms inference।
    • 
          
  • Linear/NN faster typically।
    • 
          
  • Or embedding + ANN search।
    • 
        

        

          (৩) Mobile/edge:
        

        
    
          
  • Limited memory।
    • 
          
  • Compact model essential।
    • 
        

        

          (৪) Privacy:
        

        
    
          
  • Training data must not deploy।
    • 
          
  • Differential privacy — k-NN tricky।
    • 
        

        

          Modern hybrid approaches:
        

        

          (১) Embeddings + ANN:
        

        
    
          
  • Learn compact representation (eager)।
    • 
          
  • k-NN in embedding space (lazy)।
    • 
          
  • Best of both — modern RAG, search।
    • 
        

        

          (২) Prototypes:
        

        
    
          
  • Cluster → keep centroids।
    • 
          
  • Reduced "training set"।
    • 
          
  • Memory-efficient।
    • 
        

        

          (৩) Approximate NN:
        

        
    
          
  • FAISS, Annoy, HNSW।
    • 
          
  • Sub-linear inference।
    • 
          
  • Slight accuracy loss।
    • 
        

        

          (৪) Learned indices:
        

        
    
          
  • NN-based hashing।
    • 
          
  • Locality Sensitive Hashing।
    • 
          
  • Active research।
    • 
        

        

          Vector database era:
        

        
    
          
  • Pinecone, Weaviate, Qdrant।
    • 
          
  • Production-grade k-NN at scale।
    • 
          
  • RAG, semantic search, recommendation।
    • 
          
  • k-NN renaissance।
    • 
        

        

          Production architecture:
        

        
    
          
  • Embedding model (pre-trained)।
    • 
          
  • Vector DB (managed service)।
    • 
          
  • Application layer।
    • 
          
  • Hybrid eager+lazy।
    • 
        

        

          মূল উপলব্ধি: Lazy learning niche কিন্তু persistent। Modern AI infrastructure — k-NN secret backbone। Deployment complexity → managed services bridge।
        

      

    


    

      
        প্র ০৩
        "k odd" বাছার convention কেন? Tie-breaking rules কী? Weighted k-NN কীভাবে kit-bashing উন্নত করে?
      
      

        

          k-NN-এর small but important details — production reliability।
        

        

          Odd $k$ convention:
        

        
    
          
  • Binary classification।
    • 
          
  • Even $k$ — tie possible (e.g., 2-2)।
    • 
          
  • Odd $k$ — strict majority guaranteed।
    • 
          
  • $k = 5$ standard, $k = 7$ also common।
    • 
        

        

          Multi-class — odd irrelevant:
        

        
    
          
  • ৩ class — ties possible at any $k$।
    • 
          
  • e.g., $k = 5$, vote 2-2-1।
    • 
          
  • Need explicit tie-breaking।
    • 
        

        

          Tie-breaking strategies:
        

        

          (১) Distance-weighted:
        

        
    
          
  • Close votes — more weight।
    • 
          
  • Tie almost impossible (continuous weights)।
    • 
          
  • Most popular choice।
    • 
        

        

          (২) Random:
        

        
    
          
  • Tie → random choice।
    • 
          
  • Reproducibility — fixed seed।
    • 
          
  • Simple but unsatisfying।
    • 
        

        

          (৩) Class priors:
        

        
    
          
  • Tie → most common class।
    • 
          
  • Imbalanced data biased।
    • 
        

        

          (৪) Decrement $k$:
        

        
    
          
  • $k - 1$ neighbors used।
    • 
          
  • Recursive তে possible।
    • 
          
  • Less common।
    • 
        

        

          (৫) Increment $k$:
        

        
    
          
  • $k + 1$ — break tie।
    • 
          
  • Different sample — different neighbor।
    • 
        

        

          Weighted k-NN:
        

        

          $$\hat{y} = \arg\max_c \sum_{i \in N_k} w_i \cdot \mathbb{1}[y_i = c]$$
        

        
    
          
  • $w_i$ — distance-based weight।
    • 
          
  • Closer neighbors — more influence।
    • 
        

        

          Weight choices:
        

        

          (১) Inverse distance:
        

        
    
          
  • $w_i = 1 / (d_i + \epsilon)$।
    • 
          
  • Sharp peak at $d \to 0$।
    • 
          
  • Default in scikit-learn (weights='distance')।
    • 
        

        

          (২) Gaussian kernel:
        

        
    
          
  • $w_i = \exp(-d_i^2 / 2\sigma^2)$।
    • 
          
  • Smooth weighting।
    • 
          
  • Bandwidth $\sigma$ tunable।
    • 
        

        

          (৩) Triangular:
        

        
    
          
  • $w_i = \max(0, 1 - d_i / d_k)$।
    • 
          
  • Linear decay।
    • 
        

        

          (৪) Uniform:
        

        
    
          
  • $w_i = 1$ for all neighbors।
    • 
          
  • Default standard k-NN।
    • 
        

        

          Performance impact:
        

        
    
          
  • Weighted often outperforms uniform।
    • 
          
  • Less sensitive to $k$ choice।
    • 
          
  • Smoother decision boundaries।
    • 
          
  • Computational cost similar।
    • 
        

        

          Bias-variance:
        

        
    
          
  • Weighted — effective $k$ smaller।
    • 
          
  • Lower bias, similar variance।
    • 
          
  • $k$ relatively larger usable।
    • 
        

        

          Theoretical perspective:
        

        
    
          
  • Kernel density estimation — weighted k-NN cousin।
    • 
          
  • Nadaraya-Watson regression — weighted average।
    • 
          
  • Statistical guarantees।
    • 
        

        

          Modern variants:
        

        
    
          
  • Adaptive k-NN: Local density-aware $k$।
    • 
          
  • Mahalanobis distance: Covariance-aware।
    • 
          
  • Learned metric: Deep metric learning।
    • 
        

        

          Implementation in sklearn:
        

        
    
          
  • weights='uniform' default।
    • 
          
  • weights='distance' inverse distance।
    • 
          
  • Custom callable for special cases।
    • 
        

        

          Edge cases:
        

        
    
          
  • Distance = 0 (duplicate) — divide by zero।
    • 
          
  • Add small $\epsilon$ или clip।
    • 
          
  • scikit-learn handles internally।
    • 
        

        

          Best practices:
        

        
    
          
  • Default: weighted distance।
    • 
          
  • Cross-validate $k$।
    • 
          
  • Try uniform too — sometimes better।
    • 
          
  • Distance metric — domain consideration।
    • 
        

        

          মূল উপলব্ধি: Tie-breaking small detail, robustness-এ vital। Weighted k-NN — usually free improvement। Production deployment-এ default consideration।
        

      

    


    

      
        প্র ০৪
        Modern AI-তে "vector database" ও "RAG" — k-NN-এর renaissance। কীভাবে scale-এ deploy?
      
      

        

          k-NN — academic concept থেকে production powerhouse — embedding revolution-এর সাথে।
        

        

          Modern context:
        

        
    
          
  • Pre-trained embeddings (BERT, OpenAI ada)।
    • 
          
  • Semantic similarity → meaningful distances।
    • 
          
  • k-NN over embeddings — semantic search।
    • 
        

        

          Vector database concept:
        

        
    
          
  • Specialized DB for embeddings।
    • 
          
  • Efficient k-NN at scale।
    • 
          
  • Approximate nearest neighbor (ANN)।
    • 
          
  • Million-billion vectors।
    • 
        

        

          Architecture:
        

        
    
          
  • Indexing: HNSW, IVF, LSH।
    • 
          
  • Storage: Embeddings + metadata।
    • 
          
  • Query API: Top-k, filters।
    • 
          
  • Distributed: Sharding, replication।
    • 
        

        

          Major systems:
        

        
    
          
  • Pinecone: Managed cloud service।
    • 
          
  • Weaviate: Open source + cloud।
    • 
          
  • Qdrant: Rust-based, fast।
    • 
          
  • Milvus: Open source, distributed।
    • 
          
  • FAISS: Library (Meta)।
    • 
          
  • pgvector: PostgreSQL extension।
    • 
        

        

          RAG (Retrieval-Augmented Generation):
        

        

          (১) Concept:
        

        
    
          
  • LLM + external knowledge।
    • 
          
  • Query → retrieve relevant docs → augment prompt।
    • 
          
  • Hallucination reduction।
    • 
          
  • Up-to-date information।
    • 
        

        

          (২) Architecture:
        

        
    
          
  • Document store (vector DB)।
    • 
          
  • Query embedding।
    • 
          
  • Top-k retrieval।
    • 
          
  • LLM generation।
    • 
        

        

          (৩) Use cases:
        

        
    
          
  • Customer support — knowledge base lookup।
    • 
          
  • Internal docs Q&A।
    • 
          
  • Code search (Cursor, GitHub Copilot)।
    • 
          
  • Legal research।
    • 
        

        

          ANN algorithms:
        

        

          (১) HNSW (Hierarchical NSW):
        

        
    
          
  • Graph-based, hierarchical layers।
    • 
          
  • $O(\log N)$ search।
    • 
          
  • Modern default।
    • 
        

        

          (২) IVF (Inverted File Index):
        

        
    
          
  • Cluster + search within cluster।
    • 
          
  • Compact, fast।
    • 
          
  • Quantization combined।
    • 
        

        

          (৩) LSH (Locality Sensitive Hashing):
        

        
    
          
  • Hash similar items to same bucket।
    • 
          
  • Probabilistic guarantees।
    • 
          
  • Older standard।
    • 
        

        

          (৪) Product Quantization:
        

        
    
          
  • Compression + ANN।
    • 
          
  • Memory-efficient।
    • 
          
  • Combined with IVF।
    • 
        

        

          Bangladesh applications:
        

        

          (১) Bangla document search:
        

        
    
          
  • BanglaBERT embeddings।
    • 
          
  • News article similarity।
    • 
          
  • Government document Q&A।
    • 
        

        

          (২) E-commerce:
        

        
    
          
  • Daraz product image similarity।
    • 
          
  • "Find similar product"।
    • 
          
  • Recommendation।
    • 
        

        

          (৩) Customer service:
        

        
    
          
  • FAQ matching।
    • 
          
  • Multi-lingual (Bangla + English)।
    • 
          
  • Chatbot foundation।
    • 
        

        

          (৪) Healthcare:
        

        
    
          
  • Medical literature retrieval।
    • 
          
  • Symptom matching।
    • 
          
  • Privacy-preserving setup।
    • 
        

        

          Production considerations:
        

        

          (১) Embedding model:
        

        
    
          
  • Pre-trained vs fine-tuned।
    • 
          
  • Cost vs quality।
    • 
          
  • Multi-lingual support।
    • 
        

        

          (২) Index choice:
        

        
    
          
  • Recall vs latency tradeoff।
    • 
          
  • Memory budget।
    • 
          
  • Update frequency।
    • 
        

        

          (৩) Hybrid search:
        

        
    
          
  • Vector + keyword (BM25)।
    • 
          
  • Reranking — cross-encoder।
    • 
          
  • Filter metadata first।
    • 
        

        

          (৪) Monitoring:
        

        
    
          
  • Query latency।
    • 
          
  • Recall@k metric।
    • 
          
  • Index size growth।
    • 
          
  • Drift detection।
    • 
        

        

          Evaluation:
        

        
    
          
  • Recall@k — top-k accuracy।
    • 
          
  • NDCG — ranking quality।
    • 
          
  • MRR — first-relevant rank।
    • 
          
  • Human evaluation gold standard।
    • 
        

        

          Future directions:
        

        
    
          
  • Learned indices — NN-based hashing।
    • 
          
  • Multi-modal — image + text + audio।
    • 
          
  • Federated vector search।
    • 
          
  • Real-time indexing।
    • 
        

        

          মূল উপলব্ধি: k-NN academic গ্রন্থাগার থেকে production frontier-এ। Embedding + vector DB + RAG = AI's new layer। Bangladesh AI startups massive opportunity here।
        

      

    

  


  

    
 অনুশীলন

    
    
      
  1. 
        হিসাব করুন: Training: $\{(1, A), (2, A), (5, B), (6, B)\}$। Query $x = 3$, $k = 3$।
        
      
          
    • ৩ closest কী?
      • 
          
    • Vote কী? Predicted class?
      • 
        
    
        
    
          
          
    
            
      
              
    • Distances: |3-1|=2, |3-2|=1, |3-5|=2, |3-6|=3।
      • 
              
    • ৩ closest: 2 (A, dist 1), 1 (A, dist 2), 5 (B, dist 2)।
      • 
              
    • Vote: A=2, B=1। Predicted: A।
      • 
            
    
          
    
        
    
      
    2. 

      
  2. 
        NumPy: উপরের custom knn_predict function-কে regression-এ adapt করুন (mean of neighbors)।
        
    
          
          
    
            
    def knn_regress(X_train, y_train, x_test, k=5):
    dists = np.sqrt(np.sum((X_train - x_test)**2, axis=1))
    idx = np.argsort(dists)[:k]
    return np.mean(y_train[idx])  # mean instead of vote
    
          
    
        
    
      
    3. 

      
  3. 
        চিন্তা: Bangladesh-এ একটি real estate price predictor — k-NN feasible? Pros, cons?
        
    
          
          
    
            
    Pros: similar localities-এর price avg meaningful, interpretable ("similar 5 flats avg")। Cons: high-D feature space (location, amenities), data sparse, scaling critical, distance metric choice (lat-long + amenities mix)।
    
            
    Hybrid: localized regression — k-NN-based feature extraction + linear final layer।
    
          
    
        
    
      
    4. 
    

  


  

    
 আরও পড়ুন

    

    

      কোড রানার কাজ না করলে? Google Colab ব্যবহার করুন।
    

  







  
   ভুল পেলে জানান





  
    
 পূর্ববর্তী পাঠ

    
পাঠ ১৬ · Elastic Net