MongoDB Deep Dive — Documents, Aggregation, Sharding
MongoDB — সম্পূর্ণ আলোচনা
1. Why a Document Database?
For forty years SQL ruled by forcing every fact into a flat row with fixed columns. That works beautifully for accounting and inventory — but it strains when the data is naturally nested: an order with line items, a blog post with embedded comments, a player profile with an inventory of weapons. MongoDB stores each entity as a single document — a JSON-shaped object that can have arrays, sub-objects and varying fields per document.
MongoDB is the most-deployed NoSQL database in the world. Uber stores driver/rider matchings, The New York Times stores articles, Adidas stores product catalogs, and most modern Node.js startups in Bangladesh begin with MongoDB Atlas because the on-disk layout mirrors the JSON their JavaScript code already speaks.
2. BSON Documents and Collections
On the wire and on disk MongoDB uses BSON (Binary JSON), a typed superset of JSON that
adds ObjectId, Date, Decimal128, BinData and a fast
length-prefixed encoding so the server does not have to re-parse text every time.
Date, ObjectId,
Decimal128), সেগুলো BSON-এ অন্তর্ভুক্ত। এতে server প্রতি query-তে text পার্স না করেই দ্রুত পড়তে পারে।
A collection is the document equivalent of a SQL table — but without a fixed schema. Two documents in the same collection may have different fields. A database contains many collections; a MongoDB cluster contains many databases.
{
"_id": ObjectId("65f1c2e8a1b4c3d2e5f60001"),
"customer": { "name": "Arif Hossain", "phone": "+8801711000001", "city": "Dhaka" },
"items": [
{ "sku": "PHN-IPH13", "name": "iPhone 13", "qty": 1, "price": 95000 },
{ "sku": "ACC-CASE", "name": "Silicone case", "qty": 1, "price": 1200 }
],
"total": 96200,
"status": "PAID",
"tags": ["mobile", "apple", "express-delivery"],
"placedAt": ISODate("2025-08-12T09:11:33Z")
}
Notice how the order, its customer, its items and its tags all live in one place. To rebuild this in
SQL you would need orders, order_items, customers and
order_tags tables and a 4-way JOIN to read it back.
3. CRUD with the Mongo Shell
The mongo shell exposes JavaScript-flavoured methods on the global db handle. Each of the
four classic operations has a singular and a plural form.
insertOne ও insertMany, updateOne ও updateMany।
এতে আপনি সচেতনভাবে বেছে নিতে পারেন একটি document নাকি অনেকগুলো পরিবর্তন হবে।
// 1. CREATE
db.orders.insertOne({
customer: { name: "Mim Akter", city: "Chittagong" },
items: [{ sku: "BK-DBMS", qty: 1, price: 450 }],
total: 450,
status: "NEW",
placedAt: new Date()
});
// 2. READ — find all PAID orders from Dhaka, newest first, limit 10
db.orders.find(
{ status: "PAID", "customer.city": "Dhaka" },
{ items: 1, total: 1, placedAt: 1 } // projection
).sort({ placedAt: -1 }).limit(10);
// 3. UPDATE — mark all NEW orders older than 30 min as TIMEOUT
db.orders.updateMany(
{ status: "NEW", placedAt: { $lt: new Date(Date.now() - 30*60*1000) } },
{ $set: { status: "TIMEOUT" }, $currentDate: { updatedAt: true } }
);
// 4. DELETE — remove one cancelled test order
db.orders.deleteOne({ _id: ObjectId("65f1c2e8a1b4c3d2e5f60001") });
| SQL | MongoDB | Note |
|---|---|---|
SELECT * FROM orders WHERE status='PAID' | db.orders.find({status:"PAID"}) | Filter document. |
SELECT total FROM orders | db.orders.find({}, {total:1, _id:0}) | Projection. |
UPDATE orders SET status='X' WHERE id=1 | updateOne({_id:1}, {$set:{status:"X"}}) | Always use update operators. |
DELETE FROM orders WHERE id=1 | deleteOne({_id:1}) | One vs many. |
INSERT INTO orders ... | insertOne({...}) | No fixed columns. |
COUNT(*) | countDocuments(filter) | Or aggregation $count. |
updateOne({_id:1}, {status:"X"}) replaces the whole document with
{status:"X"}. You almost always want $set, $inc, $push,
$pull, $addToSet etc.
$set ছাড়া update লিখলে পুরো document-ই overwrite হয়ে যাবে — এটি MongoDB-র সবচেয়ে cruel ফাঁদ।
4. The Aggregation Pipeline
find() can filter, project and sort. For anything more — grouping, joining, reshaping —
MongoDB has the aggregation pipeline: an array of stages, where each stage
takes a stream of documents in and produces a stream of documents out. Each stage starts with a
dollar-prefixed name like $match or $group.
SELECT ... GROUP BY ... HAVING ... ORDER BY যা করে, MongoDB-তে সেটিকেই ভেঙে কয়েকটি stage হিসেবে লেখা হয়।
// Total revenue per city for PAID orders in 2025, top 5 cities
db.orders.aggregate([
{ $match: {
status: "PAID",
placedAt: { $gte: ISODate("2025-01-01"),
$lt: ISODate("2026-01-01") } } },
{ $group: {
_id: "$customer.city", // the bucket key
orders: { $sum: 1 },
revenue:{ $sum: "$total" },
avgTicket: { $avg: "$total" } } },
{ $sort: { revenue: -1 } },
{ $limit: 5 },
{ $project: { // reshape final output
_id: 0, city: "$_id", orders: 1,
revenue: 1, avgTicket: { $round: ["$avgTicket", 0] } } }
]);
| Stage | Equivalent in SQL | Purpose |
|---|---|---|
$match | WHERE | Filter documents (use early!). |
$project | SELECT col1, col2 | Reshape, add computed fields. |
$group | GROUP BY | Bucket and aggregate. |
$sort | ORDER BY | Order the stream. |
$limit / $skip | LIMIT/OFFSET | Pagination. |
$lookup | LEFT JOIN | Pull matching docs from another collection. |
$unwind | — | Turn an array field into one doc per element. |
$facet | multiple UNIONs | Run several pipelines in parallel on the same input. |
$match to the front
The query planner can use indexes on a $match stage only if it appears before any
stage that breaks document identity (like $group or $unwind).
$match-কে যত আগে রাখবেন, তত index ব্যবহার হওয়ার সম্ভাবনা বেশি — তত দ্রুত query।
5. $lookup, $unwind and $facet in Action
Real reports rarely fit in one collection. $lookup performs a left-outer join into another
collection; $unwind explodes an array field so each element becomes its own document;
and $facet runs several mini-pipelines side-by-side on the same input — perfect for the
"products + total count + facet counts" call that powers e-commerce search pages.
$lookup = SQL-এর LEFT JOIN। $unwind = array-কে আলাদা আলাদা document-এ ছড়িয়ে দেওয়া।
$facet = একই input থেকে একসাথে কয়েকটি ভিন্ন রিপোর্ট তৈরি করা — যেমন Daraz/Pickaboo-র product list,
facet counts ও total একসাথে আসা।
// "Top 10 SKUs sold in 2025, with product name and category from products collection"
db.orders.aggregate([
{ $match: { status: "PAID",
placedAt: { $gte: ISODate("2025-01-01") } } },
{ $unwind: "$items" }, // one doc per line item
{ $group: { _id: "$items.sku",
qty: { $sum: "$items.qty" },
revenue: { $sum: { $multiply: ["$items.qty", "$items.price"] } } } },
{ $sort: { revenue: -1 } },
{ $limit: 10 },
{ $lookup: { // join into products collection
from: "products",
localField: "_id",
foreignField: "sku",
as: "product" } },
{ $project: {
_id: 0,
sku: "$_id",
name: { $arrayElemAt: ["$product.name", 0] },
category: { $arrayElemAt: ["$product.category", 0] },
qty: 1, revenue: 1 } }
]);
// E-commerce search page — items + total + facet counts in one round trip
db.products.aggregate([
{ $match: { active: true, $text: { $search: "smartphone" } } },
{ $facet: {
items: [
{ $sort: { popularity: -1 } },
{ $skip: 0 }, { $limit: 20 },
{ $project: { name:1, price:1, brand:1 } }
],
total: [ { $count: "n" } ],
brandFacets: [ { $group: { _id: "$brand", n: { $sum: 1 } } } ],
priceBuckets: [ { $bucket: {
groupBy: "$price",
boundaries: [0, 10000, 25000, 50000, 100000, 500000],
default: "500000+",
output: { n: { $sum: 1 } } } } ]
} }
]);
6. Indexes — All the Flavours
Without an index, a query on a million documents performs a collection scan: read every byte. With the right index, the same query touches a few thousand bytes. MongoDB gives you a rich menu of index types — pick the one that matches the query you actually run.
| Index Type | Created With | Use Case |
|---|---|---|
| Single field | { email: 1 } | Unique-by-email lookups, ascending sort. |
| Compound | { status:1, placedAt:-1 } | Filter+sort that hits the prefix. |
| Multikey | { "items.sku": 1 } | Auto-created on array fields. |
| Text | { name: "text", desc: "text" } | Full-text $text search. |
| 2dsphere (geo) | { location: "2dsphere" } | "Riders within 2 km of me" queries. |
| Hashed | { userId: "hashed" } | Even shard distribution. |
| Partial | { email:1 }, { partialFilterExpression: { active:true } } | Index only the rows you actually query. |
| TTL | { createdAt:1 }, { expireAfterSeconds: 3600 } | Auto-delete sessions, OTPs, tracking events. |
| Wildcard | { "$**": 1 } | Schemaless docs where any field could be queried. |
// 1. Compound index — supports filter on status AND sort on placedAt
db.orders.createIndex({ status: 1, placedAt: -1 });
// 2. Geo index — find rides near a point
db.rides.createIndex({ pickup: "2dsphere" });
db.rides.find({ pickup: { $near: {
$geometry: { type: "Point", coordinates: [90.4125, 23.8103] }, // Dhaka
$maxDistance: 2000 // metres
}}});
// 3. TTL index — sessions auto-expire 1 hour after creation
db.sessions.createIndex({ createdAt: 1 }, { expireAfterSeconds: 3600 });
// 4. Partial index — only active users get indexed → smaller, faster
db.users.createIndex(
{ email: 1 },
{ unique: true, partialFilterExpression: { active: true } }
);
{status:"PAID", placedAt: {$gt: t}} sorted by placedAt, the index
{status:1, placedAt:-1} is optimal.
7. Replica Sets — Durability and Failover
A replica set is a group of MongoDB servers (typically three) that all hold a copy of the same data. One is the primary; the rest are secondaries. Every write goes to the primary, which writes it to its oplog (operation log). Secondaries tail the oplog and apply each operation to stay in sync.
{w: "majority"} tells the driver "do not consider the write done until at least 2 of the 3
nodes have it." This is the setting that makes MongoDB safe under failover. {w:1} is faster
but can lose the last few seconds of writes if the primary crashes.
8. Sharding — Scaling Beyond One Machine
Replica sets give you availability but every node still holds the full dataset. When one machine cannot hold the data — or when write traffic exceeds one node's IO budget — you shard: split the collection across many replica sets ("shards"), each owning a slice of the key space.
mongos router
client-এর query সঠিক shard-এ পাঠায়।
| Component | Role |
|---|---|
mongos | Stateless router. Apps talk to mongos; it routes to the right shard. |
| Config servers | A small replica set that stores the shard map (which key range lives on which shard). |
| Shard | A replica set holding one slice of the data. |
| Chunk | A contiguous range of shard-key values, ~128 MB. The unit of movement. |
| Balancer | Background process that moves chunks between shards to keep them even. |
Picking a shard key
The shard key is the most consequential decision in a sharded cluster — and the hardest to change later. A good shard key has three properties:
- High cardinality — many distinct values, so chunks can be split.
- Even write distribution — no single value gets a flood of writes.
- Targeted queries — most queries include the shard key, so they touch one shard, not all.
✅ Good shard keys
{userId: "hashed"}— hashed = even spread.{tenantId: 1, _id: 1}— multi-tenant SaaS.{regionId: 1, deviceId: 1}— IoT.
⚠️ Bad shard keys
{createdAt: 1}— all today's writes hit the last chunk → hotspot.{status: 1}— only ~5 distinct values → unsplittable.- Anything monotonically increasing without hashing.
9. Embed vs Reference — The Schema Design Question
"Should this nested data live inside the parent document, or in a separate collection linked by id?" That single question accounts for 80% of MongoDB schema design. The answer depends on three numbers: cardinality, frequency of update, and document size.
| Pattern | Rule of thumb | Example |
|---|---|---|
| Embed | One-to-few, child read with parent, child rarely updated alone. | Order line items inside an order doc. |
| Reference | One-to-many or many-to-many, child queried independently, child updated often. | Customers ↔ Orders (a customer has many orders, and an order is read alone). |
| Hybrid (extended ref.) | Reference + denormalize a few read-hot fields. | Order stores customer._id + customer.name for display. |
| Bucket | Group time-series points into one doc per hour/day. | 1 sensor reading per second → 1 bucket doc per hour with 3600 readings. |
একটি document সর্বোচ্চ ১৬ MB। এর বেশি child থাকার সম্ভাবনা থাকলে embed না করে separate collection ব্যবহার করুন।
Worked example — e-commerce orders
// orders collection — line items embedded (always read together, < 50 items).
// customer is referenced by _id, with a few denormalised fields for display.
{
_id: ObjectId("..."),
customer: {
_id: ObjectId("..."), // reference into customers
name: "Arif Hossain", // denormalised — order shows customer name without a join
city: "Dhaka"
},
items: [ // embedded — small, read with order
{ sku: "PHN-IPH13", qty: 1, price: 95000 }
],
payments: [ // embedded — usually 1, max 3 retries
{ method: "bKash", txId: "BKS123", at: ISODate("...") }
],
status: "PAID",
total: 95000,
placedAt: ISODate("...")
}
// reviews — separate collection. Many reviews per product, queried alone.
{
_id: ObjectId("..."),
productId: ObjectId("..."), // reference
userId: ObjectId("..."),
rating: 5,
text: "দ্রুত delivery, খুব ভালো লেগেছে।",
createdAt: ISODate("...")
}
Why this split? An order's items never make sense without the order — embed. Reviews are read on the product page without the user document — reference. The customer's name on a placed order shouldn't change retroactively if the user later edits their profile — denormalise the snapshot.
10. Practice Problems
These problems are mostly conceptual — MongoDB does not run inside this in-browser SQLite engine. Try each one on paper or on MongoDB Atlas (free tier), then expand the answer.
-
A collection has documents with different fields — one has
phone, another hasmobile. Is this allowed in MongoDB?একই collection-এ ভিন্ন ভিন্ন field থাকা কি অনুমোদিত?Show Answer
Yes — collections are schemaless by default. But for production you usually attach a
$jsonSchemavalidator so accidental typos likemobilget rejected. Schema flexibility is a feature, schema chaos is a bug.হ্যাঁ — তবে production-এ
$jsonSchemavalidator যুক্ত করে রাখাই ভালো, যাতে ভুল field name আটকে যায়। -
Write a Mongo command that finds all orders from Dhaka in August 2025 with total > 50,000 BDT, sorted newest first, returning only customer name and total.আগস্ট ২০২৫-এ ঢাকা থেকে ৫০,০০০ টাকার বেশি অর্ডার, নতুন আগে — শুধু customer name ও total।
Show Answer
ans2.mongo.jsdb.orders.find( { "customer.city": "Dhaka", placedAt: { $gte: ISODate("2025-08-01"), $lt: ISODate("2025-09-01") }, total: { $gt: 50000 } }, { "customer.name": 1, total: 1, _id: 0 } ).sort({ placedAt: -1 }); -
What does
db.orders.updateOne({_id:1}, {status:"X"})actually do?উপরের command-টি আসলে কী করে?Show Answer
It replaces the entire document with
{status:"X"}— every other field disappears, includingcustomer,items,total. The correct form is{$set:{status:"X"}}. This is the most common MongoDB beginner bug.পুরো document overwrite হয়ে যাবে — অন্য সব field মুছে যাবে। সঠিক form:
{$set:{status:"X"}}। -
Write an aggregation that returns the top 3 customers by total spend in 2025, including their order count.২০২৫ সালে সবচেয়ে বেশি খরচ করা ৩ জন customer ও তাদের order সংখ্যা।
Show Answer
ans4.mongo.jsdb.orders.aggregate([ { $match: { status: "PAID", placedAt: { $gte: ISODate("2025-01-01") } } }, { $group: { _id: "$customer._id", name: { $first: "$customer.name" }, orders: { $sum: 1 }, spend: { $sum: "$total" } } }, { $sort: { spend: -1 } }, { $limit: 3 } ]); -
Why is
$matchusually placed at the very start of an aggregation pipeline?$matchসাধারণত pipeline-এর শুরুতে কেন রাখা হয়?Show Answer
Two reasons. (1) Index use: only stages before identity-breaking stages like
$group/$unwindcan use indexes. (2) Less data: every later stage processes fewer documents, so the whole pipeline gets faster. Always filter as early — and as aggressively — as possible.কারণ — তখনই index ব্যবহার হতে পারে, এবং পরবর্তী সব stage-এ কম document প্রসেস করতে হয়।
-
A 3-node replica set is configured with
{w: "majority"}. The primary acknowledges a write to the client. Can that write be lost if the primary crashes 1 second later?primary acknowledge করার পরও কি write হারানো সম্ভব?Show Answer
No.
{w:"majority"}means the primary returned only after at least 2 of 3 nodes had the operation in their oplog. Even if the primary dies, one of the surviving secondaries already has the write and will be elected the new primary. With{w:1}the same scenario can lose the write — that is the entire trade-off.না —
w:"majority"নিশ্চিত করে কমপক্ষে ২টি নোডে write পৌঁছেছে। তাই primary মারা গেলেও write বেঁচে যায়। -
Why is
{createdAt:1}a poor shard key for a sharded write-heavy log collection?log collection-এcreatedAt-কে shard key বানালে সমস্যা কী?Show Answer
Time keeps going up — every new document falls into the highest chunk, so all writes land on one shard while the others sit idle ("hot shard"). Use
{createdAt:"hashed"}or a compound key like{tenantId:1, createdAt:1}instead.সব নতুন write একটিই shard-এ যায় — বাকি shard বসে থাকে। তাই hashed key বা compound key ব্যবহার করুন।
-
Design: a blogging platform. Posts have title, body, tags, author, and comments. Should comments be embedded?ব্লগ পোস্টে comments কি embed করা উচিত?
Show Answer
For a hobby blog with under a few hundred comments per post — yes, embed. Reading the post returns everything in one round trip. For a Facebook-scale system where a post can have a million comments — no: the 16 MB doc limit will be hit. Use the subset pattern: embed the most recent ~20 comments and store the rest in a separate
commentscollection. -
Write a TTL index that auto-deletes documents in
otp5 minutes after creation.৫ মিনিট পর OTP document-গুলো auto-delete হবে — TTL index লিখুন।Show Answer
ans9.mongo.jsdb.otp.createIndex( { createdAt: 1 }, { expireAfterSeconds: 300 } ); // Background job removes any doc whose createdAt + 300s < now. -
Compare the index
{status:1, placedAt:-1}with{placedAt:-1, status:1}for the queryfind({status:"PAID"}).sort({placedAt:-1}).দুটি compound index-এর মধ্যে কোনটি ভালো এবং কেন?Show Answer
The first one. The ESR rule: Equality, then Sort, then Range.
statusis the equality filter, so it goes first;placedAtis the sort, so it follows. The second index would force a scan of everyplacedAtjust to filterstatusafterward.ESR rule অনুযায়ী — Equality (status) আগে, তারপর Sort (placedAt)। তাই প্রথমটিই সঠিক।
-
What is the difference between
$lookupand a SQLJOIN? Name two.$lookupও SQL JOIN-এর মধ্যে দুটি পার্থক্য বলুন।Show Answer
(1)
$lookupalways behaves like a LEFT OUTER JOIN — it never drops the left document. (2) The matched documents come back as a nested array on the result, not as flat columns. (3) Performance can be much worse than a SQL join because Mongo has no global statistics — wise to design so most reads do not need$lookupat all. -
A query
find({tags: "apple"})on a 10M-doc collection takes 4 seconds. What single change brings it under 50 ms?১০ লক্ষ document-এর query ৪ সেকেন্ড লাগছে — কী করতে হবে?Show Answer
Create an index on the array field:
db.coll.createIndex({tags:1}). MongoDB automatically makes it a multikey index — one entry per array element per document — and the lookup becomes O(log n) instead of O(n). -
In a sharded cluster, why must every unique index include the shard key?sharded cluster-এ unique index-এ shard key কেন বাধ্যতামূলক?
Show Answer
Each shard validates uniqueness only against its own data — there is no cluster-wide secondary index. If the unique field is not part of the shard key, two duplicates could legitimately live on different shards and neither shard would notice.
প্রতিটি shard কেবল নিজের data-তে uniqueness যাচাই করতে পারে; পুরো cluster-জুড়ে নয়। তাই shard key যুক্ত না থাকলে dup detect হবে না।
-
When would you choose denormalisation (storing the same fact in two places) in MongoDB?কখন একই data দুই জায়গায় রাখবেন?
Show Answer
When the field is read 100× more than it is written, and the read needs it joined with another doc. Example: an order keeps a snapshot of
customer.nameat order time. Even if the user changes their name later, the historical order should still show the name on the receipt — denormalisation is correct here.যখন data বহুবার পড়া হয় কিন্তু খুব কম পরিবর্তিত হয় — এবং historical snapshot দরকার — তখন denormalisation যুক্তিসঙ্গত।
-
Aggregation: explode a per-order document so each line item gets its own row, then sum revenue per SKU.প্রতিটি line item আলাদা document বানিয়ে SKU-অনুসারে revenue বের করুন।
Show Answer
ans15.mongo.jsdb.orders.aggregate([ { $unwind: "$items" }, { $group: { _id: "$items.sku", revenue: { $sum: { $multiply: ["$items.qty", "$items.price"] } } } }, { $sort: { revenue: -1 } } ]); -
In two sentences: when should you not use MongoDB?কখন MongoDB ব্যবহার করা উচিত নয়?
Show Answer
Answer: When the workload is dominated by multi-record transactions across many entities (banking ledgers, double-entry accounting), or when complex ad-hoc analytical joins across 5+ tables are the norm. A relational engine like PostgreSQL is purpose-built for those — Mongo can do them, but slower and with more application-side complexity.
যখন বহু-entity multi-record transaction দরকার (যেমন banking) অথবা ৫+ table-এর উপর জটিল ad-hoc JOIN দরকার — তখন PostgreSQL ভালো।
Summary — Module 44
MongoDB stores data as BSON documents in collections. CRUD uses
insertOne/find/updateOne/deleteOne with rich
update operators. The aggregation pipeline ($match, $group,
$lookup, $unwind, $facet) handles everything beyond simple
filters. Indexes come in many flavours — single, compound, multikey, text, 2dsphere, partial, TTL.
Replica sets give durability and automatic failover; sharding scales
beyond one machine — but the shard-key choice is irreversible. Schema design boils down to one question:
embed (small, read-together) or reference (large, queried-alone).