Asymptotic Analysis: Big-O, Θ, Ω
অ্যাসিম্পটোটিক বিশ্লেষণ — Big-O, Θ, Ω
1. Why "Asymptotic" At All?
We never measure an algorithm's speed in seconds, because seconds depend on the CPU, the compiler, the operating system, even the room temperature. Instead, we measure how the number of operations grows as the input size n grows. That growth function is what asymptotic notation captures.
That single fact is why binary search, balanced BSTs, segment trees and merge sort dominate modern computing.
2. Formal Definitions
f(n) = O(g(n)) iff there exist constants c > 0 and n₀ ≥ 0 such that
0 ≤ f(n) ≤ c · g(n) for all n ≥ n₀.
f(n) = Ω(g(n)) iff ∃ c > 0, n₀ ≥ 0 such that 0 ≤ c · g(n) ≤ f(n) for all n ≥ n₀.
f(n) = Θ(g(n)) iff f(n) = O(g(n)) and f(n) = Ω(g(n)).
3. The Growth Hierarchy
Memorise this order — bigger functions on the right dominate everything to their left for large n:
O(1) ⊂ O(log n) ⊂ O(√n) ⊂ O(n) ⊂ O(n log n) ⊂ O(n²) ⊂ O(n³) ⊂ O(2ⁿ) ⊂ O(n!)
4. Worst, Average, Best Case
For the same algorithm and the same n, runtime can vary wildly depending on the actual input. Linear search for example:
- Best: target is the first element → 1 comparison → Θ(1)
- Worst: target absent or last → n comparisons → Θ(n)
- Average: uniformly random target → ~n/2 comparisons → Θ(n)
মনে রাখার নিয়ম — ICPC বা competitive programming-এ আমরা প্রায় সবসময় worst case ধরেই হিসাব করি, কারণ judge-এর adversarial input থাকে।
5. Live Demo — Timing Naive Sum vs Gauss Formula
We'll compute 1 + 2 + … + n two ways and time each with <chrono>.
One is O(n), the other is O(1). Watch the difference for n = 100,000,000.
#include <bits/stdc++.h>
using namespace std;
using namespace chrono;
int main() {
long long n = 100000000; // 10^8
auto t1 = high_resolution_clock::now();
long long s1 = 0;
for (long long i = 1; i <= n; i++) s1 += i;
auto t2 = high_resolution_clock::now();
auto t3 = high_resolution_clock::now();
long long s2 = n * (n + 1) / 2;
auto t4 = high_resolution_clock::now();
auto ms_loop = duration_cast<microseconds>(t2 - t1).count();
auto ms_gauss = duration_cast<microseconds>(t4 - t3).count();
cout << "loop sum = " << s1 << " in " << ms_loop << " us\n";
cout << "gauss sum = " << s2 << " in " << ms_gauss << " us\n";
return 0;
}6. Pattern Recognition — Read Complexity by Inspection
The fastest way to estimate complexity is to recognise patterns. Here are the seven you must memorise:
| Pattern | Complexity | Example |
|---|---|---|
| Single loop 1..n | O(n) | for (i=0; i<n; i++) |
| Two nested loops 1..n | O(n²) | two-sum brute force |
| Loop halving each step | O(log n) | for (i=n; i>0; i/=2) |
| Loop multiplying by 2 | O(log n) | for (i=1; i<=n; i*=2) |
| Sort then linear scan | O(n log n) | sort + sweep |
| Recursion: T(n)=2T(n/2)+n | O(n log n) | merge sort |
| Recursion: T(n)=2T(n-1)+1 | O(2ⁿ) | Tower of Hanoi |
7. Common Pitfalls
⚠️ Mistakes Beginners Make
- Saying O(2n) — drop constants, it's O(n).
- Saying O(n + n²) — keep the dominant term: O(n²).
- Confusing O(log n) base — bases differ by a constant, dropped.
- Ignoring hidden costs of
stringconcat orvectorcopy. - Forgetting that
set::insertis O(log n), not O(1).
✅ Pro Habits
- Always state the complexity in your own head before you submit.
- Plug n into the formula, compare with 10⁸.
- Track both time and memory — both can TLE/MLE.
- Use Θ when you can prove tight bounds; otherwise O.
8. Concrete Numbers For n
| Function | n=10 | n=100 | n=10⁴ | n=10⁶ |
|---|---|---|---|---|
| O(1) | 1 | 1 | 1 | 1 |
| O(log n) | 3 | 7 | 14 | 20 |
| O(n) | 10 | 100 | 10⁴ | 10⁶ |
| O(n log n) | 33 | 664 | 1.3×10⁵ | 2×10⁷ |
| O(n²) | 100 | 10⁴ | 10⁸ | 10¹² 💀 |
| O(2ⁿ) | 1024 | 10³⁰ 💀 | — | — |
| O(n!) | 3.6×10⁶ | 10¹⁵⁸ 💀 | — | — |
9. Practice Problems
For each snippet, give the tightest Θ-bound. Try first, then check.
-
for (i=1; i<=n; i++) for (j=1; j<=n; j++) sum++;উপরের nested loop-এর complexity কত?✨ Show Answer (উত্তর দেখুন)
Θ(n²) — outer loop n times, inner loop n times each → n × n.
-
for (i=1; i<=n; i++) for (j=i; j<=n; j++) sum++;দ্বিতীয় loop i থেকে শুরু — তখন complexity কত?✨ Show Answer (উত্তর দেখুন)
Θ(n²). Total iterations = n + (n-1) + (n-2) + … + 1 = n(n+1)/2 ≈ n²/2. Constants are dropped.
মোট iteration = n + (n-1) + … + 1 = n(n+1)/2। constant বাদ দিলে Θ(n²)।
-
for (i=1; i<=n; i*=2) work();i প্রতিবার দ্বিগুণ — complexity কত?✨ Show Answer (উত্তর দেখুন)
Θ(log n). i takes values 1, 2, 4, …, n — that's log₂ n values.
-
for (i=1; i<=n; i++) for (j=1; j<=n; j*=2) work();বাইরের loop linear, ভেতরের loop double হচ্ছে — মোট কত?✨ Show Answer (উত্তর দেখুন)
Θ(n log n). Outer = n, inner = log n. Multiply.
-
for (i=1; i*i<=n; i++) work();i² ≤ n পর্যন্ত চলে — complexity?✨ Show Answer (উত্তর দেখুন)
Θ(√n). The loop stops when i ≈ √n.
-
Two-pointer technique: i moves only forward, j moves only forward, both bounded by n. Total work?Two-pointer পদ্ধতি — দুটি pointer-ই কেবল সামনে যায় এবং দুজনই সর্বোচ্চ n বার সরে। মোট কাজ?
✨ Show Answer (উত্তর দেখুন)
Θ(n). Even though there are nested-looking loops, each pointer advances at most n times total — the work is amortised over both, giving 2n = Θ(n).
দুটি pointer একসাথে সর্বোচ্চ 2n বার সরবে — তাই মোট Θ(n)।
-
Predict whether 10⁸ operations fit inside a 1-second time limit.10⁸ operation কি ১ সেকেন্ডের time limit-এর মধ্যে শেষ হবে?
✨ Show Answer (উত্তর দেখুন)
Borderline. A modern CPU executes ~10⁸ simple ops/sec. Tight loops with primitive ints — yes. Loops with string operations, map lookups, or modular arithmetic — usually no, expect TLE.
CPU প্রতি সেকেন্ডে আনুমানিক 10⁸ অপারেশন করতে পারে। শুধু int দিয়ে tight loop হলে চলে যাবে; কিন্তু string বা map থাকলে TLE হবে।
-
Live: count how many times the inner statement runs in the snippet below for n = 1000.নিচের কোডে n = 1000-এ inner statement কতবার চলে?
✨ Show Answer (উত্তর দেখুন)
count.cpp#include <bits/stdc++.h> using namespace std; int main() { int n = 1000; long long cnt = 0; for (int i = 1; i <= n; i++) for (int j = 1; j <= n; j *= 2) cnt++; cout << "runs = " << cnt << "\n"; cout << "theory n*log2(n) = " << n * (long long)ceil(log2(n + 1)) << "\n"; return 0; }Approx 1000 × 10 = 10,000 iterations — Θ(n log n).
Summary — Module 02
Big-O upper-bounds growth, Big-Ω lower-bounds it, and Big-Θ pins it down. Drop constants and lower-order terms; keep only the dominant term. Recognise the seven canonical patterns (single loop, nested loop, halving loop, two-pointer, sort+sweep, divide-and-conquer recursion, exponential recursion) and you can call complexity by sight.