Solving Recurrences: Master Theorem & Beyond

Recurrence সমাধান — Master Theorem

Read: ~40 min Advanced 6 practice problems Live code runner

1. What is a Recurrence?

A recurrence relation expresses the running time T(n) of a recursive algorithm in terms of the running time of its smaller subproblems. For merge sort: T(n) = 2·T(n/2) + Θ(n) — split into two halves (2 sub-calls of size n/2), plus Θ(n) work to merge.

একটি recursive অ্যালগরিদমের চলার সময় T(n) যখন তার ছোট সাব-প্রবলেমগুলোর সময়ের ভিত্তিতে প্রকাশ করা হয়, সেটিই recurrence relation। merge sort-এর জন্য: T(n) = 2·T(n/2) + Θ(n)।

2. The Three Methods

Substitution — guess a closed form, prove by induction.
Recursion tree — draw the tree, sum work per level.
Master Theorem — pattern-match recurrences of the form T(n) = a·T(n/b) + f(n).

তিনটি পদ্ধতি — substitution (অনুমান + ইনডাকশন), recursion tree (গাছ এঁকে যোগ), এবং Master Theorem (প্যাটার্ন মিলিয়ে সরাসরি উত্তর)।

3. Recursion Tree for T(n) = 2T(n/2) + n

Figure 5.1 — Each level does Θ(n) work; there are log₂ n levels. Total Θ(n log n).

4. The Master Theorem

For T(n) = a·T(n/b) + f(n) with a ≥ 1, b > 1, define c = log_b(a):

Case	Condition	T(n)
1	f(n) = O(n^c⁻ε) for some ε > 0	Θ(n^c)
2	f(n) = Θ(n^c · log^k n)	Θ(n^c · log^(k+1) n)
3	f(n) = Ω(n^c+ε), and a·f(n/b) ≤ k·f(n) for k < 1	Θ(f(n))

Quick read Compare f(n) to n^c. If f is smaller → Case 1 → answer is n^c. If equal (with possibly a log) → Case 2 → answer gets one extra log. If f is bigger → Case 3 → answer is just f(n).
f(n) ছোট → Case 1 → n^c। f(n) সমান → Case 2 → একটি অতিরিক্ত log। f(n) বড় → Case 3 → f(n) নিজেই।

5. Worked Examples

Recurrence	a, b, c	Case	T(n)
2T(n/2) + n (merge sort)	2, 2, 1	2	Θ(n log n)
T(n/2) + 1 (binary search)	1, 2, 0	2	Θ(log n)
3T(n/2) + n (Karatsuba)	3, 2, log₂3≈1.585	1	Θ(n^1.585)
4T(n/2) + n	4, 2, 2	1	Θ(n²)
2T(n/2) + n²	2, 2, 1	3	Θ(n²)
8T(n/2) + n² (naive matrix mul)	8, 2, 3	1	Θ(n³)

একই pattern — শুধু a, b, এবং f(n)-এর তুলনা। মাত্র ৫ সেকেন্ডে যেকোনো D&C-এর complexity বের করতে পারবেন।

6. Verify with Real Code

Run merge sort on increasing n and observe that comparisons grow as ~n log₂ n.

verify_nlogn.cpp

#include <bits/stdc++.h>
using namespace std;
long long cmp = 0;

void merge(vector<int>& a, int l, int m, int r) {
    vector<int> t; t.reserve(r-l);
    int i=l, j=m;
    while (i<m && j<r) {
        cmp++;
        if (a[i] <= a[j]) t.push_back(a[i++]);
        else t.push_back(a[j++]);
    }
    while (i<m) t.push_back(a[i++]);
    while (j<r) t.push_back(a[j++]);
    for (int k=0; k<(int)t.size(); k++) a[l+k]=t[k];
}
void ms(vector<int>& a, int l, int r) {
    if (r-l<2) return;
    int m = (l+r)/2;
    ms(a,l,m); ms(a,m,r); merge(a,l,m,r);
}
int main() {
    mt19937 rng(42);
    for (int n : {1024, 2048, 4096, 8192}) {
        vector<int> a(n);
        for (auto& x : a) x = rng();
        cmp = 0;
        ms(a, 0, n);
        double ratio = (double)cmp / (n * log2(n));
        cout << "n=" << n << " cmp=" << cmp
             << " cmp/(n log n)=" << ratio << "\n";
    }
}

The ratio cmp / (n log n) stays roughly constant — empirical confirmation of Θ(n log n).

7. Akra-Bazzi (Preview)

Master Theorem fails for unequal splits like T(n) = T(n/3) + T(2n/3) + n. The Akra-Bazzi method handles such cases. For now: just memorise that this particular recurrence also resolves to Θ(n log n) — same as merge sort with uneven halves.

অসমান split-এর recurrence-এর জন্য Master Theorem কাজ করে না। তখন Akra-Bazzi method লাগে। তবে অধিকাংশ ICPC সমস্যায় Master Theorem-ই যথেষ্ট।

8. Practice Problems

Solve T(n) = 4T(n/2) + n.

T(n) = 4T(n/2) + n — সমাধান করুন।

✨ Show Answer (উত্তর দেখুন)

Answer: a=4, b=2, c=log₂4=2. f(n)=n=O(n²⁻¹). Case 1 → T(n) = Θ(n²).
Solve T(n) = 2T(n/2) + n log n.

T(n) = 2T(n/2) + n log n — সমাধান করুন।

✨ Show Answer (উত্তর দেখুন)

Answer: a=2, b=2, c=1. f(n)=n log n=Θ(n¹·log¹n) → Case 2 with k=1 → T(n)=Θ(n log² n).
Solve T(n) = 2T(n−1) + 1 (NOT a Master-Theorem recurrence).

T(n) = 2T(n−1) + 1 — Master-Theorem-এর form নয়।

✨ Show Answer (উত্তর দেখুন)

Answer: Substitution gives 1+2+4+…+2ⁿ = 2^(n+1)−1, so T(n)=Θ(2ⁿ). This is the Tower of Hanoi recurrence.
Solve T(n) = 7T(n/2) + n² (Strassen's matrix mul).

T(n) = 7T(n/2) + n² — Strassen-এর matrix multiplication।

✨ Show Answer (উত্তর দেখুন)

Answer: a=7, b=2, c=log₂7≈2.807. f(n)=n²=O(n^(2.807−ε)). Case 1 → T(n)=Θ(n^2.807).
Solve T(n) = T(n/2) + n.

T(n) = T(n/2) + n — সমাধান।

✨ Show Answer (উত্তর দেখুন)

Answer: a=1, b=2, c=0. f(n)=n=Ω(n^(0+ε)) and 1·(n/2) = n/2 = ½·f(n) → Case 3 → T(n)=Θ(n).

Write a memoised C++ function T(n) that follows T(n) = 2T(n/2) + n with T(1) = 1, and confirm experimentally that T(2^k) ≈ k · 2^k.

উপরের recurrence কোডে memoise করে দেখুন: T(2^k) ≈ k·2^k।

✨ Show Answer (উত্তর দেখুন)

a6.cpp

#include <bits/stdc++.h>
using namespace std;
unordered_map<long long,long long> M;
long long T(long long n) {
    if (n <= 1) return 1;
    if (M.count(n)) return M[n];
    return M[n] = 2 * T(n/2) + n;
}
int main() {
    for (int k=1; k<=12; k++) {
        long long n = 1LL << k;
        cout << "T(" << n << ") = " << T(n)
             << "  vs k·2^k = " << (long long)k*n << "\n";
    }
}

Summary — Module 05

Master Theorem turns recurrences of the form T(n) = a·T(n/b) + f(n) into closed-form complexities. Compare f(n) to n^(log_b a) and read off Case 1, 2, or 3. For uneven splits, fall back to recursion-tree or Akra-Bazzi. This finishes Phase 1 — you now have the math to analyse anything.

Master Theorem-এর তিনটি case মুখস্থ থাকলেই 90% D&C recurrence এক ঝলকেই সমাধান হয়ে যায়। Phase 1 শেষ — এখন আমরা data structure-এ ঢুকব।

Next Module → Arrays: Static, Dynamic & Multi-dimensional — Phase 2 শুরু।

Module 04 Next: Arrays