Iterators, Generators & yield

ইটারেটর, জেনারেটর ও yield — lazy evaluation-এর জাদু

Read: ~30 min Advanced 5 practice problems Live code runner

1. Why Lazy Evaluation Matters

An iterator is any object that produces values one at a time, on demand. A generator is an iterator you can write as a simple function using the yield keyword. Together they let you process sequences of any length — even infinite — with constant memory. This is the foundation of modern data pipelines, streaming I/O, and most of Python's standard library of iteration tools.

Iterator এমন একটি object যা একটি করে মান dispense করে, চাহিদা অনুযায়ী। Generator হলো yield keyword দিয়ে লেখা সহজ ফাংশন-রূপের iterator। একসাথে এরা যেকোনো দৈর্ঘ্যের (এমনকি অসীম) sequence-কে constant মেমরিতে process করতে দেয়। আধুনিক data pipeline, streaming I/O এবং Python-এর অধিকাংশ iteration tool এই ভিত্তির উপর দাঁড়িয়ে।

2. The Iterator Protocol

Any object that defines __iter__() (return self) and __next__() (return the next item, raise StopIteration when done) is an iterator. for x in obj calls iter(obj) and then next() until StopIteration.

protocol.py
class Countdown:
    def __init__(self, n):
        self.n = n

    def __iter__(self):
        return self

    def __next__(self):
        if self.n <= 0:
            raise StopIteration
        self.n -= 1
        return self.n + 1

for x in Countdown(5):
    print(x, end=" ")
print()

3. Generators — A Function with yield

A function that contains yield is a generator function. Calling it does not run the body — it returns a generator object. Every time you iterate, Python runs code until the next yield, hands you the value, and freezes the function's state.

gen_fn.py
def countdown(n):
    while n > 0:
        yield n
        n -= 1

# Same behaviour as the class version, 1/3 the code
for x in countdown(5):
    print(x, end=" ")
print()

# Fibonacci — infinite generator
def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

from itertools import islice
print(list(islice(fib(), 10)))

4. yield from — Delegating to Another Iterable

yield_from.py
def chain(*iterables):
    for it in iterables:
        yield from it

print(list(chain([1, 2], (3, 4), "hi")))

# Flatten a nested structure
def flatten(lst):
    for item in lst:
        if isinstance(item, list):
            yield from flatten(item)
        else:
            yield item

print(list(flatten([1, [2, [3, 4]], [[[5]]], 6])))

5. Real-World Use: Streaming a Huge File

A generator is the right tool whenever you cannot — or do not want to — load an entire dataset into memory. The classic example is processing a large log file line by line.

stream.py
def grep(lines, needle):
    for line in lines:
        if needle in line:
            yield line

# Simulate a log file
log = ["INFO  user login", "ERROR db timeout",
       "INFO  user logout", "ERROR disk full"]

for hit in grep(log, "ERROR"):
    print(hit)
Real file version: for line in open("huge.log"): ... — file objects are themselves iterators, so you get streaming for free.

6. Vocabulary (শব্দভাণ্ডার)

TermMeaningবাংলায়
IterableAnything iter() can turn into an iterator.iter() দিয়ে iterator বানানো যায় এমন কিছু।
IteratorObject with __next__, yields one value at a time.__next__-বিশিষ্ট object, একবারে একটি মান দেয়।
GeneratorA function with yield.yield-সহ ফাংশন।
LazyCompute only when asked.চাওয়া হলে তবেই হিসাব।
StopIterationException signaling end of iteration.Iteration শেষ হওয়ার signal exception।

7. Practice Problems

  1. Write a generator that yields the first N positive even numbers.
    প্রথম N পজিটিভ জোড় সংখ্যা দেয় এমন generator লিখুন।
    ✨ Show Answer (উত্তর দেখুন)
    ans1.py
    def evens(n):
    for i in range(1, n + 1):
        yield i * 2

print(list(evens(5)))
  2. Write a generator that produces squares 1², 2², 3², ... indefinitely; use islice to take 10 of them.
    অসীম square generator লিখুন; islice দিয়ে প্রথম ১০টি নিন।
    ✨ Show Answer (উত্তর দেখুন)
    ans2.py
    from itertools import islice

def squares():
    n = 1
    while True:
        yield n * n
        n += 1

print(list(islice(squares(), 10)))
  3. Use a generator to find the first 5 multiples of 7 greater than 100.
    ১০০-এর চেয়ে বড় ৭-এর প্রথম ৫টি multiple generator দিয়ে বের করুন।
    ✨ Show Answer (উত্তর দেখুন)
    ans3.py
    from itertools import islice

def multiples_of(k, start=0):
    n = start
    while True:
        n += 1
        if n % k == 0:
            yield n

print(list(islice(multiples_of(7, 100), 5)))
  4. Explain the difference between a list comprehension and a generator expression in one sentence each.
    এক বাক্যে বলুন — list comprehension ও generator expression-এর পার্থক্য।
    ✨ Show Answer (উত্তর দেখুন)

    Answer: A list comprehension builds the entire list in memory up front; a generator expression produces items one at a time on demand and keeps only the current item in memory.

    List comprehension পুরো list আগে মেমরিতে বানিয়ে ফেলে; generator expression চাহিদামতো একটা একটা করে item তৈরি করে — মেমরিতে একবারে শুধু একটি মান থাকে।

  5. Use yield from to write a generator that flattens a list of lists one level deep.
    yield from দিয়ে এমন generator লিখুন যা এক-স্তরের list-of-lists flatten করে।
    ✨ Show Answer (উত্তর দেখুন)
    ans5.py
    def flatten1(lists):
    for lst in lists:
        yield from lst

print(list(flatten1([[1, 2], [3, 4], [5]])))

Summary — Module 18

Iterators produce values one at a time via the __iter__/__next__ protocol. Generators are the easy way to make iterators using yield. They enable lazy evaluation — process streams of any size with constant memory. yield from delegates cleanly between generators. Master them and you unlock Python's most elegant data processing style.

Iterator __iter__/__next__ protocol দিয়ে এক-একটি মান দেয়। Generator হলো yield দিয়ে সহজে iterator বানানোর উপায় — lazy evaluation দিয়ে যেকোনো আকারের stream constant মেমরিতে process। এই দক্ষতা Python-এর সবচেয়ে elegant data processing style খুলে দেয়।

Next Module → Midterm Project — একটি বাস্তব CLI টুল তৈরি করুন।

Previous: Comprehensions Next: Midterm Project