16. Generators#
In this notebook, we cover the following subjects:
Understanding Generators;
Generator Functions;
Generator Expressions;
Factories.
# To enable type hints for lists, dicts, tuples, and sets we need to import the following:
from typing import List, Dict, Tuple, Set
16.1. Understanding Generators#
Generators are a powerful feature in Python that allow you to iterate over data efficiently without taking up unnecessary memory. Unlike lists, which hoard all their elements in memory, generators produce items one by one, as needed. This technique, called "lazy evaluation", makes generators incredibly efficient when working with large datasets. Once the last value is yielded, the generator stops, and it cannot be used again unless it is reinitialized or recreated. Think of generators as the chefs of the programming world—they prepare each dish just in time, saving resources compared to preparing everything in advance.
16.1.1. Generator Syntax#
There are two main ways to create a generator in Python:
Creating a Generator Function:
These functions use the yield keyword to return values one at a time. Every time yield is encountered, the state of the function is preserved, making it ideal for producing a series of values without storing them all at once.
Generator Expressions:
These are similar to list comprehensions but use parentheses instead of square brackets. Generator expressions offer a concise way to build generators in one line — think of them as the compact sports car compared to the family minivan of list comprehensions.
16.2. Generator Functions#
16.2.1. Creating a Generator Function#
A generator function looks very similar to a regular function but uses yield instead of return. The magic of yield is that it allows the function to remember where it left off. This is perfect for scenarios where you need a stream of data but want to keep resource usage light.
16.2.2. Using Generators#
Let’s see an example of a generator function:
def count_up_to(max_value: int):
count = 1
while count <= max_value:
yield count
count += 1
This generator function starts counting from 1 up to max_value. Every time it encounters yield, it produces the current count and pauses until the next value is requested.
Generators are most often used with loops, which consume the values one at a time:
for number in count_up_to(5):
print(number)
1
2
3
4
5
This will print numbers from 1 to 5. Notice how generators can save memory, especially if max_value were very large—you don’t need to hold all those numbers in memory at once.
16.3. Generator Expressions#
If you love list comprehensions, you’re going to enjoy generator expressions. They look just like list comprehensions, but use parentheses to keep things efficient:
squares = (x * x for x in range(10))
This generator expression will yield the squares of numbers from 0 to 9, one at a time. To see the values, we can loop through them:
for square in squares:
print(square)
0
1
4
9
16
25
36
49
64
81
16.3.1. Examples of Generator Expressions#
Generator expressions are particularly handy for tasks like calculating sums or filtering values without the overhead of creating a full list:
sum_of_squares = sum(x * x for x in range(1000))
Instead of building a list of 1,000 squares, the generator computes each square on the fly, saving memory — especially important if you’re working with a large range().
print(f'The sum of squares between 0 and 999 is : {sum_of_squares}')
The sum of squares between 0 and 999 is : 332833500
16.3.2. any and all#
Generators are an ideal companion for Python’s any() and all() functions, especially when working with conditions across large datasets. Imagine checking if any value in a large collection is negative or if all values are positive. Instead of creating a giant list, let the generator handle each element one by one.
any(): This function returns True if at least one element of the iterable is True. If the generator finds even one value that meets the condition, it stops and returns True immediately.
all(): This function returns True only if all elements of the iterable are True. If it encounters a False value, it stops and returns False immediately.
Let’s see an example:
numbers = range(-10, 10)
is_any_negative = any(num < 0 for num in numbers)
is_all_positive = all(num > 0 for num in numbers)
print(is_any_negative)
print(is_all_positive)
True
False
Using any() and all() with generator expressions keeps your code efficient and clean—no cluttered lists, just direct checks.
16.4. Factories#
Factories in Python are functions that return generators, allowing you to create a pipeline of values on demand. Think of a factory as a workshop: it has a blueprint (your function), and each time you use it, it produces a unique item (the generator).
For instance:
def fibonacci_factory():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
Calling fibonacci_factory() gives you a generator capable of producing Fibonacci numbers indefinitely:
fib = fibonacci_factory()
for _ in range(10):
print(next(fib))
0
1
1
2
3
5
8
13
21
34
This factory generates as many Fibonacci numbers as you need, without storing all of them—it’s a production line ready to roll out values on request.
16.5. Exercises#
Let’s practice! Mind that each exercise is designed with multiple levels to help you progressively build your skills. Level 1 is the foundational level, designed to be straightforward so that everyone can successfully complete it. In Level 2, we step it up a notch, expecting you to use more complex concepts or combine them in new ways. Finally, in Level 3, we get closest to exam level questions, but we may use some concepts that are not covered in this notebook. However, in programming, you often encounter situations where you’re unsure how to proceed. Fortunately, you can often solve these problems by starting to work on them and figuring things out as you go. Practicing this skill is extremely helpful, so we highly recommend completing these exercises.
For each of the exercises, make sure to add a docstring and type hints, and do not import any libraries unless specified otherwise.
16.5.1. Exercise 1#
Level 1: Create a generator function called even_numbers_up_to_n(n) that yields all even numbers from 0 to n. Remember to add type hints and a descriptive docstring.
Example input:
n = 6
Example output:
0
2
4
6
# TODO
Level 2: Write a generator function called filtered_numbers(numbers, condition) that yields numbers from the given iterable numbers only if they meet the specified condition function. This should be a versatile function that can be used for any condition.
Example input:
numbers: List[int] = range(10)
def is_odd(x):
return x % 2 != 0
condition = is_odd # Filter odd numbers
Example output:
1
3
5
7
9
# TODO
Level 3: Write a function called nested_generators() that yields three different generator functions: one that yields numbers from 0 to 2, another that yields numbers from 3 to 5, and another that yields numbers from 6 to 8. Then, write a loop to iterate over these generators to display their values.
Example output:
0
1
2
3
4
5
6
7
8
# TODO
16.5.2. Exercise 2#
Level 1: Create a generator function called countdown(n) that yields numbers starting from n down to 0.
Example input:
n = 5
Example output:
5
4
3
2
1
0
# TODO.
Level 2: Write a generator function called infinite_multiples(base) that yields multiples of the given base value indefinitely. Use this generator to produce a potentially infinite sequence of numbers.
Example input:
base = 3
Example output:
3
6
9
12
15
# TODO.
Level 3: Create a generator function called prime_numbers() that yields an infinite sequence of prime numbers, starting from 2. Use efficient logic to determine whether each number is prime.
Example output:
2
3
5
7
11
13
# TODO.
16.5.3. Exercise 3#
Level 1: Write a generator function called repeat_character(char, times) that yields a given character char a specified number of times.
Example input:
char = '*'
times = 5
Example output:
*
*
*
*
*
# TODO.
Level 2: Write a generator function called uppercase_characters(text) that yields each character from the input string text in uppercase.
Example input:
text = 'hello'
Example output:
H
E
L
L
O
# TODO.
Level 3: Write a generator function called christmas_tree(levels) that yields a string representation of a Christmas tree with the given number of levels. Each level should be wider than the one above, creating the classic triangular tree shape.
Example input:
levels = 5
Example output:
*
***
*****
*******
*********
# TODO.
Material for the VU Amsterdam course “Introduction to Python Programming” for BSc Artificial Intelligence students. These notebooks are created using the following sources:
Learning Python by Doing: This book, developed by teachers of TU/e Eindhoven and VU Amsterdam, is the main source for the course materials. Code snippets or text explanations from the book may be used in the notebooks, sometimes with slight adjustments.