Python Iterators

In this tutorial, we will explore the concept of iterators in Python. An iterator is an object that allows you to traverse through a sequence (like a list or tuple) one element at a time. Understanding iterators is crucial for working with large datasets efficiently and for creating custom data structures.

Introduction

Iterators are essential in Python, especially when dealing with large collections of data. They allow you to iterate over elements without loading the entire dataset into memory at once. This makes them particularly useful for handling infinite sequences or very large files.

In this tutorial, we will cover:

• The iterator protocol: __iter__() and __next__()
• Creating custom iterators
• Using the built-in iter() and next() functions

The Iterator Protocol

An iterator in Python is an object that implements two special methods: __iter__() and __next__(). These methods define how an object can be iterated over.

__iter__() Method

The __iter__() method returns the iterator object itself. This method is called when you use the built-in iter() function on an iterable object.

__next__() Method

The __next__() method returns the next item in the sequence. If there are no more items, it raises a StopIteration exception.

Creating Custom Iterators

To create a custom iterator, you need to define a class that implements both __iter__() and __next__() methods.

Example 1: Simple Iterator for a List

Let's create an iterator that iterates over a list of numbers.

1class NumberIterator:
2  def __init__(self, numbers):
3      self.numbers = numbers
4      self.index = 0
5 
6  def __iter__(self):
7      return self
8 
9  def __next__(self):
10      if self.index < len(self.numbers):
11          result = self.numbers[self.index]
12          self.index += 1
13          return result
14      else:
15          raise StopIteration
16 
17# Usage
18numbers = [1, 2, 3, 4, 5]
19iterator = NumberIterator(numbers)
20 
21for number in iterator:
22  print(number)

1
2
3
4
5

Example 2: Infinite Iterator

Now, let's create an infinite iterator that generates consecutive integers.

1class CountIterator:
2  def __init__(self, start=0):
3      self.current = start
4 
5  def __iter__(self):
6      return self
7 
8  def __next__(self):
9      result = self.current
10      self.current += 1
11      return result
12 
13# Usage
14counter = CountIterator(1)
15 
16for _ in range(5):
17  print(next(counter))

1
2
3
4
5

Using Built-in iter() and next() Functions

Python provides two built-in functions, iter() and next(), that are used to work with iterators.

iter() Function

The iter() function returns an iterator object for the given iterable. It is equivalent to calling the __iter__() method on the iterable.

1numbers = [1, 2, 3, 4, 5]
2iterator = iter(numbers)
3 
4for number in iterator:
5  print(number)

1
2
3
4
5

next() Function

The next() function returns the next item from the iterator. If there are no more items, it raises a StopIteration exception.

1numbers = [1, 2, 3, 4, 5]
2iterator = iter(numbers)
3 
4while True:
5  try:
6      print(next(iterator))
7  except StopIteration:
8      break

1
2
3
4
5

Practical Example

Let's create a practical example where we use an iterator to read lines from a large file one at a time, without loading the entire file into memory.

1class FileIterator:
2  def __init__(self, filename):
3      self.file = open(filename, 'r')
4 
5  def __iter__(self):
6      return self
7 
8  def __next__(self):
9      line = self.file.readline()
10      if not line:
11          self.file.close()
12          raise StopIteration
13      return line.strip()
14 
15# Usage
16filename = 'large_file.txt'
17file_iterator = FileIterator(filename)
18 
19for line in file_iterator:
20  print(line)

Summary

• Iterator Protocol: An iterator must implement __iter__() and __next__() methods.
• Custom Iterators: Define a class with these methods to create custom iterators.
• Built-in Functions: Use iter() to get an iterator and next() to retrieve the next item.

What's Next?

In the next tutorial, we will explore Python Generators, which provide a simpler way to create iterators. Generators use the yield keyword to produce values one at a time, making them easier to work with compared to manually implementing the iterator protocol.

Stay tuned for more advanced Python topics!