Iterators and Generators in python

Iterators in python:  Let's start discussion about python iterator with a simple python program involving for loop.

mylist = [1,4,6,7,98]
for x in mylist:
    print x 

Sample output: 1,4,6,7,98

The sample program is self explanatory. Using for loop we are iterating mylist and display its elements. Similarly other basic container types tuple and set can be iterated. Lets understand the
iteration mechanism which python is doing for us.
In Python, the mechanism for iteration is based upon iterator and  iterable.
An iterator is an object that manages an iteration using next() method, here x is a iterator and an iterable is an object that produces an iterator via the syntax iter(obj), mylist is iterable object.

In for loop also, Python has used the next() and iter() method and automated the process to display the elements of list.
Python performs the following two steps:(iterator protocol)
1. Gets an iterator for mylist:
     Call iter(mylist) -> this returns an object with a next() method (or __next__() in Python 3).
2. Uses the iterator to loop over items:
   Keep calling next() method and store the elements in x, until an exception StopIteration is raised
   from within next() which indicate no more element is left to iterated.

What makes mylist iterable?:  It is iterator protocol which is implemented by mylist(list class).

Along with these basic container types, instance of user defined class can be made iterable by implementing itr() which returns an iterator.An iterator is an object with a next() method.

Generators in python(yield() method): Generators are iterators, but you can only iterate over them once. It's because they do not store all the values in memory, they generate the values on the fly.
A generator function produces a sequence of results instead of a single value. Python uses yield keyword to return generator object.
Note :
1. When a generator function is called first time , it returns an generator object without even 
     beginning execution of the function.
2. When next() method is called for the first time, the function starts executing until it reaches yield
     statement. The yielded value is returned by the next call.
Lets write a sample code  to understand generators in python:Open IDLE and create a file and copy following code line in it. 

def counter(n):
    i = 0
    while i < n:
        yield i

        print "incremented"
        i += 1

  
#call above function and display values using next()
y= counter(3)
print y

print("First call")
print y.next()

print("Second call")
print y.next()
print("Third call")

print y.next()

print("Fourth call")

print y.next()#exception occurred here

Sample output:
-----------------------------------------------------------------
<generator object yrange at 0x02C2B2B0>
First call
0
Second call
incremented
1
Third call
incremented
2
Fourth call
incremented

Traceback (most recent call last):
  File "C:/Python27/test5.py", line 48, in <module>
    print y.next()#exception occured here
StopIteration
-------------------------------------------------------------------
We will execute the above code lines line by line. When function yrange() is executed the very first time, function return a generator object. Display the vale of y in terminal:
>>> y
<generator object yrange at 0x02DDB2B0>  #generator object is returned 
On calling next() the very first time on returned object earlier, runs function yrange() from the beginning until it hits yield and returns initialized value of i,  i.e ,0.
>>> y.next() #First call
0
In second call of y.next(), code lines after yield is executed and value of i is incremented.Again while loop start execution and returns current value of  i when it reaches at yield.
>>> y.next()  #Second call
incremented
1
Similarly,on subsequent call it displays 2 and 3. When fourth time next() is called then StopIteration exception is raised and same is displayed in output. The generator is considered empty once the function runs but does not hit yield any more.

Generator Expressions:-
Generators can be coded succinctly as expressions using a syntax similar to list comprehensions using braces () instead of bracket [] .
[i*i for i in range(10)] is list comprehensions  and (i*i for i in range(10)) is a generator.
Note :
1. generator expressions used as arguments to various functions that consume iterators.
    >>> sum(((x*x for x in range(10)))
    285
2. When there is only one argument to the calling function, the parenthesis around generator 
    expression can be omitted.
    >>> sum(x*x for x in range(10))
    285

---

Previous : Exception handling in python Next : Object oriented programming in python