INTRODUCTION TO PYTHON

BY — RAVI SHANKAR

1. Python Modules

What are modules in Python?

Modules refer to a file containing Python statements and definitions.

A file containing Python code, for example: example.py, is called a module, and its module name would be example.

We use modules to break down large programs into small manageable and organized files. Furthermore, modules provide reusability of code.

We can define our most used functions in a module and import it, instead of copying their definitions into different programs.

Let us create a module. Type the following and save it as example.py.

# Python Module example
def add(a, b):
   """This program adds two
   numbers and return the result"""
   result = a + b
   return result

Here, we have defined a function add() inside a module named example. The function takes in two numbers and returns their sum.

How to import modules in Python?

We can import the definitions inside a module to another module or the interactive interpreter in Python.

We use the import keyword to do this. To import our previously defined module example, we type the following in the Python prompt.

>>> import example

This does not import the names of the functions defined in example directly in the current symbol table. It only imports the module name example there.

Using the module name we can access the function using the dot . operator. For example:

>>> example.add(4,5.5)
9.5

Python has tons of standard modules. You can check out the full list of Python standard modules and their use cases. These files are in the Lib directory inside the location where you installed Python.

Standard modules can be imported the same way as we import our user-defined modules.

There are various ways to import modules. They are listed below..

Python import statement

We can import a module using the import statement and access the definitions inside it using the dot operator as described above. Here is an example.

# import statement example
# to import standard module math
import math
print("The value of pi is", math.pi)

When you run the program, the output will be:

The value of pi is 3.141592653589793

Import with renaming

We can import a module by renaming it as follows:

# import module by renaming it
import math as m
print("The value of pi is", m.pi)

We have renamed the math module as m. This can save us typing time in some cases.

Note that the name math is not recognized in our scope. Hence, math.pi is invalid, and m.pi is the correct implementation.

Python from…import statement

We can import specific names from a module without importing the module as a whole. Here is an example.

# import only pi from math module
from math import pi
print("The value of pi is", pi)

Here, we imported only the pi attribute from the math module.

In such cases, we don’t use the dot operator. We can also import multiple attributes as follows:

>>> from math import pi, e
>>> pi
3.141592653589793
>>> e
2.718281828459045

Import all names

We can import all names(definitions) from a module using the following construct:

# import all names from the standard module math
from math import *
print("The value of pi is", pi)

Here, we have imported all the definitions from the math module. This includes all names visible in our scope except those beginning with an underscore(private definitions).

Importing everything with the asterisk (*) symbol is not a good programming practice. This can lead to duplicate definitions for an identifier. It also hampers the readability of our code.

Python Module Search Path

While importing a module, Python looks at several places. Interpreter first looks for a built-in module. Then(if built-in module not found), Python looks into a list of directories defined in sys.path. The search is in this order.

• The current directory.
• PYTHONPATH (an environment variable with a list of directories).
• The installation-dependent default directory.

>>> import sys
>>> sys.path
['',
'C:\\Python33\\Lib\\idlelib',
'C:\\Windows\\system32\\python33.zip',
'C:\\Python33\\DLLs',
'C:\\Python33\\lib',
'C:\\Python33',
'C:\\Python33\\lib\\site-packages']

We can add and modify this list to add our own path.

Reloading a module

The Python interpreter imports a module only once during a session. This makes things more efficient. Here is an example to show how this works.

Suppose we have the following code in a module named my_module.

# This module shows the effect of
#  multiple imports and reload
print("This code got executed")

Now we see the effect of multiple imports.

>>> import my_module
This code got executed
>>> import my_module
>>> import my_module

We can see that our code got executed only once. This goes to say that our module was imported only once.

Now if our module changed during the course of the program, we would have to reload it.One way to do this is to restart the interpreter. But this does not help much.

Python provides a more efficient way of doing this. We can use the reload() function inside the imp module to reload a module. We can do it in the following ways:

>>> import imp
>>> import my_module
This code got executed
>>> import my_module
>>> imp.reload(my_module)
This code got executed
<module 'my_module' from '.\\my_module.py'>

The dir() built-in function

We can use the dir() function to find out names that are defined inside a module.

For example, we have defined a function add() in the module example that we had in the beginning.

We can use dir in example module in the following way:

>>> dir(example)
['__builtins__',
'__cached__',
'__doc__',
'__file__',
'__initializing__',
'__loader__',
'__name__',
'__package__',
'add']

Here, we can see a sorted list of names (along with add). All other names that begin with an underscore are default Python attributes associated with the module (not user-defined).

For example, the __name__ attribute contains the name of the module.

>>> import example
>>> example.__name__
'example'

All the names defined in our current namespace can be found out using the dir() function without any arguments.

>>> a = 1
>>> b = "hello"
>>> import math
>>> dir()
['__builtins__', '__doc__', '__name__', 'a', 'b', 'math', 'pyscripter']

2. Python List Comprehension

List Comprehension vs For Loop in Python

Suppose, we want to separate the letters of the word human and add the letters as items of a list. The first thing that comes in mind would be using for loop.

Example 1: Iterating through a string Using for Loop

h_letters = []
for letter in 'human':
    h_letters.append(letter)
print(h_letters)

When we run the program, the output will be:

['h', 'u', 'm', 'a', 'n']

However, Python has an easier way to solve this issue using List Comprehension. List comprehension is an elegant way to define and create lists based on existing lists.

Let’s see how the above program can be written using list comprehensions.

Example 2: Iterating through a string Using List Comprehension

h_letters = [ letter for letter in 'human' ]
print( h_letters)

When we run the program, the output will be:

['h', 'u', 'm', 'a', 'n']

In the above example, a new list is assigned to variable h_letters, and list contains the items of the iterable string ‘human’. We call print() function to receive the output.

Syntax of List Comprehension

[expression for item in list]

We can now identify where list comprehensions are used.

If you noticed, human is a string, not a list. This is the power of list comprehension. It can identify when it receives a string or a tuple and work on it like a list.

You can do that using loops. However, not every loop can be rewritten as list comprehension. But as you learn and get comfortable with list comprehensions, you will find yourself replacing more and more loops with this elegant syntax.

List Comprehensions vs Lambda functions

List comprehensions aren’t the only way to work on lists. Various built-in functions and lambda functions can create and modify lists in less lines of code.

Example 3: Using Lambda functions inside List

letters = list(map(lambda x: x, 'human'))
print(letters)

When we run the program, the output will be

['h','u','m','a','n']

However, list comprehensions are usually more human readable than lambda functions. It is easier to understand what the programmer was trying to accomplish when list comprehensions are used.

Conditionals in List Comprehension

List comprehensions can utilize conditional statement to modify existing list (or other tuples). We will create list that uses mathematical operators, integers, and range().

Example 4: Using if with List Comprehension

number_list = [ x for x in range(20) if x % 2 == 0]
print(number_list)

When we run the above program, the output will be:

[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

The list ,number_list, will be populated by the items in range from 0–19 if the item’s value is divisible by 2.

Example 5: Nested IF with List Comprehension

num_list = [y for y in range(100) if y % 2 == 0 if y % 5 == 0]
print(num_list)

When we run the above program, the output will be:

[0, 10, 20, 30, 40, 50, 60, 70, 80, 90]

Here, list comprehension checks:

1. Is y divisible by 2 or not?
2. Is y divisible by 5 or not?

If y satisfies both conditions, y is appended to num_list.

Example 6: if…else With List Comprehension

obj = ["Even" if i%2==0 else "Odd" for i in range(10)]
print(obj)

When we run the above program, the output will be:

['Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd']

Here, list comprehension will check the 10 numbers from 0 to 9. If i is divisible by 2, then Even is appended to the obj list. If not, Odd is appended.

Nested Loops in List Comprehension

Suppose, we need to compute the transpose of a matrix that requires nested for loop. Let’s see how it is done using normal for loop first.

Example 7: Transpose of Matrix using Nested Loops

transposed = []
matrix = [[1, 2, 3, 4], [4, 5, 6, 8]]
for i in range(len(matrix[0])):
    transposed_row = []
    for row in matrix:
        transposed_row.append(row[i])
    transposed.append(transposed_row)
print(transposed)

Output

[[1, 4], [2, 5], [3, 6], [4, 8]]

The above code use two for loops to find transpose of the matrix.

We can also perform nested iteration inside a list comprehension. In this section, we will find transpose of a matrix using nested loop inside list comprehension.

Example 8: Transpose of a Matrix using List Comprehension

matrix = [[1, 2], [3,4], [5,6], [7,8]]
transpose = [[row[i] for row in matrix] for i in range(2)]
print (transpose)

When we run the above program, the output will be:

[[1, 3, 5, 7], [2, 4, 6, 8]]

In above program, we have a variable matrix which have 4 rows and 2 columns.We need to find transpose of the matrix. For that, we used list comprehension.

• *Note: The nested loops in list comprehension don’t work like normal nested loops. In the above program, for i in range(2) is executed before row[i] for row in matrix. Hence at first, a value is assigned to i then item directed by row[i] is appended in the transpose variable.

3. Iterators, generators and decorators

Iterators

Python iterator objects are required to support two methods while following the iterator protocol.

__iter__ returns the iterator object itself. This is used in for and in statements.

__next__ method returns the next value from the iterator. If there is no more items to return then it should raise StopIteration exception.

class Counter(object):
    def __init__(self, low, high):
        self.current = low
        self.high = high
    def __iter__(self):
        'Returns itself as an iterator object'
        return self
    def __next__(self):
        'Returns the next value till current is lower than high'
        if self.current > self.high:
            raise StopIteration
        else:
            self.current += 1
            return self.current - 1

Now we can use this iterator in our code.

>>> c = Counter(5,10)
>>> for i in c:
...   print(i, end=' ')
...
5 6 7 8 9 10

Remember that an iterator object can be used only once. It means after it raises StopIteration once, it will keep raising the same exception.

>>> c = Counter(5,6)
>>> next(c)
5
>>> next(c)
6
>>> next(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in next
StopIteration
>>> next(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in next
StopIteration

Using the iterator in for loop example we saw, the following example tries to show the code behind the scenes.

>>> iterator = iter(c)
>>> while True:
...     try:
...         x = iterator.__next__()
...         print(x, end=' ')
...     except StopIteration as e:
...         break
...
5 6 7 8 9 10

Generators

In this section we learn about Python generators. They were introduced in Python 2.3. It is an easier way to create iterators using a keyword yield from a function.

>>> def my_generator():
...     print("Inside my generator")
...     yield 'a'
...     yield 'b'
...     yield 'c'
...
>>> my_generator()
<generator object my_generator at 0x7fbcfa0a6aa0>

In the above example we create a simple generator using the yield statements. We can use it in a for loop just like we use any other iterators.

>>> for char in my_generator():
...     print(char)
...
Inside my generator
a
b
c

In the next example we will create the same Counter class using a generator function and use it in a for loop.

def counter_generator(low, high):
    while low <= high:
       yield low
       low += 1
>>> for i in counter_generator(5,10):
...     print(i, end=' ')
...
5 6 7 8 9 10

Inside the while loop when it reaches to the yield statement, the value of low is returned and the generator state is suspended. During the second next call the generator resumed where it freeze-ed before and then the value of low is increased by one. It continues with the while loop and comes to the yield statement again.

When you call an generator function it returns a *generator* object. If you call *dir* on this object you will find that it contains __iter__ and *__next__* methods among the other methods.

>>> c = counter_generator(5,10)
   >>> dir(c)
   ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__',
'__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__',
'__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'close', 'gi_code', 'gi_frame', 'gi_running', 'send', 'throw']

We mostly use generators for laze evaluations. This way generators become a good approach to work with lots of data. If you don’t want to load all the data in the memory, you can use a generator which will pass you each piece of data at a time.

One of the biggest example of such example is os.path.walk() function which uses a callback function and current os.walk generator. Using the generator implementation saves memory.

We can have generators which produces infinite values. The following is a one such example.

>>> def infinite_generator(start=0):
...     while True:
...         yield start
...         start += 1
...
>>> for num in infinite_generator(4):
...     print(num, end=' ')
...     if num > 20:
...         break
...
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

If we go back to the example of my_generator we will find one feature of generators. They are not re-usable.

>>> g = my_generator()
>>> for c in g:
...     print(c)
...
Inside my generator
a
b
c
>>> for c in g:
...     print(c)
...

One way to create a reusable generator is Object based generators which do not hold any state. Any class with a __iter__ method which yields data can be used as an object generator. In the following example we will recreate our counter generator.

>>> class Counter(object):
...     def __init__(self, low, high):
...         self.low = low
...         self.high = high
...     def __iter__(self):
...          counter = self.low
...          while self.high >= counter:
...              yield counter
...              counter += 1
...
>>> gobj = Counter(5, 10)
>>> for num in gobj:
...     print(num, end=' ')
...
5 6 7 8 9 10
>>> for num in gobj:
...     print(num, end=' ')
...
5 6 7 8 9 10

Generator expressions

In this section we will learn about generator expressions which is a high performance, memory efficient generalization of list comprehensions and generators.

For example we will try to sum the squares of all numbers from 1 to 9.

>>> sum([x*x for x in range(1,10)])

The example actually first creates a list of the square values in memory and then it iterates over it and finally after sum it frees the memory. You can understand the memory usage in case of a big list.

We can save memory usage by using a generator expression.

sum(x*x for x in range(1,10))

The syntax of generator expression says that always needs to be directly inside a set of parentheses and cannot have a comma on either side. Which basically means both the examples below are valid generator expression usage example.

>>> sum(x*x for x in range(1,10))
285
>>> g = (x*x for x in range(1,10))
>>> g
<generator object <genexpr> at 0x7fc559516b90>

We can have chaining of generators or generator expressions. In the following example we will read the file */var/log/cron* and will find if any particular job (in the example we are searching for anacron) is running successfully or not.

We can do the same using a shell command tail -f /var/log/cron |grep anacron

>>> jobtext = 'anacron'
>>> all_lines = (line for line in open('/var/log/cron', 'r') )
>>> job = ( line for line in all_lines if line.find(jobtext) != -1)
>>> text = next(job)
>>> text
"May  6 12:17:15 dhcp193-104 anacron[23052]: Job `cron.daily' terminated\n"
>>> text = next(job)
>>> text
'May  6 12:17:15 dhcp193-104 anacron[23052]: Normal exit (1 job run)\n'
>>> text = next(job)
>>> text
'May  6 13:01:01 dhcp193-104 run-parts(/etc/cron.hourly)[25907]: starting 0anacron\n'

You can write a for loop to the lines.