Class 2B: Review of Programming in Python II

Class 2B: Review of Programming in Python II#

We will begin soon!

Photo by Christina Morillo from Pexels

Download the (logistics) Slides from today

Firas Moosvi

Lecture Outline#

Comments (0 min)
Loops
Comprehensions
Break (5 min)
Functions
Exceptions
Testing
Questions and Recap

Attribution#

The original version of these Python lectures were by Patrick Walls.
These lectures were originally delivered by Mike Gelbart and are available publicly here.

Comments in python#

x = 1 # 5 + 5 this is a comment

# """
# this is a string, which does nothing
# and can be used as a comment
# """

7


x = 1

Loops#

Loops allow us to execute a block of code multiple times.
We will focus on for loops

"I have 5 apples"
"I have 6 apples"

'I have 6 apples'

`for` loops#

for n in [2, 7, -1, 5]:
    print("The number is", n, "its square is", n**2)
    if n > 2:
        print("hallelujah")
    # this is inside the loop
# this is outside the loop

The number is 2 its square is 4
The number is 7 its square is 49
hallelujah
The number is -1 its square is 1
The number is 5 its square is 25
hallelujah

import numpy as np
np.arange(0,100,2)

array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32,
       34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66,
       68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98])

The main points to notice:

Keyword for begins the loop
Colon : ends the first line of the loop
We can iterate over any kind of iterable: list, tuple, range, string. In this case, we are iterating over the values in a list
Block of code indented is executed for each value in the list (hence the name “for” loops, sometimes also called “for each” loops)
The loop ends after the variable n has taken all the values in the list

"abc" + "def"

'abcdef'

word = "Python"
for letter in word:
    print("Gimme a " + letter + "!")

print("\tWhat's that spell?!! " + word + "!")

Gimme a P!
Gimme a y!
Gimme a t!
Gimme a h!
Gimme a o!
Gimme a n!
	What's that spell?!! Python!

A very common pattern is to use for with range.
range gives you a sequence of integers up to some value.

for i in range(10):
    print(i)

We can also specify a start value and a skip-by value with range:

for i in range(1,101,10):
    print(i)

We can write a loop inside another loop to iterate over multiple dimensions of data. Consider the following loop as enumerating the coordinates in a 3 by 3 grid of points.

for x in [1,2,3]:
    for y in ["a","b","c"]:
        print((x,y))

(1, 'a')
(1, 'b')
(1, 'c')
(2, 'a')
(2, 'b')
(2, 'c')
(3, 'a')
(3, 'b')
(3, 'c')

list_1 = [1,2,3,4,5]
list_2 = ["a","b","c","d"]
for i in range(5):
    print(list_1[i], list_2[i])

a
b
c
d

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[11], line 4
      2 list_2 = ["a","b","c","d"]
      3 for i in range(5):
----> 4     print(list_1[i], list_2[i])

IndexError: list index out of range

for i,j in zip(list_1,list_2):
    print(i,j)

a
b
c
d

We can loop through key-value pairs of a dictionary using .items():

print("Hello\\nWorld")

Hello\nWorld

h = """Hello \\n
World"""

print(h)

Hello \n
World

courses = {521 : "awesome",
           551 : "riveting",
           511 : "naptime!"}

for course_num, description in courses.items():
    print("DSCI", course_num, "is", description)

DSCI 521 is awesome
DSCI 551 is riveting
DSCI 511 is naptime!

for course_num,course_desc in courses.items():
    print(course_num, course_desc)
#    print(course_num, courses[course_num])

awesome
riveting
naptime!

Above: the general syntax is for key, value in dictionary.items():

`while` loops#

We can also use a while loop to excute a block of code several times.
In reality, I rarely use these.
Beware! If the conditional expression is always True, then you’ve got an infintite loop!
- (Use the “Stop” button in the toolbar above, or Ctrl-C in the terminal, to kill the program if you get an infinite loop.)

n = 10
while n > 0:
    print(n)
    n = n - 1

print("Blast off!")

10
9
8
7
6
5
4
3
2
1
Blast off!

Comprehensions#

Comprehensions allow us to build lists/sets/dictionaries in one convenient, compact line of code.

List Comprehensions#

List comprehensions are one of my favourite features of Python!

[hh*10 for hh in range(10)]

[0, 10, 20, 30, 40, 50, 60, 70, 80, 90]

words = ["hello", "goodbye", "the", "antidisestablishmentarianism"]

y = [word[-3:] for word in words]  # list comprehension
y

['llo', 'bye', 'the', 'ism']

for word in words:
    print(word[-3:])

llo
bye
the
ism

y = list()
for word in words:
    y.append(word[-3:])
y

['llo', 'bye', 'the', 'ism']

Dictionary comprehension#

word_lengths = {word : len(word) for word in words} # dictionary comprehension
word_lengths

{'hello': 5, 'goodbye': 7, 'the': 3, 'antidisestablishmentarianism': 28}

word_upper = {word : word.upper() for word in words}
word_upper

{'hello': 'HELLO',
 'goodbye': 'GOODBYE',
 'the': 'THE',
 'antidisestablishmentarianism': 'ANTIDISESTABLISHMENTARIANISM'}

Set Comprehensions#

y = {word[-1] for word in words}  # set comprehension
print(y)

{'o', 'm', 'e'}

Tuple Comprehensions#

Tuple comprehensions actually don’t exist! But, you can use generators.

See this StackOverflow post for (some fun) details!

Break (5 min)#

Functions#

Define a function to re-use a block of code with different input parameters, also known as arguments.
For example, define a function called square which takes one input parameter n and returns the square n**2.

def square(n):
    n_squared = n**2
    return n_squared

# quick test:
assert square(2) ==4

square(12345)

152399025

square(2)

square(100)

# this will fail 
# square('hello')

Begins with def keyword, function name, input parameters and then colon (:)
Function block defined by indentation
Output or “return” value of the function is given by the return keyword

Side effects#

If a function changes the variables passed into it, then it is said to have side effects
Example:

def silly_sum(sri):
    sri.append(0)
    return sum(sri)

silly_sum([1,2,3,4])

Looks good, like it sums the numbers? But wait…

lst = [1,2,3,4]
silly_sum(lst)

silly_sum(lst)

lst

[1, 2, 3, 4, 0, 0]

If your function has side effects like this, you must mention it in the documentation (later today).
More on how this works next class.

Null return type#

If you do not specify a return value, the function returns None when it terminates:

def f(x):
    x + 1 # no return!
    if x == 999:
        return
    else:
        return('hello')
print(f(998))

Designing good functions - DRY principle (15 min)#

DRY: Don’t Repeat Yourself
See Wikipedia article
Consider the task of, for each element of a list, turning it into a palindrome
- e.g. “mike” –> “mikeekim”

names = ["milad", "rodolfo", "tiffany", "khalad", "jeff", "christel","firas"]

name = "mike"
name[::-1]

names_backwards = list()

names_backwards.append(names[0] + names[0][::-1])
names_backwards.append(names[1] + names[1][::-1])
names_backwards.append(names[2] + names[2][::-1])
names_backwards.append(names[3] + names[3][::-1])
names_backwards.append(names[4] + names[4][::-1])
names_backwards.append(names[5] + names[5][::-1])
names_backwards.append(names[6] + names[6][::-1])


names_backwards

Above: this is gross, terrible, yucky code
1. It only works for a list with 3 elements
2. It only works for a list named names
3. If we want to change its functionality, we need to change 3 similar lines of code (Don’t Repeat Yourself!!)
4. It is hard to understand what it does just by looking at it

names_backwards = list()

for name in names:
    names_backwards.append(name + name[::-1])
    
names_backwards

Above: this is slightly better. We have solved problems (1) and (3).

def make_palindromes(names):
    names_backwards = list()
    
    for name in names:
        names_backwards.append(name + name[::-1])
    
    return names_backwards

p_name = make_palindromes(names)

for p in p_name:
    print(p.upper())

Above: this is even better. We have now also solved problem (2), because you can call the function with any list, not just names.
For example, what if we had multiple lists:

names1 = ["milad", "rodolfo", "tiffany"]
names2 = ["Trudeau", "Scheer", "Singh", "Blanchet", "May"]
names3 = ["apple", "orange", "banana"]

names_backwards_1 = list()

for name in names1:
    names_backwards_1.append(name + name[::-1])
    
names_backwards_1

names_backwards_2 = list()

for name in names2:
    names_backwards_2.append(name + name[::-1])
    
names_backwards_2

names_backwards_3 = list()

for name in names3:
    names_backwards_3.append(name + name[::-1])
    
names_backwards_3

Above: this is very bad also (and imagine if it was 20 lines of code instead of 2). This was problem (2). Our function makes it much better:

make_palindromes(names1)

make_palindromes(names2)

make_palindromes(names3)

You could get even more fancy, and put the lists of names into a list (so you have a list of lists).
Then you could loop over the list and call the function each time:

for list_of_names in [names1, names2, names3]:
    print(make_palindromes(list_of_names))

Designing good functions#

How far you go with this is sort of a matter of personal style, and how you choose to apply the DRY principle: DON’T REPEAT YOURSELF!
These decisions are often ambiguous. For example:
- Should make_palindromes be a function if I’m only ever doing it once? Twice?
- Should the loop be inside the function, or outside?
- Or should there be TWO functions, one that loops over the other??

In my personal opinion, make_palindromes does a bit too much to be understandable.
I prefer this:

def make_palindrome(name):
    return name + name[::-1]

make_palindrome("milad")

From here, we want to “apply make_palindrome to every element of a list”
It turns out this is an extremely common desire, so Python has built-in functions.
One of these is map, which we’ll cover later. But for now, just a comprehension will do:

[make_palindrome(name) for name in names]

Other function design considerations:

Should we print output or produce plots inside or outside functions?
- I would usually say outside, because this is a “side effect” of sorts
Should the function do one thing or many things?
- This is a tough one, hard to answer in general

Optional & keyword arguments#

Sometimes it is convenient to have default values for some arguments in a function.
Because they have default values, these arguments are optional, hence “optional arguments”
Example:

def repeat_string(s, n=2):
    return s*n

repeat_string("mds", 2)

repeat_string("mds", 5)

Sane defaults:

Ideally, the default should be carefully chosen.
Here, the idea of “repeating” something makes me think of having 2 copies, so n=2 feels like a sane default.

Syntax:

You can have any number of arguments and any number of optional arguments
**All the optional arguments must come after the regular arguments
The regular arguments are mapped by the order they appear
The optional arguments can be specified out of order

def example(a, b, c="DEFAULT", d="DEFAULT"):
    print(a,b,c,d)
    
example(1,2,3,4)

Using the defaults for c and d:

example(1,2)

Specifying c and d as keyword arguments (i.e. by name):

example(1,2,c=3,d=4)

Specifying only one of the optional arguments, by keyword:

example(1,2,c=3)

Or the other:

example(1,2,d=4)

Specifying all the arguments as keyword arguments, even though only c and d are optional:

example(a=1,b=2,c=3,d=4)

Specifying c by the fact that it comes 3rd (I do not recommend this because I find it is confusing):

example(1,2,3) 

Specifying the optional arguments by keyword, but in the wrong order (this is also somewhat confusing, but not so terrible - I am OK with it):

example(1,2,d=4,c=3) 

Specifying the non-optional arguments by keyword (I am fine with this):

example(a=1,b=2)

Specifying the non-optional arguments by keyword, but in the wrong order (not recommended, I find it confusing):

example(b=2,a=1)

Specifying keyword arguments before non-keyword arguments (this throws an error):

example(a=2,b=1)

In general, I am used to calling non-optional arguments by order, and optional arguments by keyword.
The language allows us to deviate from this, but it can be unnecessarily confusing sometimes.

`kwargs`:#

You can also call/define functions with *args and **kwargs; see, e.g. here
Do not instantiate objects in the function definition - see here under “Mutable Default Arguments”

def example(a, b=[]): # don't do this!
    return 0

def example(a, b=None): # insted, do this
    if b is None:
        b = []
    return 0

Anonymous functions (5 min)#

There are two ways to define functions in Python:

def add_one(x):
    return x+1

add_one(7.2)

8.2

add_one = lambda x: x+1 

type(add_one)

function

add_one(7.2)

8.2

The two approaches above are identical. The one with lambda is called an anonymous function.

Some differences:

anonymous functions can only take up one line of code, so they aren’t appropriate in most cases.
anonymous functions evaluate to a function (remember, functions are first-class objects) immediate, so we can do weird stuff with them.

(lambda x,y: x+y)(6,7)

evaluate_function_on_x_plus_1(lambda x: x**2, 5)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[75], line 1
----> 1 evaluate_function_on_x_plus_1(lambda x: x**2, 5)

NameError: name 'evaluate_function_on_x_plus_1' is not defined

Above:

First, lambda x: x**2 evaluates to a value of type function
- Notice that this function is never given a name - hence “anonymous functions” !
Then, the function and the integer 5 are passed into evaluate_function_on_x_plus_1
At which point the anonymous function is evaluated on 5+1, and we get 36.

Exceptions, `try`/`except` (10 min)#

If something goes wrong, we don’t want the code to crash - we want it to fail gracefully.
In Python, this can be accomplished using try/except:
Here is a basic example:

this_variable_does_not_exist

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[76], line 1
----> 1 this_variable_does_not_exist

NameError: name 'this_variable_does_not_exist' is not defined

try:
    this_variable_does_not_exist
except:
#     pass
    print("You did something bad!")

You did something bad!

Python tries to execute the code in the try block.
If an error is encountered, we “catch” this in the except block (also called try/catch in other languages).
There are many different error types, or exceptions - we saw NameError above.

5/0

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
Cell In[78], line 1
----> 1 5/0

ZeroDivisionError: division by zero

my_list = [1,2,3]
my_list[5]

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[79], line 2
      1 my_list = [1,2,3]
----> 2 my_list[5]

IndexError: list index out of range

# (note: this is also valid syntax, just very confusing)
[1,2,3][5]

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[80], line 2
      1 # (note: this is also valid syntax, just very confusing)
----> 2 [1,2,3][5]

IndexError: list index out of range

my_tuple = (1,2,3)
my_tuple[0] = 0

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[81], line 2
      1 my_tuple = (1,2,3)
----> 2 my_tuple[0] = 0

TypeError: 'tuple' object does not support item assignment

Ok, so there are apparently a bunch of different errors one could run into.
With try/except you can also catch the exception itself:

try:
    this_variable_does_not_exist
except Exception as ex:
    print("You did something bad!")
    print(ex)
    print(type(ex))

You did something bad!
name 'this_variable_does_not_exist' is not defined
<class 'NameError'>

In the above, we caught the exception and assigned it to the variable ex so that we could print it out.
This is useful because you can see what the error message would have been, without crashing your program.

You can also catch specific exceptions types, like so:

try:
    this_variable_does_not_exist
except TypeError:
    print("You made a type error!")
except NameError:
    print("You made a name error!")
except:
    print("You made some other sort of error")

You made a name error!

The final except would trigger if the error is none of the above types, so this sort of has an if/elif/else feel to it.
There are some extra features, in particular an else and finally block; if you are interested, see e.g., here.

try:
    5/0
except TypeError:
    print("You made a type error!")
except NameError:
    print("You made a name error!")
except Exception as ex:
    print("You made some other sort of error")

You made some other sort of error

Ideally, try to make your try/except blocks specific, and try not to put more errors inside the except…

try:
    this_variable_does_not_exist
except:
    5/0

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[85], line 2
      1 try:
----> 2     this_variable_does_not_exist
      3 except:

NameError: name 'this_variable_does_not_exist' is not defined

During handling of the above exception, another exception occurred:

ZeroDivisionError                         Traceback (most recent call last)
Cell In[85], line 4
      2     this_variable_does_not_exist
      3 except:
----> 4     5/0

ZeroDivisionError: division by zero

This is a bit much, but it does happen sometimes :(

Using `raise`#

You can also write code that raises an exception on purpose, using raise

def add_one(x):
    return x+1

add_one("blah")

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[87], line 1
----> 1 add_one("blah")

Cell In[86], line 2, in add_one(x)
      1 def add_one(x):
----> 2     return x+1

TypeError: can only concatenate str (not "int") to str

def add_one(x):
    if not isinstance(x, float) and not isinstance(x, int):
        raise Exception("Sorry, x must be numeric")
        
    return x+1

add_one("blah")

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[89], line 1
----> 1 add_one("blah")

Cell In[88], line 3, in add_one(x)
      1 def add_one(x):
      2     if not isinstance(x, float) and not isinstance(x, int):
----> 3         raise Exception("Sorry, x must be numeric")
      5     return x+1

Exception: Sorry, x must be numeric

This is useful when your function is complicated and would fail in a complicated way, with a weird error message.
You can make the cause of the error much clearer to the caller of the function.
Thus, your function is more usable this way.
If you do this, you should ideally describe these exceptions in the function documentation, so a user knows what to expect if they call your function.

You can also raise other types of exceptions, or even define your own exception types, as in lab 2.
You can also use raise by itself to raise whatever exception was going on:

try:
    this_variable_does_not_exist
except:
    print("You did something bad!")
    raise

You did something bad!

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[90], line 2
      1 try:
----> 2     this_variable_does_not_exist
      3 except:
      4     print("You did something bad!")

NameError: name 'this_variable_does_not_exist' is not defined

Here, the original exception is raised after we ran some other code.

Testing#

`assert` statements#

assert statementS cause your program to fail if the condition is False.
They can be used as sanity checks for your program.
There are more sophisticated way to “test” your programs, which we’ll discuss in Workflows.
The syntax is:

assert expression , "Error message if expression is False or raises an error."

a = 4
b = 5

# Change this assert statement so a and b is used instead of "hard-coded 1 and 2"
assert a == b , f"{a} is not equal to {b}."

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[91], line 5
      2 b = 5
      4 # Change this assert statement so a and b is used instead of "hard-coded 1 and 2"
----> 5 assert a == b , f"{a} is not equal to {b}."

AssertionError: 4 is not equal to 5.

Systematic Program Design#

A systematic approach to program design is a general set of steps to follow when writing programs. Our approach includes:

Write a stub: a function that does nothing but accept all input parameters and return the correct datatype.
Write tests to satisfy the design specifications.
Outline the program with pseudo-code.
Write code and test frequently.
Write documentation.

The key point: write tests BEFORE you write code.

You do not have to do this in MDS, but you may find it surprisingly helpful.
Often writing tests helps you think through what you are trying to accomplish.
It’s best to have that clear before you write the actual code.

# Task: Let's calculate area of a rectangle, given length and width

def calculate_rectangle_area(length, width):
    """ Computes the area of a rectangle given a length and a width
    inputs: length (float) and width(float)
    
    oututs: area (float)
    """
    
    # Check the inputs
    # check if length and width are floats
    
    # Compute area
    area = length * width
    
    print(f"the area should be : {length * width} and is {area}")
    
    return area

assert calculate_rectangle_area(5,5) == 25, "the function is not quite correct because the area of a 5x5 should be 25."
calculate_rectangle_area(5,5)

the area should be : 25 and is 25
the area should be : 25 and is 25

Testing woes - false positives#

Just because all your tests pass, this does not mean your program is correct!!
This happens all the time. How to deal with it?
- Write a lot of tests!
- Don’t be overconfident, even after writing a lot of tests!

def sample_median(x):
    """Finds the median of a list of numbers."""
    x_sorted = sorted(x)
    return x_sorted[len(x_sorted)//2]

assert sample_median([1,2,3,4,5]) == 3
assert sample_median([0,0,0,0]) == 0

Looks good? … ?

4//2 

#is equivalent to

int(4/2)

assert sample_median([1,2,3,4]) == 2.5

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[103], line 1
----> 1 assert sample_median([1,2,3,4]) == 2.5

AssertionError: 

assert sample_median([1,3,2]) == 2

Testing woes - false negatives#

It can also happen, though more rarely, that your tests fail but your program is correct.
This means there is something wrong with your test.
For example, in the autograding for lab1 this happened to some people, because of tiny roundoff errors.

Corner cases#

A corner case is an input that is reasonable but a bit unusual, and may trip up your code.
For example, taking the median of an empty list, or a list with only one element.
Often it is desirable to add test cases to address corner cases.

assert sample_median([1]) == 1

In this case the code worked with no extra effort, but sometimes we need if statements to handle the weird cases.
Sometimes we want the code to throw an error (e.g. median of an empty list)

Class 2B: Review of Programming in Python II

Contents

Class 2B: Review of Programming in Python II#

Lecture Outline#

Attribution#

Comments in python#

Loops#

`for` loops#

`while` loops#

Comprehensions#

List Comprehensions#

Dictionary comprehension#

Set Comprehensions#

Tuple Comprehensions#

Break (5 min)#

Functions#

Side effects#

Null return type#

Designing good functions - DRY principle (15 min)#

Designing good functions#

Optional & keyword arguments#

`kwargs`:#

Anonymous functions (5 min)#

Exceptions, `try`/`except` (10 min)#

Using `raise`#

Testing#

`assert` statements#

Systematic Program Design#

Testing woes - false positives#

Testing woes - false negatives#

Corner cases#

Questions and Recap?#

Class 2B: Review of Programming in Python II

Contents

Class 2B: Review of Programming in Python II#

Lecture Outline#

Attribution#

Comments in python#

Loops#

for loops#

while loops#

Comprehensions#

List Comprehensions#

Dictionary comprehension#

Set Comprehensions#

Tuple Comprehensions#

Break (5 min)#

Functions#

Side effects#

Null return type#

Designing good functions - DRY principle (15 min)#

Designing good functions#

Optional & keyword arguments#

kwargs:#

Anonymous functions (5 min)#

Exceptions, try/except (10 min)#

Using raise#

Testing#

assert statements#

Systematic Program Design#

Testing woes - false positives#

Testing woes - false negatives#

Corner cases#

Questions and Recap?#

`for` loops#

`while` loops#

`kwargs`:#

Exceptions, `try`/`except` (10 min)#

Using `raise`#

`assert` statements#