Algorithms, Data Structures

and Computability
 Agenda

1- Python:
Overview, Hello, World!, Variables and Types, Lists, Basic Operators,
String Formatting, Basic String Operations, Conditions, Loops,
Functions, Classes and Objects, Dictionaries, Modules and Packages
2- What is computation and computational thinking?
3- Complexity analysis:
• Asymptotic performance
• Asymptotic (big-O) notation
• Analysis of Algorithms
 Self-Learning Resources
Interactive tutorials

https://www.programiz.com/python-
1 programming

http://www.learnpython.org/
2
https://python-
3 course.eu/python3_course.php

https://www.tutorialspoint.com/pyth
4 on/
 Self-Learning Resources
books

Starting Out with Python for Everybody: Programming Python

Python (4th Edition) Exploring Data In Python 3 (4th Edition)
 Python Video Courses

• Tutorial 1 Series:
https://www.youtube.com/watch?v=P74
JAYCD45A&list=PLGzru6ACxEALhcvY1
8A-iox-mEoieHMVG

• Tutorial2 Series:
https://www.youtube.com/watch?v=HB
xCHonP6Ro&list=PL6gx4Cwl9DGAcbMi
1sH6oAMk4JHw91mC_

• Learn Python in 45 Minutes:

https://www.youtube.com/watch?v=N4
mEzFDjqtA
 Python features
no compiling or linking rapid development cycle
no type declarations simpler, shorter, more flexible
automatic memory management garbage collection
high-level data types and operations fast development
object-oriented programming code structuring and reuse, C++
embedding and extending in C mixed language systems
classes, modules, exceptions "programming-in-the-large" support
dynamic loading of C modules simplified extensions, smaller
binaries
dynamic reloading of C modules programs can be modified without
stopping
 Python features
universal "first-class" object model fewer restrictions and rules

run-time program construction handles unforeseen needs, end-user coding

interactive, dynamic nature incremental development and testing

access to interpreter information metaprogramming, introspective objects

wide portability cross-platform programming without ports

compilation to portable byte-code execution speed, protecting source code

built-in interfaces to external system tools, GUIs, persistence, databases,

services etc.
 Language comparison
Tcl Perl Python JavaScr Visual
ipt Basic
Speed development ✓ ✓ ✓ ✓ ✓
regexp ✓ ✓ ✓
breadth extensible ✓ ✓ ✓
embeddable ✓ ✓
easy GUI ✓ ✓ (Tk) ✓
net/web ✓ ✓ ✓ ✓ ✓
enterprise cross-platform ✓ ✓ ✓ ✓
I18N ✓ ✓ ✓ ✓
thread-safe ✓ ✓ ✓
database access ✓ ✓ ✓ ✓ ✓
 Uses of Python
• shell tools
• system admin tools, command line programs

• extension-language work
• rapid prototyping and development
• language-based modules
• instead of special-purpose parsers

• graphical user interfaces

• database access
• distributed programming
• Internet scripting
 Brief History of Python
• Invented in the Netherlands, early 90s by Guido van
Rossum
• Named after Monty Python
• Open sourced from the beginning
• Considered a scripting language, but is much more
• Scalable, object oriented and functional from the
beginning
• Used by Google from the beginning
• Increasingly popular
 Installing
• Python is pre-installed on most Unix systems, including
Linux and MAC OS X
• Download from http://python.org/download/
• Python comes with a large library of standard modules
• There are several options for an IDE
• IDLE – works well with Windows
• Emacs with python-mode or your favorite text editor
• Eclipse with Pydev (http://pydev.sourceforge.net/)
 Now Let’s Start
We will use the following interactive tutorial website

http://www.learnpython.org/
What is Computation?
What does each of them mean (try to write something down in your own words,
without looking them up)?
• Computational thinking
• Computational problem
Answer:
• computational thinking as not merely knowing how to use an algorithm or
a data structure, but, when faced with a problem, to be able to analyse it
with the techniques and skills that computer science puts at our disposal.

• A computational problem is described as a problem that is expressed

sufficiently precisely that it is possible to attempt to build an algorithm to
solve it.
 The point is that computational thinking is not about thinking like a
computer rather, computational thinking is first and foremost.
Computational thinking consists of the skills to:
• formulate a problem as a computational problem
• construct a good computational solution (i.e. an algorithm) for the
problem, or explain why there is no such solution.

A computational thinker won’t, however, be satisfied with just any solution: the solution has
to be a ‘good’ one. You have already seen that some solutions for finding a word in a
dictionary are much better (in particular, faster) than others. The search for good
computational solutions is a theme that runs throughout this module. Finally,
computational thinking goes beyond finding solutions: if no good solution exists, one
should be able to explain why this is so. This requires insight into the limits of computational
problem solving.
 Analysis of Algorithms

Input Algorithm Output

17
 Our goal:

Analysis of Algorithms
Measure Performance
Measure Space Complexity
 Running Time
• Most algorithms transform best case
input objects into output average case
objects. worst case

• The running time of an

120

algorithm typically grows with 100

the input size.

Running Time
80

• Average case time is often 60

difficult to determine.
40
• We focus on the worst case 20
running time.
• Easier to analyze 0
1000 2000 3000 4000

• Crucial to applications such as Input Size

games, finance and robotics
19
 Why discarding average case,
And choose worst case instead?
• An algorithm may run faster on some inputs than it
does on others of the same size. Thus, we may wish to
express the running time of an algorithm as the function
of the input size

• Average-case analysis is typically quite challenging. It

requires us to define a probability distribution on the set
of inputs, which is often a difficult task.

• An average-case analysis usually requires that we

calculate expected running times based on a given input
distribution, which usually involves sophisticated
probability theory. Therefore we will characterize
running times in terms of the worst case, as a function
of the input size, n, of the algorithm.
• Worst-case analysis is much easier than average-case
analysis, as it requires only the ability to identify the
worst-case input, which is often simple.
 Experimental Studies
• Write a program 9000

8000
implementing the algorithm
• Run the program with inputs 7000

of varying size and 6000

Time (ms)
composition, noting the time 5000
needed:
4000

3000

2000

1000

0
• Plot the results 0 50 100
Input Size 21
 Limitations of Experiments

• It is necessary to implement the whole algorithm before

conducting any experiment, which may be difficult.
• Results may not be indicative of the running time on other
inputs not included in the experiment.
• In order to compare two algorithms, the same hardware
and software environments must be used

22
 So we need another way to measure the
performance of the algorithms
• So we need to learn about Theoretical analysis or
Asymptotic analysis.
• Uses a high-level description of the algorithm
instead of an implementation (Pseudo code).
• Characterizes running time as a function of the
input size, n.
• Takes into account all possible inputs.
• Allows us to evaluate the speed of an algorithm
independent of the hardware/software
environment.

Pseudo code
• High-level description of an
algorithm.
• More structured than English prose.
• Less detailed than a program.
• Preferred notation for describing
algorithms.
• Hides program design issues.
 Big-Oh Notation
• Given functions f(n) and g(n),
we say that f(n) is O(g(n)) if
there are positive constants
c and n0 such that
f(n)  cg(n) for n  n0
• Example: 2n + 10 is O(n)
• 2n + 10  cn
• (c − 2) n  10
• n  10/(c − 2)
• Pick c = 3 and n0 = 10

24
 Big-Oh and Growth Rate
• The big-Oh notation gives an upper bound on the growth
rate of a function
• The statement “f(n) is O(g(n))” means that the growth rate
of f(n) is no more than the growth rate of g(n)
• We can use the big-Oh notation to rank functions according
to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more Yes No
f(n) grows more No Yes
Same growth Yes Yes 25
Relatives of Big-Oh
big-Omega
◼ f(n) is (g(n)) if there is a constant c > 0

and an integer constant n0  1 such that

f(n)  c•g(n) for n  n0

big-Theta
◼ f(n) is (g(n)) if there are constants c’ > 0 and c’’
> 0 and an integer constant n0  1 such that
c’•g(n)  f(n)  c’’•g(n) for n  n0

26
 Essential Seven functions to estimate algorithms
performance

g(n) = 1

Print(“Hello Algorithms”)
Essential Seven functions to estimate algorithms
performance

g(n) = n

for i in range(0, n):

Print(i)
Essential Seven functions to estimate algorithms
performance

Def power_of_2(a):
x = 0
g(n) = lg n while a > 1:
a = a/2
x = x+1
return x
Essential Seven functions to estimate algorithms
performance

for i in range(0,n):
Def power_of_2(a):
x = 0
while a > 1:
a = a/2
x = x+1
return x

g(n) = n lg n
Essential Seven functions to estimate algorithms
performance

for i in range(0,n):
for j in range(0,n):
print(i*j);

g(n) = n2
Essential Seven functions to estimate algorithms
performance

for i in range(0, k):

for i in range(0,n):
for j in range(0,n):
print(i*j);

g(n) = n3
 Essential Seven functions to estimate algorithms
performance

def F(n):
if n == 0:
return 0
elif n == 1:
return 1
g(n) = 2n else: return
F(n-1) + F(n-2)
 Seven Important Functions
• Seven functions that often appear in
algorithm analysis:
• Constant  1
• Logarithmic  log n
• Linear  n
• N-Log-N  n log n
• Quadratic  n2
• Cubic  n3
• Exponential  2n

• In a log-log chart, the slope of the line

Analysis of Algorithms 34
corresponds to the growth rate
 Slide by Matt Stallmann included with
Why Growth Rate Matters permission.

if runtime is... time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2) runtime

quadruples
cn c (n + 1) 2c n 4c n when problem
size doubles
~ c n lg n
c n lg n 2c n lg n + 2cn 4c n lg n + 4cn
+ cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n
Comparison of Two Algorithms

insertion sort is
n2 / 4
merge sort is
2 n lg n
sort a million items?
insertion sort takes
roughly 70 hours
while
merge sort takes
roughly 40 seconds

This is a slow machine, but if

100 x as fast then it’s 40 minutes
versus less than 0.5 seconds
36
 How to calculate the algorithm’s complexity

◼ We may not be able to predict to the nanosecond how long a

Python program will take, but do know some things about
timing: for i in range(0, n):
print(i);

◼ This loop takes time k*n, for some constants k.

k : How long it takes to go through the loop once

n : The number of times through the loop

(we can use this as the “size” of the problem)
◼ The total time k*n is linear in n
 Constant time
• Constant time means there is some
constant k such that this operation
always takes k nanoseconds
• A Java statement takes constant time
if:
• It does not include a loop
• It does not include calling a
method whose time is
unknown or is not a constant
• If a statement involves a choice (if or
switch) among operations, each of
which takes constant time, we
consider the statement to take
constant time
• This is consistent with worst-case
analysis 38
Prefix Averages (Quadratic)
The following algorithm computes prefix averages in
quadratic time by applying the definition

39
Prefix Averages 2 (Looks Better)
The following algorithm uses an internal Python
function to simplify the code

Algorithm prefixAverage2 still runs in O(n2) time! 40

Prefix Averages 3 (Linear Time)
The following algorithm computes prefix averages in
linear time by keeping a running sum

Algorithm prefixAverage3 runs in O(n) time 41

Activity
Activity
Thank You for
Your Attention