CENG 205

Data structures
Lecture 2:
Recursion & Performance analysis
System Life Cycle

• Programs pass through a period called ‘system life cycle’, which is

defined by the following steps:

• 1. Requirements: It is defined by the inputs given to the program and outputs

that will be produced by the program.

• 2. Analysis: The first job after defining the requirements is to analyze the
problem. There are two ways for this:
• Bottom-up
• Top-down
System Life Cycle (cont’)

• 3. Design: Data objects and the possible operations between the objects are
defined in this step. Therefore, abstract data objects and the algorithm are
defined. They are all independent of the programming language.

• 4. Refinement and Coding: Algorithms are used in this step in order to do

operations on the data objects.

• 5. Verification: Correctness proofs, testing and error removal are performed

in this step.
Cast of Characters
Recursion
Recursion

• Recursion is all about breaking a big problem into smaller

occurrences of that same problem.
• Each person can solve a small part of the problem.
• What is a small version of the problem that would be easy to answer?
• What information from a neighbor might help me?
Recursive Algorithm

• Number of people behind me:

• If there is someone behind me,
ask him/her how many people are behind him/her.
• When they respond with a value N, then I will answer N + 1.

• If there is nobody behind me, I will answer 0.

Recursive Algorithms

• Functions can call themselves (direct recursion)

• A function that calls another function is called by the second function
again. (indirect recursion)
Recursion

• Consider the following method to print a line of * characters:

// Prints a line containing the given number of
stars.
// Precondition: n >= 0
void printStars(int n) {
for (int i = 0; i < n; i++) {
printf("*");
}
printf(“\n"); // end the line of output
}

• Write a recursive version of this method (that calls itself).

• Solve the problem without using any loops.
• Hint: Your solution should print just one star at a time.
A basic case

• What are the cases to consider?

• What is a very easy number of stars to print without a loop?

void printStars(int n) {
if (n == 1) {
// base case; just print one star
printf("*\n");
} else {
...
}
}
Handling more cases

• Handling additional cases, with no loops (in a bad way):

void printStars(int n) {
if (n == 1) {
// base case; just print one star
printf("*\n");
} else if (n == 2) {
printf("*");
printf("*\n");
} else if (n == 3) {
printf("*");
printf("*");
printf("*\n");
} else if (n == 4) {
printf("*");
printf("*");
printf("*");
printf("*\n");
} else ...
}
Handling more cases 2

• Taking advantage of the repeated pattern (somewhat better):

void printStars(int n) {
if (n == 1) {
// base case; just print one star
printf("*\n");
} else if (n == 2) {
printf("*");
printStars(1); // prints "*"
} else if (n == 3) {
printf("*");
printStars(2); // prints "**"
} else if (n == 4) {
printf("*");
printStars(3); // prints "***"
} else ...
}
Using recursion properly

• Condensing the recursive cases into a single case:

void printStars(int n) {
if (n == 1) {
// base case; just print one star
printf("*\n");
} else {
// recursive case; print one more star
printf("*");
printStars(n - 1);
}
}
Even simpler

• The real, even simpler, base case is an n of 0, not 1:

void printStars(int n) {
if (n == 0) {
// base case; just end the line of
output
printf(“\n”);
} else {
// recursive case; print one more star
printf("*");
printStars(n - 1);
}
}
Recursive tracing

• Consider the following recursive method:

int mystery(int n) {
if (n < 10) {
return n;
} else {
int a = n / 10;
int b = n % 10;
return mystery(a + b);
}
}

• What is the result of the following call?

mystery(648)
A recursive trace

mystery(648):
▪ int a = 648 / 10; // 64
▪ int b = 648 % 10; // 8
▪ return mystery(a + b); // mystery(72)
mystery(72):
▪ int a = 72 / 10; // 7
▪ int b = 72 % 10; // 2
▪ return mystery(a + b); // mystery(9)

mystery(9):
▪ return 9;
Recursive tracing 2

• Consider the following recursive method:

int mystery(int n) {
if (n < 10) {
return (10 * n) + n;
} else {
int a = mystery(n / 10);
int b = mystery(n % 10);
return (100 * a) + b;
}
}

• What is the result of the following call?

mystery(348)
A recursive trace 2

mystery(348)
▪ int a = mystery(34);
• int a = mystery(3);
return (10 * 3) + 3; // 33
• int b = mystery(4);
return (10 * 4) + 4; // 44
• return (100 * 33) + 44; // 3344

▪ int b = mystery(8);
return (10 * 8) + 8; // 88

▪ return (100 * 3344) + 88; // 334488

• What is this method really doing?

Selection Sort

• Let’s sort an array that consists of randomly inserted items.

for(i=0; i<n; i++){

examine list[i] to list[n-1] and suppose that
the smallest integer is at list[min];
interchange list[i] and list[min];
}
 Selection Sort
#include <stdio.h> void sort(int list[], int n)
#include <math.h> {
#define MAX_SIZE 101 int i, j, min, temp;
#define SWAP(x, y, z) (z = x; x = y; y = z) for(i = 0; i < n-1; i++){
void sort(int [], int); min = i;
for(j = i+1; j < n; j++)
void main() if(list[j] < list[i]){
{ min = j;
int i, n; SWAP(list[i], list[min], temp);
int list[MAX_SIZE]; }
printf(”array size"); }
scanf(“%d", &n);

for(i = 0; i < n; i++){

list[i] = rand() % 100;
printf("%d ", list[i]);
}
sort(list,n);
}
SWAP

• prototype:

void swap(int *, int *);

void swap(int *x, int *y)

{
int temp = *x;
*x = *y ;
*y = temp;
}
 Recursive Selection Sort
int minIndex int list[], int i, int j)
{
if(i==j)
return i;
int k = minIndex (list, i+1, j);
return (list[i]<list[k]) ? i : k;
}

void recursive_selection_sort(int list[], int index, int n)

{
if(index==n)
return;
int k = minIndex (list, index, n-1);
if(k != index)
SWAP(list[k], list[index]);
recursive_selection_sort(list, index+1, n);
}

recursive_selection_sort(list, 0, N);
Binary Search

• A value is searched in a sorted array. If found, the index of the item is

returned, otherwise -1 is returned.
int binsearch(int list[], int searchnum, int left, int right)
{
int middle;
while(left <= right){
middle = (left + right)/2;
switch(compare(list[middle], searchnum)){
case -1: left = middle + 1; break;
case 0: return middle;
case 1: right = middle –1;
}
}
return -1;
}

• Macro for ‘COMPARE’:

#define COMPARE(x,y) (((x)<(y))? -1 : ((x)==(y))?0:1)
Binary Search
Let’s search for 18:
Binary Search (revisited)

int recbinsearch(int list[], int searchnum, int left, int right)

{
int middle;
if(left<=right){
middle = (left+right)/2;
switch(COMPARE(list[middle],searchnum)){
case -1: return recbinsearch(list, searchnum, middle+1, right);
case 0: return middle;
case 1: return recbinsearch(list, searchnum, left, middle-1);
}
}
return -1;
}
Binary Search (revisited)
Permutation problem

• Finding all the permutations of a given set with size n≥1.

• Remember there are n! different sequences of this set.
• Example: Find all the permutations of {a,b,c}
Permutation problem

• Example: Find all the permutations of {a,b,c,d}

• All the permutations of {b,c,d} follow ‘a’

• All the permutations of {a,c,d} follow ‘b’
• All the permutations of {a,b,d} follow ‘c’
• All the permutations of {a,b,c} follow ‘d’
Factorial Function

• n! = n*(n -1)*(n - 2)*... *2 *1 for any integer n>0

0! =1

• Iterative Definition:
•
int fval=1;

for(i=n; i>=1; i--)

fval = fval*i;
Factorial Function
Recursive Definition

• To define n! recursively, n! must be defined in terms of the factorial of a

smaller number.

• Observation (problem size is reduced):

n! = n *(n -1)!

• Base case:
0! =1

• We can reach the base case by subtracting 1 from n if n is a positive

integer.
Factorial Function
Recursive Definition

Recursive definition
n! =1 if n = 0
n! = n *(n -1)! if n > 0
int fact(int n)
{

if(n==0)
return (1);
else
return (n*fact(n-1));
}

This fact function satisfies the four criteria of a recursive solution.

Four Criteria of a Recursive Solution
Fibonacci Sequence

• It is the sequence of integers:

t0 t1 t2 t3 t4 t5 t6 …

0 1 1 2 3 5 8 …

• Each element in this sequence is the sum of the two preceding

elements.
• The specification of the terms in the Fibonacci sequence:

n if n is 0 or 1 (i.e. n < 2)
tn =
tn-1 + tn-2 {otherwise
 Fibonacci Sequence
• Iterative solution

int fibonacci(int n)
{

int prev1, prev2, temp, j;

if(n==0 || n==1)
return n;
else{
prev1=0;
prev2=1;
for(j=1;j<=n;j++)
{
temp = prev1+prev2;
prev1=prev2;
prev2=temp;
}
return prev2;
}
}
Fibonacci Sequence

• Recursive solution
int recfibonacci(int n)
{

if(n<2)
return n;
else
return (recfibonacci(n-2)+recfibonacci(n-1));
}
Performance Analysis
Running Time
Observations
Example-1
Observations
Example-1: 3-SUM brute force algorithm
int count(int a[], int size)
{
int count = 0;
for (int i=0; i<size; i++)
for(int j=i+1; j<size; j++)
for(int k=j+1; k<size; k++)
if(a[i]+a[j]+a[k] ==0) Check each triple
count++; For simplicity, ignore integer overflow
return count;
}
Empirical Analysis
Data Analysis
Mathematical Models for Running Time
Cost of Basic Operations
Cost of Basic Operations
Example: 1-Sum
Example: 2-Sum
Example: 2-Sum
Simplifying the Calculations
Simplification: cost model
Asymptotic Notation: 2-SUM problem
Asymptotic Notation: 3-SUM problem
Binary search
Invariant: If key appears in array a[ ], then a[lo] ≤ key ≤ a[hi]

int binarySearch(int a[], int size, int key)

{
int lo = 0, hi = size - 1;
while(lo <= hi) Why not mid = (lo + hi) / 2 ?
{
int mid = lo + (hi - lo) / 2;
if(key < a[mid]) hi = mid - 1;
one 3-way
else if(key > a[mid]) lo = mid + 1; compare
else return mid;
}
return -1;
}
Binary search: mathematical analysis
Types of Analyses
Types of Analyses

morphemes
Mathematical Models for Running Time
Asymptotic notation: O (Big Oh)
Question
Performance Analysis

• There are various criteria in order to evaluate a program.

• The questions:
• Does the program meet the requirements determined at the beginning?
• Does it work correctly?
• Does it have any documentation on the subject of how it works?
• Does it use the function effectively in order to produce the logical values:
TRUE and FALSE?
• Is the program code readable?
• Does the program use the primary and the secondary memory effectively?
• Is the running time of the program acceptable?
Performance Analysis

Performance Evaluation

Space Complexity Time Complexity

 Space Complexity

• Memory space needed by a program to complete its execution:

• A fixed part → c
• A variable part → Sp(instance)

S(P) = c + Sp(instance)

float sum(float list[], int n){

float tempsum=0;
float abc(float a, float b, float c){ int i;
return a+b+b*c+(a+b+c)/(b+c); for(i=0;i<n;i++){
}
Sabc(I)=0 tempsum += list[i];
Ssumreturn
(n)=0 tempsum;
}
Sabc(instance) = 0
Ssum(n) = 0
Space Complexity

• Recursive function call:

float rsum(float list[], int n){

if(n)
return rsum(list, n-1)+list[n-1];
return 0;
}

• What is the total variable memory space of this method for an input
size of 2000?
Time Complexity

• Total time used by the program is: T(P)

• Where T(P)=compile time + run (execution) time (TP)

float sum(float list[], int n){

float tempsum=0; …………………………………………………… 1
int i;
for(i=0;i<n;i++){ …………………………………………………… n+1
tempsum += list[i]; ………………………………………… n
return tempsum; ……………………………………………………………… 1
}

Total step number = 2n+3

Time Complexity

float rsum(float list[], int n){

if(n)
return rsum(list, n-1)+list[n-1];
return 0;
}

• What is the number of steps required to execute this method?

2n+2
Time Complexity

void add(int a[MAX_SIZE]){

int i,j;
for(i=0; i<rows; i++)
for(j=0; j<cols; j++)
c[i][j]=a[i][j]+b[i][j];

• What is the number of steps required to execute this method?

2rows*cols+2rows+1
O (Big Oh) Notation

• Time complexity = algorithm complexity

• f(n)<cg(n) → f(n)=O(g(n))

• Tsum(n)=2n+3 → Tsum(n)=O(n)

• Trsum(n)=2n+2 → Tsum(n)=O(n)

• Tadd(rows,cols)=2rows*cols+2rows+1 → Tadd(n)=O(rows*cols)
=O(n2)
Asymptotic Notation

• Example: f(n)=3n3+6n2+7n

• What is the algorithm complexity in O notation?

Practical Complexities

Complexity Name 1 2 4 8 16 32
(time)
1 constant 1 1 1 1 1 1

logn logarithmic 0 1 2 3 4 5

n linear 1 2 4 8 16 32

nlogn log linear 0 2 8 24 64 160

n2 quadratic 1 4 16 64 256 1024

n3 cubic 1 8 64 512 4096 32768

2n exponential 2 4 16 256 65536 4294967296

n! factorial 1 2 24 40326 2092278 26313*1033

9888000
Practical Complexities
n f(n)=n f(n)=log2n f(n)=n2 f(n)=n3 f(n)=n4 f(n)=n10 f(n)=2n

10 0.01μ 0.03μ 0.1μ 1μ 10μ 10 sec 1μ

20 0.02μ 0.09μ 0.4μ 8μ 160μ 2.84 hr 1 ms

30 0.03μ 0.15μ 0.9μ 27μ 810μ 6.83 d 1 sec

40 0.04μ 0.21μ 1.6μ 64μ 2.56ms 121.36 d 18.3 min

50 0.05μ 0.28μ 2.5μ 125μ 6.25ms 3.1 yr 13 d

100 0.10μ 0.66μ 10μ 1ms 100ms 3171 yr 4*1013 yr

1000 1μ 9.96μ 1ms 1sec 16.67min 3.17*1013 yr

10.000 10μ 130.03μ 100ms 16.67min 115.7d 3.17*1023 yr

100.000 100μ 1.66ms 10sec 11.57d 3171yr 3.17*1033 yr

1.000.000 1ms 19.92ms 16.67min 31.71yr 3.17*107 3.17*1043 yr

yr
Survey of Common Running Times
Tilde notation
Tilde Notation (~) (same as θ-notation)
Asymptotic notation: Ω (Omega)
Asymptotic notation: Θ (theta)
(same as “~” notation)
Linear time: O(n)
Linear time: O(n)
Linearithmic time: O(n log n)
Quadratic time: O(n2)
Cubic time: O(n3)
Polynomial time: O(nk)
Exponential time
Sublinear time
References

• Kevin Wayne, “Analysis of Algorithms”

• Sartaj Sahni, “Analysis of Algorithms”
• Marty Stepp and Helene Martin, Recursion