Introduction/Overview

Insertion Sort
Getting started
 We will use the insertion sort algorithm to:
 Define the pseudo code we are use.
 Analyze its running time.

 Introduce the divide-and-conquer approach to the

design of algorithms and use it to develop an
algorithm called merge sort.

 We end with analysis of the merge sort running

time.

2
Insertion Sort
 If first two books are out of order
 Remove second book
 Slide first book to right
 Insert removed book into first slot
 Then look at third book, if it is out of order
 Remove that book
 Slide 2nd book to right
 Insert removed book into 2nd slot
 Recheck first two books again
 Etc.

3
Insertion Sort

The placement of the

third book during an
insertion sort.

4
Insertion Sort

An insertion sort of books

• Insertion Sort can be written iteratively or recursively
5
Insertion Sort Execution Example
9 7 6 15 16 5 10 11
9 7 6 15 16 5 10 11
7 9 6 15 16 5 10 11

6 7 9 15 16 5 10 11
6 7 9 15 16 5 10 11
6 7 9 15 16 5 10 11

5 6 7 9 15 16 10 11

5 6 7 9 10 15 16 11

5 6 7 9 10 11 15 16
6
 Insertion sort
 Input: A sequence of n numbers {a3 , a1, a2,...,an }
 Output: A reordered sequence of the input {a1 , a2,
a3,...,an } such that a1≤a2 ≤a3… ≤an

 Sorted array/list is by inserting one item at a

time
– Simple to implement
– Efficient on small data sets
– Efficient on already almost ordered data sets
7
Insertion Sort
 Start with a sequence of size 1
 Repeatedly insert remaining elements

8
An Example: Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i] next key
j = i - 1; go left
while (j > 0) and (A[j] > key) { find place for key
A[j+1] = A[j] shift sorted right
j=j–1 go left
}
A[j+1] = key put key in place

}
}

9
An Example: Insertion Sort
30 10 40 20 i =  j =  key = 
A[j] =  A[j+1] = 
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

10
An Example: Insertion Sort
30 10 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

11
An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

12
An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

13
An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

14
An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

15
An Example: Insertion Sort
10 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

16
An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

17
An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

18
An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

19
An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

20
An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

21
An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

22
An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

23
An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

24
An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

25
An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

26
An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

27
An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

28
An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

29
An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

30
An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

31
An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

32
An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

33
An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

34
An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

35
An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

36
An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

37
An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
Done!
38
Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
How many times will
this while loop execute?

39
Insertion Sort
 This algorithm takes as parameter an array of the
sequence numbers to be sorted A[1 … n], .
 When this algorithm finish, the input array will
contains the sorted sequence numbers (sort in
place: the sorted items (output) occupy the same
storage as the original ones (input) ).

40
Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm on any

input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm over all

inputs of size n.
• Need assumption of statistical distribution of

inputs.
Best-case: (NEVER)
• Cheat with a slow algorithm that works fast on

some input.
41
Analyzing algorithms
 We need to predict the resources ( memory,
communication bandwidth, or computer hardware) that
the algorithm requires.

 Most often it is computational time that we want to

measure.

 By analyzing several candidate algorithms for a

problem, the most efficient one can be easily identified.

 Unless other thing specified, we shall assume that our

algorithms will be implemented on a generic one
processor computer with the instructions being executed
one after another, with no concurrent operations.
42
Analysis of insertion sort
 The time taken by insertion sort depends on the input
(sorting thousand items takes more time than sorting three
items).

 In general, the time taken by an algorithm grows with the

size of the input, so it is traditional to describe the running
time of a program as a function of the size of its input.

 Input size depends on the problem being studied

( Ex: sort algorithm: the number of items to be sorted.
Multiplying two integers: the total number of bits
needed to represent the input in ordinary binary notation.
In graphs: the input size can be described by the
numbers of vertices and edges in the graph).
43
44
Analysis of insertion sort
 The running time of an algorithm on a particular input is
the number of primitive operations or "steps" executed.

 A constant amount of time is required to execute each

line of our pseudo-code. One line may take a different
amount of time than another line, but we shall assume
that each execution of the ith line takes time ci , where ci
is a constant.

 We start by presenting the INSERTION-SORT procedure

with the time "cost" of each statement and the number of
times each statement is executed.
45
Analysis of insertion sort
 For each i = 2, 3, . . . , n, where n = length[A], we let j be
the number of times the while loop test is executed for
that value of i. When a for or while loop exits in the usual
way (i.e., due to the test in the loop header), the test is
executed one time more than the loop body.

 We assume that comments are not executable statements,

and so they take no time.

 The running time of the algorithm is the sum of running

times for each statement executed; a statement that takes
ci steps to execute and is executed n times will contribute
ci n to the total running time.

46
Insertion Sort
Statement Effort
InsertionSort(A, n) {
for i = 2 to n { c1 n
key = A[i] c2(n-1)
j = i - 1 c3(n-1)
while (j > 0) and (A[j] > key) { c4 C
A[j+1] = A[j] c5(C-(n-1))
j = j – 1} c6(C-(n-1))
A[j+1] = key c7(n-1)
}}

C = t2 + t3 + … + tn where ti is number of while expression evaluations

for the ith for loop iteration

Body of the while statement is executed= (t2 - 1) + (t3 – 1) + … + (tn – 1)

= t2 + t3 + … + tn – (n-1) = C – (n-1)

47
 Analyzing Insertion Sort
T(n) = c1n + c2(n-1) + c3(n-1) + c4C + c5(C - (n-1)) + c6(C - (n-
1)) + c7(n-1)

= c8C + c9n + c10

 What can C be? c1n

c2(n-1)
 Best case -- inner loop body never executed c3(n-1)
(array is sorted) c4C
 t = 1  T(n) is a linear function = a.n + b c5(C-(n-1))
i
 Worst case -- inner loop body executed for all c6(C-(n-1))
c7(n-1)
previous elements (array sorted in reverse order)
 t = i  T(n) is a quadratic function = a.n2 +
i
b.n + c
 If T is a quadratic function, which terms in the
above equation matter?

48
Analyzing Insertion Sort (Cont.)
 Best Case
 If A is sorted: O(n) comparisons
 linear function of n

 Worst Case
 If A is reversed sorted: O(n2) comparisons
 quadratic function of n

 Average Case
 If A is randomly sorted: O(n2) comparisons

49
 Worst-case and average-case analysis
 The worst-case running time of an algorithm is an upper
bound on the running time for any input.
 Knowing it gives us a guarantee that the algorithm will never take any
longer.

 The "average case" is often roughly as bad as the worst case.

 Suppose that we randomly choose n numbers and apply insertion sort.
How long does it take to determine where in sub-array A[1… i -1]to
insert element A[i], On average, half the elements in A[1…i - 1] are
less than A[i], and half the elements are greater.

50
Insertion sort – Order of growth
 This worst-case running time can be expressed as an 2 +
bn + c for constants a, b, and c

 To find the order of growth we consider only the leading

term of a formula (e.g, an2 ), since the lower-order terms
are relatively insignificant for large n.

 We also ignore the leading term's constant coefficient,

since constant factors are less significant than the rate of
growth in determining computational efficiency for large
inputs.

51
Insertion sort – Order of growth
 We usually consider one algorithm to be more
efficient than another if its worst-case running
time has a lower order of growth.

 Due to constant factors and lower-order terms,

this evaluation may be in error for small inputs.
But not for large enough inputs

52