Chapter 4 – Sorting Techniques Data Structures and Algorithm

Sorting refers to arranging data in a particular format. Sorting algorithm specifies the way to
arrange data in a particular order. Most common orders are in numerical or lexicographical order.
The importance of sorting lies in the fact that data searching can be optimized to a very high
level, if data is stored in a sorted manner. Sorting is also used to represent data in more readable
formats. Following are some of the examples of sorting in real-life scenarios −
 Telephone Directory − The telephone directory stores the telephone numbers of people
sorted by their names, so that the names can be searched easily.
 Dictionary − The dictionary stores words in an alphabetical order so that searching of any
word becomes easy.

In-place Sorting and Not-in-place Sorting

Sorting algorithms may require some extra space for comparison and temporary storage of few
data elements. These algorithms do not require any extra space and sorting is said to happen in-
place, or for example, within the array itself. This is called in-place sorting. Bubble sort is an
example of in-place sorting.
However, in some sorting algorithms, the program requires space which is more than or equal to
the elements being sorted. Sorting which uses equal or more space is called not-in-place
sorting. Merge-sort is an example of not-in-place sorting.

Stable and Not Stable Sorting

If a sorting algorithm, after sorting the contents, does not change the sequence of similar content
in which they appear, it is called stable sorting.

If a sorting algorithm, after sorting the contents, changes the sequence of similar content in which
they appear, it is called unstable sorting.

45
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Stability of an algorithm matters when we wish to maintain the sequence of original elements, like
in a tuple for example.

Adaptive and Non-Adaptive Sorting Algorithm

A sorting algorithm is said to be adaptive, if it takes advantage of already 'sorted' elements in the
list that is to be sorted. That is, while sorting if the source list has some element already sorted,
adaptive algorithms will take this into account and will try not to re-order them.
A non-adaptive algorithm is one which does not take into account the elements which are already
sorted. They try to force every single element to be re-ordered to confirm their sortedness.

Important Terms
Some terms are generally coined while discussing sorting techniques, here is a brief introduction
to them −

Increasing Order
A sequence of values is said to be in increasing order, if the successive element is greater than
the previous one. For example, 1, 3, 4, 6, 8, 9 are in increasing order, as every next element is
greater than the previous element.

Decreasing Order
A sequence of values is said to be in decreasing order, if the successive element is less than
the current one. For example, 9, 8, 6, 4, 3, 1 are in decreasing order, as every next element is
less than the previous element.

Non-Increasing Order
A sequence of values is said to be in non-increasing order, if the successive element is less
than or equal to its previous element in the sequence. This order occurs when the sequence
contains duplicate values. For example, 9, 8, 6, 3, 3, 1 are in non-increasing order, as every next
element is less than or equal to (in case of 3) but not greater than any previous element.

Non-Decreasing Order
A sequence of values is said to be in non-decreasing order, if the successive element is greater
than or equal to its previous element in the sequence. This order occurs when the sequence
contains duplicate values. For example, 1, 3, 3, 6, 8, 9 are in non-decreasing order, as every next
element is greater than or equal to (in case of 3) but not less than the previous one.
Bubble sort is a simple sorting algorithm. This sorting algorithm is comparison-based algorithm in
which each pair of adjacent elements is compared and the elements are swapped if they are not
in order. This algorithm is not suitable for large data sets as its average and worst case
complexity are of Ο(n2) where n is the number of items.

46
Chapter 4 – Sorting Techniques Data Structures and Algorithm

How Bubble Sort Works?

We take an unsorted array for our example. Bubble sort takes Ο(n 2) time so we're keeping it short
and precise.

Bubble sort starts with very first two elements, comparing them to check which one is greater.

In this case, value 33 is greater than 14, so it is already in sorted locations. Next, we compare 33
with 27.

We find that 27 is smaller than 33 and these two values must be swapped.

The new array should look like this −

Next we compare 33 and 35. We find that both are in already sorted positions.

Then we move to the next two values, 35 and 10.

We know then that 10 is smaller 35. Hence they are not sorted.

We swap these values. We find that we have reached the end of the array. After one iteration, the
array should look like this −

47
Chapter 4 – Sorting Techniques Data Structures and Algorithm

To be precise, we are now showing how an array should look like after each iteration. After the
second iteration, it should look like this −

Notice that after each iteration, at least one value moves at the end.

And when there's no swap required, bubble sorts learns that an array is completely sorted.

Now we should look into some practical aspects of bubble sort.

Algorithm
We assume list is an array of n elements. We further assume that swap function swaps the
values of the given array elements.
begin BubbleSort(list)

for all elements of list

if list[i] > list[i+1]
swap(list[i], list[i+1])
end if
end for

return list

end BubbleSort

Pseudocode
We observe in algorithm that Bubble Sort compares each pair of array element unless the whole
array is completely sorted in an ascending order. This may cause a few complexity issues like
what if the array needs no more swapping as all the elements are already ascending.
To ease-out the issue, we use one flag variable swapped which will help us see if any swap has
happened or not. If no swap has occurred, i.e. the array requires no more processing to be
sorted, it will come out of the loop.
Pseudocode of BubbleSort algorithm can be written as follows −
procedure bubbleSort( list : array of items )

loop = list.count;

for i = 0 to loop-1 do:

swapped = false

48
Chapter 4 – Sorting Techniques Data Structures and Algorithm

for j = 0 to loop-1 do:

/* compare the adjacent elements */

if list[j] > list[j+1] then
/* swap them */
swap( list[j], list[j+1] )
swapped = true
end if

end for

/*if no number was swapped that means

array is sorted now, break the loop.*/

if(not swapped) then

break
end if

end for

end procedure return list

Implementation
One more issue we did not address in our original algorithm and its improvised pseudocode, is
that, after every iteration the highest values settles down at the end of the array. Hence, the next
iteration need not include already sorted elements. For this purpose, in our implementation, we
restrict the inner loop to avoid already sorted values.
This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained which is
always sorted. For example, the lower part of an array is maintained to be sorted. An element
which is to be 'insert'ed in this sorted sub-list, has to find its appropriate place and then it has to
be inserted there. Hence the name, insertion sort.
The array is searched sequentially and unsorted items are moved and inserted into the sorted
sub-list (in the same array). This algorithm is not suitable for large data sets as its average and
worst case complexity are of Ο(n2), where n is the number of items.

How Insertion Sort Works?

We take an unsorted array for our example.

Insertion sort compares the first two elements.

It finds that both 14 and 33 are already in ascending order. For now, 14 is in sorted sub-list.

49
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Insertion sort moves ahead and compares 33 with 27.

And finds that 33 is not in the correct position.

It swaps 33 with 27. It also checks with all the elements of sorted sub-list. Here we see that the
sorted sub-list has only one element 14, and 27 is greater than 14. Hence, the sorted sub-list
remains sorted after swapping.

By now we have 14 and 27 in the sorted sub-list. Next, it compares 33 with 10.

These values are not in a sorted order.

So we swap them.

However, swapping makes 27 and 10 unsorted.

Hence, we swap them too.

Again we find 14 and 10 in an unsorted order.

50
Chapter 4 – Sorting Techniques Data Structures and Algorithm

We swap them again. By the end of third iteration, we have a sorted sub-list of 4 items.

This process goes on until all the unsorted values are covered in a sorted sub-list. Now we shall
see some programming aspects of insertion sort.

Algorithm
Now we have a bigger picture of how this sorting technique works, so we can derive simple steps
by which we can achieve insertion sort.
Step 1 − If it is the first element, it is already sorted. return 1;
Step 2 − Pick next element
Step 3 − Compare with all elements in the sorted sub-list
Step 4 − Shift all the elements in the sorted sub-list that is greater than
the
value to be sorted
Step 5 − Insert the value
Step 6 − Repeat until list is sorted

Pseudocode
procedure insertionSort( A : array of items )
int holePosition
int valueToInsert

for i = 1 to length(A) inclusive do:

/* select value to be inserted */

valueToInsert = A[i]
holePosition = i

/*locate hole position for the element to be inserted */

while holePosition > 0 and A[holePosition-1] > valueToInsert do:

A[holePosition] = A[holePosition-1]
holePosition = holePosition -1
end while

/* insert the number at hole position */

A[holePosition] = valueToInsert

end for

end procedure

Selection sort is a simple sorting algorithm. This sorting algorithm is an in-place comparison-
based algorithm in which the list is divided into two parts, the sorted part at the left end and the
unsorted part at the right end. Initially, the sorted part is empty and the unsorted part is the entire
list.

51
Chapter 4 – Sorting Techniques Data Structures and Algorithm

The smallest element is selected from the unsorted array and swapped with the leftmost element,
and that element becomes a part of the sorted array. This process continues moving unsorted
array boundary by one element to the right.
This algorithm is not suitable for large data sets as its average and worst case complexities are of
Ο(n2), where n is the number of items.

How Selection Sort Works?

Consider the following depicted array as an example.

For the first position in the sorted list, the whole list is scanned sequentially. The first position
where 14 is stored presently, we search the whole list and find that 10 is the lowest value.

So we replace 14 with 10. After one iteration 10, which happens to be the minimum value in the
list, appears in the first position of the sorted list.

For the second position, where 33 is residing, we start scanning the rest of the list in a linear
manner.

We find that 14 is the second lowest value in the list and it should appear at the second place.
We swap these values.

After two iterations, two least values are positioned at the beginning in a sorted manner.

The same process is applied to the rest of the items in the array.
Following is a pictorial depiction of the entire sorting process −

52
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Now, let us learn some programming aspects of selection sort.

Algorithm
Step 1 − Set MIN to location 0
Step 2 − Search the minimum element in the list
Step 3 − Swap with value at location MIN
Step 4 − Increment MIN to point to next element
Step 5 − Repeat until list is sorted

Pseudocode
procedure selection sort
list : array of items
n : size of list

53
Chapter 4 – Sorting Techniques Data Structures and Algorithm

for i = 1 to n - 1
/* set current element as minimum*/
min = i

/* check the element to be minimum */

for j = i+1 to n
if list[j] < list[min] then
min = j;
end if
end for

/* swap the minimum element with the current element*/

if indexMin != i then
swap list[min] and list[i]
end if
end for

end procedure

Merge sort is a sorting technique based on divide and conquer technique. With worst-case time
complexity being Ο(n log n), it is one of the most respected algorithms.
Merge sort first divides the array into equal halves and then combines them in a sorted manner.

How Merge Sort Works?

To understand merge sort, we take an unsorted array as the following −

We know that merge sort first divides the whole array iteratively into equal halves unless the
atomic values are achieved. We see here that an array of 8 items is divided into two arrays of
size 4.

This does not change the sequence of appearance of items in the original. Now we divide these
two arrays into halves.

We further divide these arrays and we achieve atomic value which can no more be divided.

Now, we combine them in exactly the same manner as they were broken down. Please note the
color codes given to these lists.

54
Chapter 4 – Sorting Techniques Data Structures and Algorithm

We first compare the element for each list and then combine them into another list in a sorted
manner. We see that 14 and 33 are in sorted positions. We compare 27 and 10 and in the target
list of 2 values we put 10 first, followed by 27. We change the order of 19 and 35 whereas 42 and
44 are placed sequentially.

In the next iteration of the combining phase, we compare lists of two data values, and merge
them into a list of found data values placing all in a sorted order.

After the final merging, the list should look like this −

Now we should learn some programming aspects of merge sorting.

Algorithm
Merge sort keeps on dividing the list into equal halves until it can no more be divided. By
definition, if it is only one element in the list, it is sorted. Then, merge sort combines the smaller
sorted lists keeping the new list sorted too.
Step 1 − if it is only one element in the list it is already sorted,
return.
Step 2 − divide the list recursively into two halves until it can no more
be divided.
Step 3 − merge the smaller lists into new list in sorted order.

Pseudocode
We shall now see the pseudocodes for merge sort functions. As our algorithms point out two
main functions − divide & merge.
Merge sort works with recursion and we shall see our implementation in the same way.
procedure mergesort( var a as array )
if ( n == 1 ) return a

var l1 as array = a[0] ... a[n/2]

var l2 as array = a[n/2+1] ... a[n]

l1 = mergesort( l1 )
l2 = mergesort( l2 )

return merge( l1, l2 )

end procedure

procedure merge( var a as array, var b as array )

55
Chapter 4 – Sorting Techniques Data Structures and Algorithm

var c as array
while ( a and b have elements )
if ( a[0] > b[0] )
add b[0] to the end of c
remove b[0] from b
else
add a[0] to the end of c
remove a[0] from a
end if
end while

while ( a has elements )

add a[0] to the end of c
remove a[0] from a
end while

while ( b has elements )

add b[0] to the end of c
remove b[0] from b
end while

return c

end procedure

Shell sort is a highly efficient sorting algorithm and is based on insertion sort algorithm. This
algorithm avoids large shifts as in case of insertion sort, if the smaller value is to the far right and
has to be moved to the far left.
This algorithm uses insertion sort on a widely spread elements, first to sort them and then sorts
the less widely spaced elements. This spacing is termed as interval. This interval is calculated
based on Knuth's formula as −

Knuth's Formula
h = h * 3 + 1
where −
h is interval with initial value 1
This algorithm is quite efficient for medium-sized data sets as its average and worst-case
complexity of this algorithm depends on the gap sequence the best known is Ο(n), where n is the
number of items. And the worst case space complexity is O(n).

How Shell Sort Works?

Let us consider the following example to have an idea of how shell sort works. We take the same
array we have used in our previous examples. For our example and ease of understanding, we
take the interval of 4. Make a virtual sub-list of all values located at the interval of 4 positions.
Here these values are {35, 14}, {33, 19}, {42, 27} and {10, 44}

56
Chapter 4 – Sorting Techniques Data Structures and Algorithm

We compare values in each sub-list and swap them (if necessary) in the original array. After this
step, the new array should look like this −

Then, we take interval of 1 and this gap generates two sub-lists - {14, 27, 35, 42}, {19, 10, 33, 44}

We compare and swap the values, if required, in the original array. After this step, the array
should look like this −

Finally, we sort the rest of the array using interval of value 1. Shell sort uses insertion sort to sort
the array.
Following is the step-by-step depiction −

57
Chapter 4 – Sorting Techniques Data Structures and Algorithm

We see that it required only four swaps to sort the rest of the array.

Algorithm
Following is the algorithm for shell sort.
Step 1 − Initialize the value of h
Step 2 − Divide the list into smaller sub-list of equal interval h
Step 3 − Sort these sub-lists using insertion sort
Step 3 − Repeat until complete list is sorted

Pseudocode

58
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Following is the pseudocode for shell sort.

procedure shellSort()
A : array of items

/* calculate interval*/
while interval < A.length /3 do:
interval = interval * 3 + 1
end while

while interval > 0 do:

for outer = interval; outer < A.length; outer ++ do:

/* select value to be inserted */

valueToInsert = A[outer]
inner = outer;

/*shift element towards right*/

while inner > interval -1 && A[inner - interval] >= valueToInsert
do:
A[inner] = A[inner - interval]
inner = inner - interval
end while

/* insert the number at hole position */

A[inner] = valueToInsert

end for

/* calculate interval*/
interval = (interval -1) /3;

end while

end procedure

Quick sort is a highly efficient sorting algorithm and is based on partitioning of array of data into
smaller arrays. A large array is partitioned into two arrays one of which holds values smaller than
the specified value, say pivot, based on which the partition is made and another array holds
values greater than the pivot value.
Quicksort partitions an array and then calls itself recursively twice to sort the two resulting
subarrays. This algorithm is quite efficient for large-sized data sets as its average and worst-case
complexity are O(n2), respectively.

Partition in Quick Sort

Following animated representation explains how to find the pivot value in an array.

59
Chapter 4 – Sorting Techniques Data Structures and Algorithm

The pivot value divides the list into two parts. And recursively, we find the pivot for each sub-lists
until all lists contains only one element.

Quick Sort Pivot Algorithm

Based on our understanding of partitioning in quick sort, we will now try to write an algorithm for
it, which is as follows.
Step 1 − Choose the highest index value has pivot
Step 2 − Take two variables to point left and right of the list excluding
pivot
Step 3 − left points to the low index
Step 4 − right points to the high
Step 5 − while value at left is less than pivot move right
Step 6 − while value at right is greater than pivot move left
Step 7 − if both step 5 and step 6 does not match swap left and right
Step 8 − if left ≥ right, the point where they met is new pivot

Quick Sort Pivot Pseudocode

The pseudocode for the above algorithm can be derived as −
function partitionFunc(left, right, pivot)
leftPointer = left
rightPointer = right - 1

while True do
while A[++leftPointer] < pivot do
//do-nothing
end while

while rightPointer > 0 && A[--rightPointer] > pivot do

//do-nothing
end while

if leftPointer >= rightPointer

break
else
swap leftPointer,rightPointer
end if

end while

60
Chapter 4 – Sorting Techniques Data Structures and Algorithm

swap leftPointer,right
return leftPointer

end function

Quick Sort Algorithm

Using pivot algorithm recursively, we end up with smaller possible partitions. Each partition is
then processed for quick sort. We define recursive algorithm for quicksort as follows −
Step 1 − Make the right-most index value pivot
Step 2 − partition the array using pivot value
Step 3 − quicksort left partition recursively
Step 4 − quicksort right partition recursively

Quick Sort Pseudocode

To get more into it, let see the pseudocode for quick sort algorithm −
procedure quickSort(left, right)

if right-left <= 0
return
else
pivot = A[right]
partition = partitionFunc(left, right, pivot)
quickSort(left,partition-1)
quickSort(partition+1,right)
end if

end procedure

A graph is a pictorial representation of a set of objects where some pairs of objects are connected
by links. The interconnected objects are represented by points termed as vertices, and the links
that connect the vertices are called edges.
Formally, a graph is a pair of sets (V, E), where V is the set of vertices and E is the set of edges,
connecting the pairs of vertices. Take a look at the following graph −

In the above graph,

V = {a, b, c, d, e}
E = {ab, ac, bd, cd, de}

Graph Data Structure

61
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Mathematical graphs can be represented in data structure. We can represent a graph using an
array of vertices and a two-dimensional array of edges. Before we proceed further, let's
familiarize ourselves with some important terms −
 Vertex − Each node of the graph is represented as a vertex. In the following example, the
labeled circle represents vertices. Thus, A to G are vertices. We can represent them using
an array as shown in the following image. Here A can be identified by index 0. B can be
identified using index 1 and so on.
 Edge − Edge represents a path between two vertices or a line between two vertices. In the
following example, the lines from A to B, B to C, and so on represents edges. We can use
a two-dimensional array to represent an array as shown in the following image. Here AB
can be represented as 1 at row 0, column 1, BC as 1 at row 1, column 2 and so on,
keeping other combinations as 0.
 Adjacency − Two node or vertices are adjacent if they are connected to each other
through an edge. In the following example, B is adjacent to A, C is adjacent to B, and so
on.
 Path − Path represents a sequence of edges between the two vertices. In the following
example, ABCD represents a path from A to D.

Basic Operations
Following are basic primary operations of a Graph −
 Add Vertex − Adds a vertex to the graph.
 Add Edge − Adds an edge between the two vertices of the graph.
 Display Vertex − Displays a vertex of the graph.
Depth First Search (DFS) algorithm traverses a graph in a depthward motion and uses a stack to
remember to get the next vertex to start a search, when a dead end occurs in any iteration.

62
Chapter 4 – Sorting Techniques Data Structures and Algorithm

As in the example given above, DFS algorithm traverses from S to A to D to G to E to B first, then
to F and lastly to C. It employs the following rules.
 Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it. Push it in a stack.
 Rule 2 − If no adjacent vertex is found, pop up a vertex from the stack. (It will pop up all the
vertices from the stack, which do not have adjacent vertices.)
 Rule 3 − Repeat Rule 1 and Rule 2 until the stack is empty.

Step Traversal Description

Initialize the stack.

63
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Mark S as visited and put it onto the

stack. Explore any unvisited adjacent
node from S. We have three nodes and
we can pick any of them. For this
example, we shall take the node in an
alphabetical order.

Mark A as visited and put it onto the

stack. Explore any unvisited adjacent
node from A. Both S and D are
adjacent to A but we are concerned for
unvisited nodes only.

Visit D and mark it as visited and put

onto the stack. Here, we
have B and C nodes, which are
adjacent to D and both are unvisited.
However, we shall again choose in an
alphabetical order.

We choose B, mark it as visited and

put onto the stack. Here B does not
have any unvisited adjacent node. So,
we pop B from the stack.

64
Chapter 4 – Sorting Techniques Data Structures and Algorithm

We check the stack top for return to the

previous node and check if it has any
unvisited nodes. Here, we find D to be
on the top of the stack.

Only unvisited adjacent node is

from D is C now. So we visit C, mark it
as visited and put it onto the stack.

As C does not have any unvisited adjacent node so we keep popping the stack until we find a
node that has an unvisited adjacent node. In this case, there's none and we keep popping until
the stack is empty.
Breadth First Search (BFS) algorithm traverses a graph in a breadthward motion and uses a
queue to remember to get the next vertex to start a search, when a dead end occurs in any
iteration.

65
Chapter 4 – Sorting Techniques Data Structures and Algorithm

As in the example given above, BFS algorithm traverses from A to B to E to F first then to C and
G lastly to D. It employs the following rules.
 Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it. Insert it in a
queue.
 Rule 2 − If no adjacent vertex is found, remove the first vertex from the queue.
 Rule 3 − Repeat Rule 1 and Rule 2 until the queue is empty.

Step Traversal Description

Initialize the queue.

We start from visiting S (starting node),

and mark it as visited.

We then see an unvisited adjacent

node from S. In this example, we have
three nodes but alphabetically we
choose A, mark it as visited and
enqueue it.

66
Chapter 4 – Sorting Techniques Data Structures and Algorithm

Next, the unvisited adjacent node

from S is B. We mark it as visited and
enqueue it.

Next, the unvisited adjacent node

from S is C. We mark it as visited and
enqueue it.

Now, S is left with no unvisited

adjacent nodes. So, we dequeue and
find A.

From A we have D as unvisited

adjacent node. We mark it as visited
and enqueue it.

At this stage, we are left with no unmarked (unvisited) nodes. But as per the algorithm we keep
on dequeuing in order to get all unvisited nodes. When the queue gets emptied, the program is
over.

67