SORTING

B SIVA JYOTHI
SORTING THEORY
The basic problem in sorting is as follows:
Given an array that contains n elements, your task is to
sort the elements in increasing order.
For example, the array
1 2 3 4 5 6 7 8
1 3 8 2 9 2 5 6

will be as follows after sorting:

1 2 3 4 5 6 7 8
1 2 2 3 5 6 8 9
Introduction

• Sorting refers to arranging a set of data in some

logical order

• For ex. A telephone directory can be considered

as a list where each record has three fields -
name, address and phone number.

• Being unique, phone number can work as a key to

locate any record in the list.

3
Introduct
ion
• Sorting is among the most basic problems in
algorithm design.

• We are given a sequence of items, each associated

with a given key value. And the problem is to
rearrange the items so that they are in an
increasing(or decreasing) order by key.

• The methods of sorting can be divided into two

categories:
• Internal Sorting
• External Sorting
4
• Internal Sorting
If all the data that is to be sorted can be adjusted at
a time in main memory, then internal sorting
methods are used

• External Sorting
When the data to be sorted can’t be accommodated in
the memory at the same time and some has to be
kept in auxiliary memory, then external sorting
methods are used.

 NOTE: We will only consider internal sorting

5
Stable and Not Stable
Sorting
• If a sorting algorithm, after sorting the contents,
does not change the sequence of similar content
in which they appear, it is called stable sorting.

• If a sorting algorithm, after sorting the contents,

changes the sequence of similar content in which
they appear, it is called unstable sorting.

6
 Efficiency of Sorting
Algorithm
• The complexity of a sorting algorithm measures the
running time of a function in which n number of items
are to be sorted.

• The choice of sorting method depends on efficiency

considerations for different problems.
• Three most important of these considerations are:
• The length of time spent by programmer in coding a particular
sorting program
• Amount of machine time necessary for running the program
• The amount of memory necessary for running the program

7
Efficiency of Sorting
Algorithm
• Various sorting methods are analyzed in the
cases like – best case, worst case or average
case.

• Most of the sort methods we consider have

requirements that range from 0(nlogn) to 0(n2).

• A sort should not be selected only because its

sorting time is 0(nlogn); the relation of the file
size n and the other factors affecting the actual
sorting time must be considered
8
Efficiency of Sorting
Algorithm
• Determining the time requirement of sorting
technique is to actually run the program and
measure its efficiency.

• Once a particular sorting technique is selected the

need is to make the program as efficient as
possible.

• Any improvement in sorting time

significantly affect the overall efficiency
and saves a great deal of computer time.
9
Efficiency of Sorting
Algorithm
• Space constraints are usually less important than
time considerations.

• The reason for this can be, as for most sorting

programs, the amount of space needed is closer
to 0(n) than to 0(n2)

• The second reason is that, if more space is

required, it can almost always be found in
auxiliary storage.

10
Things to
remember
• An ideal sort is an in-place sort where the
algorithm uses no additional array storage, and
hence it is possible to sort very large lists without
the need to allocate additional working storage.

• A sorting algorithm is said to be stable if two

objects with equal keys appear in the same order
in sorted output as they appear in the input
unsorted array.

• Some sorting algorithms are stable by nature like

Insertion sort, Merge Sort, Bubble Sort, etc. And
some sorting algorithms are not, like Heap Sort,
Quick Sort, etc. 11
Things to
remember
• Sorting can be performed in many ways.

• Over a time several methods (or algorithms) are

being developed to sort data(s).
• Bubble sort, Selection sort, Quick sort, Merge sort,
Insertion sort are the few sorting techniques
discussed in this chapter.

• It is very difficult to select a sorting algorithm over

another. And there is no sorting algorithm better
than all others in all circumstances.

12
Inversions
A useful concept when analyzing sortingalgorithms is
an inversion: a pair of elements(t[a],t[b])in the array
such that a<b and t[a]>t[b], i.e., the elements are in
the wrong order. For example, in the array
1 2 3 4 5 6 7 8
1 2 2 6 3 5 8 6
For example, in the array the inversions are (6,3), (6,5)
and (6,8). The number of inversions tells us how much
work is needed to sort the array.
O(n2) algorithms Simple algorithms for sorting an
array work in O(n2) time.
 Such algorithms are short and usually consist of two
nested loops. A famous O(n2) time sorting
algorithm is bubble sort where the elements ”bubble”
in the array according to their values.
O(nlogn) algorithms
 It is possible to sort an array efficiently in O(nlogn) time using algorithms
that are not limited to swapping consecutive elements. One such algorithm
is merge sort1, which is based on recursion.
 Merge sort sorts the subarray t[a,b] as follows:
1. If a=b, do not do anything, because the subarray is already sorted.
2. Calculate the position of the middle element: k=b(a+b)/2c.
3. Recursively sort the subarray t[a,k].
4. Recursively sort the subarray t[k+1,b].
5. Merge the sorted subarrays t[a,k] and t[k+1,b] into a sorted subarray t[a,b].

Merge sort is an efficient algorithm, because it halves the size of the subarray at
each step. The recursion consists of O(logn) levels, and processing each level
takes O(n) time. Merging the subarrays t[a,k] and t[k+1,b] is possible in
linear time, because they are already sorted.
BUBBLE
SORT
• In bubble sort, each element is compared with its
adjacent element.

• We begin with the 0th element and compare it with the

1st element.

• If it is found to be greater than the 1st element, then

they are interchanged.

• In this way all the elements are compared

(excluding last) with their next element and are
interchanged if required

• On completing the first iteration, largest element gets 16

14
Algorit
hm

15
 TIME
COMPLEXITY

 The time complexity for bubble sort is calculated in

terms of the number of comparisons f(n) (or of
number of loops)

 Here two loops(outer loop and inner loop) iterates(or

repeated) the comparison.

 The inner loop is iterated one less than the number of

f in= the
elements (n – list
1) +(i.e.,
(n –n-1
2)times) and is reiterated upon
(n) + ......
every iteration of+the
2 +outer
1 loop
= n(n – 1) = O(n2).
16
TIME
COMPLEXITY
• Best Case
• sorting a sorted array by bubble sort algorithm
• In best case outer loop will terminate after one iteration,
i.e it involves performing one pass which requires n-1
comparison
f (n) = O(n2)
• Worst Case
• Suppose an array [5,4,3,2,1], we need to move first
element to end of an array
• n-1 times the swapping procedure is to be called
f (n) = O(n2)
• Average Case
• Difficult to analyse than the other cases
• Random inputs, so in general
f (n) = O(n2)
• Space Complexity
• O(n)
17
Counting sort
The lower bound nlogn does not apply to algorithms that
do not compare array elements but use some other
information.
An example of such an algorithm is counting sort that
sorts an array in O(n) time assuming that every element in
the array is an integer between 0...c and c=O(n).
The algorithm creates a bookkeeping array, whose indices
are elements in the original array. The algorithm iterates
through the original array and calculates how many times
each element appears in the array.
For example, the array
1 2 3 4 5 6 7 8
1 3 6 9 9 3 5 9
corresponds to the following bookkeeping array:
1 2 3 4 5 6 7 8 9
1 0 2 0 1 1 0 0 3
Construction of the bookkeeping array takes O(n) time.
After this, the sorted array can be created in O(n) time
because the number of occurrences of each element
can be retrieved from the bookkeeping array. Thus, the
total time complexity of counting sort is O(n).
 Heap
• specialized binary tree-based data structure
• Every heap data structure has the following properties...
• Property #1 (Ordering): Nodes must be arranged in an order according to their
values based on Max heap or Min heap.
• Property #2 (Structural): All levels in a heap must be full except the last level
and all nodes must be filled from left to right strictly. (Complete Binary Tree)

• Two types of heap data structures

– Max Heap – every parent node contains >= than its child nodes
– Min Heap - every parent node contains <= than its child nodes
 Heap Sort
 Build a max heap from the input data.
 The largest item is stored at the root of the
heap.
 Replace root with the last item of the heap
followed by reducing the size of heap by 1.
 Finally, heapify the root of tree.
 Repeat above steps while size of heap is
greater than 1
Example
void heapify(int arr[], int size, int i) void heapSort(int arr[], int size)
{ {
int largest = i; int i;
int left = 2*i + 1; for (i = size / 2 - 1; i >= 0; i--)
int right = 2*i + 2; heapify(arr, size, i);
for (i=size-1; i>=0; i--)
if (left < size && arr[left] >arr[largest]) {
largest = left; temp = arr[0];
arr[0]= arr[i];
if (right < size && arr[right] > arr[largest]) arr[i] = temp;
largest = right; heapify(arr, i, 0);
}
if (largest != i) }
{
temp = arr[i];
arr[i]= arr[largest];
arr[largest] = temp;
heapify(arr, size,
largest);
}
}
Quick
Sort
• Quick sort is one of the most popular sorting
techniques.

• As the name suggests the quick sort is the

fastest known sorting algorithm in practice.

• It has the best average time performance.

• It works by partitioning the array to be sorted

and each partition in turn sorted recursively.
Hence also called partition exchange sort.
45
 piv
wa curre ot
ll nt

6 5 1 3 8 4 7 9 2
curre piv
wa ot
ll nt

6 5 1 3 8 4 7 9 2
piv
wa curre
ot
ll nt

6 5 1 3 8 4 7 9 2
piv
curre
wa ot
nt
ll
1 5 6 3 8 4 7 9 2

piv
curre
wa ot
nt
ll
1 2 6 3 8 4 7 9 5

47
Algorithm
• Choosing a pivot
• To partition the list we first choose a pivot element

• Partitioning
• Then we partition the elements so that all those with
values less than pivot are placed on the left side and
the higher vale on the right
• Check if the current element is less than the pivot.
• If lesser replace it with the current element and move the wall
up one position
• else move the pivot element to current element and vice
versa
• Recur 4
8
• Repeat the same partitioning step unless all elements
Analysis of Quick Sort
• Best case
• The best case analysis assumes that the pivot is always
in the middle
• To simplify the math, we assume that the two sublists
are each exactly half the size of the original
T(N)=T(N/2)+T(N/2)….+1 leads to T(N)=O(nlogn)

• Average case
• T(N)=O(nlogn)

• Worst case
• When we pick minimum or maximum as pivot then we
have to go through
each and every element so
• T(N) = O(nAshim
2) Lamichhane 50
 Radix Sort

• First, sort elements based on the value of

the unit place.
• Then, sort elements based on the value of
the tenth place.
• This process goes on until the last
significant place
 Example
[121, 432, 564, 23, 1, 45, 788]
• Find the largest element in the array,
i.e. max.
• Let X be the number of digits in max.
– X is calculated because we have to go
through all the significant places of all
elements.
• In this array [121, 432, 564, 23, 1,
45,
788],
– we have the largest number 788.
– It has 3 digits.
– Therefore, the loop should go up to hundreds
place (3 times)
• Step 1 - Define 10 queues each representing a
bucket for each digit from 0 to 9.
• Step 2 - Consider the least significant digit of each
number in the list which is to be sorted.
• Step 3 - Insert each number into their respective
queue based on the least significant digit.
• Step 4 - Group all the numbers from queue 0 to
queue 9 in the order they have inserted into their
respective queues.
• Step 5 - Repeat from step 3 based on the next
least significant digit.
• Step 6 - Repeat from step 2 until all the numbers
are grouped based on the most significant digit.
 void countSort(int list[], int n, int exp) {
int getMax(int list[], int n) { int output[n];
int mx = list[0]; int i, count[10] = { 0 };
int i;
for (i = 1; i < n; i++) for (i = 0; i < n; i++)
if (list[i] > mx) count[(list[i] / exp) % 10]++;
mx = list[i];
return mx; for (i = 1; i < 10; i++)
} count[i] += count[i - 1];

for (i = n - 1; i >= 0; i--) {

output[count[(list[i] / exp) % 10] - 1] =
void radixsort(int list[], int n) { list[i];
int m = getMax(list, n); count[(list[i] / exp) % 10]--;
}
int exp;
for (exp = 1; m / exp > 0; exp *= 10) for (i = 0; i < n; i++)
countSort(list, n, exp); list[i] = output[i];
} }
Bucket Sorting
 Bucket Sort
 Bucket sort assumes that the input is generated by a random
process and drawn from a uniform distribution.
 In other words the elements are distributed uniformly and
independently over the interval [0,1].
 Bucket sort divides the interval [0,1] into n equal sized
subintervals or buckets. Then distributes the n inputs into
these buckets.
 After that the elements of each buckets are sorted using a
sorting algorithm generally using insertion or quick sort.
 Finally the buckets are concatenated together in
order.
 Consider that the input is an n-element array A and each
element A[i] in the array satisfies the 0<=A[i]<1
 Bucket Sort Algorithm
 Bucket-Sort(A)
1. Let B[0….n-1] be a new array
2. n = length[A]
3. for i = 0 to n-1
4. make B[i] an empty list
5. for i = 1 to n
6. do insert A[i] into list B[  n A[i]  ]
7. for i = 0 to n-1
8. do sort list B[i] with Insertion-Sort
9. Concatenate lists B[0], B[1],…,B[n-1] together in
order
 Bucket

1 2 3 4 5 6
A .74 .17 .26 .72 .39 .21
 Bucket: Loop 1
n=6
1 2 3 4 5 6
A .74 .17 .26 .72 .39 .21

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=1
1 2 3 4 5 6
A .74 .17 .26 .72 .39 .21
B[  n A[i]  ] = B[  6X.74  ]=B[  4.44  ]=B[4]

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=2
1 2 3 4 5
6
A .74 .17
.26 .72 .39 .21
B[  n A[i]  ] = B[  6X.17  ]=B[  1.02  ]=B[1]

.74

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=3
1 2 3 4 5
6
A .74 .17 .26
.72 .39 .21
B[  n A[i]  ] = B[  6X.26  ]=B[  1.56  ]=B[1]

.74
.17

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=4
1 2 3 4 5
6
A .74 .17 .26 .72
.39 .21
B[  n A[i]  ] = B[  6X.72  ]=B[  4.32  ]=B[4]

.26 .74
.17

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=5
1 2 3 5 6
4
A .39 .21
.74 .17 .26 .72
B[  n A[i]  ] = B[  6X.39  ]=B[  2.34  ]=B[2]

.26 .74
.17 .72

0 1 2 3 4 5
B
 Bucket: Loop 2
FOR n=6, i=6
1 2 3 4 5 6
A .74 .17 .26 .72 .39 .94

B[  n A[i]  ] = B[  6X.94  ]=B[  5.64  ]=B[5]

.26 .74
.17 .39
.72

0 1 2 3 4 5
B
 Bucket: End of Loop 2

1 2 3 4 5 6
A .74 .17 .26 .72 .39 .94

.26 .74
.39 .94
.17 .72

0 1 2 3 4 5
B
 Bucket: Loop 3
Apply insertion sort
on each bucket
1 2 3 4 5 6
A .74 .17 .26 .72 .39 .94

.17 .39 .72 .94

.26 .74

0 1 2 3 4 5
B
 Bucket
Concatenate the
buckets in order
1 2 3 4 5 6
A .74 .17 .26 .72 .39 .94

Sorted
0 1 2 3 4 5
B output

.17 .26 .39 .72 .74 .94

.17 .39 .72 .94
.26 .74

0 1 2 3 4 5
B
 Example - Bucket Sort
A 1 .78 B 0 /

2 .17 1 .17 .12 /

3 .39 2 .26 .21 .23 /

4 .26 3 .39 /

5 .72 4 / Distribute
6 .94 5 / Into buckets
7 .21 6 .68 /

8 .12 7 .78 .72 /

9 .23
8 /
10 .68 9 .94 /
 Example - Bucket Sort
0 /

1 .12 .17 /

2 .21 .23 .26 /

3 .39 /
Sort within each
4 / bucket
5 /

6 .68 /

7 .72 .78 /

8 /
9 .94 /
 Example - Bucket Sort

.12 .17 .21 .23 .26 .39 .68 .72 .78 .94 /
 Analysis of Bucket Sort
 Bucket-Sort(A)

Let B[0….n-1] be a new array

1.
n = length[A]
2.
for i = 0 to n-1
3. Step 5 and 6
4. make B[i] an empty list takes O(n)
time
for i = 1 to n
5.
Step 7 and 8
6. do insert A[i] into list B[ floor of n A[i] ] takes O(n
8. 7.do sort
for i = 0list B[i] with Insertion-Sort
to n-1 log(n/k) time

9. Concatenate lists B[0], B[1],…,B[n-1] Step 9 takes

together in order O(k) time

In total Bucket sort takes : O(n) (if k=Θ(n))

 Bucket Sort Review
• Assumption: input is uniformly distributed across a range
• Basic idea:
– Partition the range into a fixed number of buckets.
– Toss each element into its appropriate bucket.
–Sort each bucket.
• Pro’s:
– Fast
– Asymptotically
fast (i.e., O(n)
when
distribution is
• uniform)
– Simple to code
–Good for a rough sort.
analysis
 Application of sorting
• Searching
– Binary search enables you to test whether an item is in a
dictionary in time, once the keys are all sorted.

• Closest pair
– Given a set of n numbers
– Find the pair of numbers that have the smallest difference
between them
– After the numbers are sorted, the closest pair of numbers
will lie next to each other somewhere in sorted order.

• Element uniqueness
– Are there any duplicates in a given a set of n items?
– The most efficient algorithm is to sort them and then do a
linear scan though them checking all adjacent pairs.
Application contd..
• Selection
– What is the kth largest item in the set?
– If the keys are placed in sorted order in an array,
the kth largest can be found in constant time by
simply looking at the kth position of the array
• Government organizations, financial institutions,
and commercial enterprises
– Organize much of this information by sorting it
• Load-balancing problem
– M identical processors and N jobs to complete
– Goal is to schedule all of the jobs on the processors
so that the time when the last job completes as early
as possible.