Week 12 : Sorting

STIA2024
Data Structures & Algorithm
Analysis

1
 Chapter Contents
 Insertion Sort
 Introduction
 Insertion Sort of a Chain of Linked Nodes
 Java Code for Insertion Sort
 The Efficiency of Insertion Sort

 Selection Sort
 Introduction
 The Efficiency of Selection Sort

 Shell Sort
 Introduction
 The Efficiency of Shell Sort

2
 Chapter Contents
Merge Sort
- Introduction
- Recursive Merge Sort
- Java Code for Merge Sort
- The Efficiency of Merge Sort
Quick Sort
- Introduction
- Creating the Partition
- Java Code for Quick Sort
- The Efficiency of Quick Sort

3
 Chapter Contents
Heap Sort
- Reprise: The ADT Heap
- Using an Array to Represent a Heap
- Adding an Entry
- Removing the Root
- Creating a Heap
- Heap Sort
Comparing the Algorithms

4
 Learning Objective

 To sort an array into ascending order by using

the following methods: selection sort, insertion
sort, shell sort, merge sort, quick sort and heap
sort,
 To sort a chain of nodes into ascending order
by using an insertion sort, and
 To determine the efficiency of a sort and
comparing the algorithms of sorting.

5
 Introduction
 Sorting – rearranging data in an array or list
so that it is in ascending or descending
order.
 Sorting can be done to any collection of
items that can be compared with one
another.
 Eg: arrange a row of books on bookshelf by
title.
 The efficiency of a sorting algorithm is
significant, particularly when large amounts
of data are involved.
6
INSERTION SORT

7
 Insertion Sort
 If first two books are out of order
 Remove second book
 Slide first book to right
 Insert removed book into first slot

 Then look at third book, if it is out of

order
 Remove that book
 Slide 2nd book to right
 Insert removed book into 2nd slot

 Recheck first two books again

 Etc. 8
Insertion Sort

The placement of the

third book during an
insertion sort.

9
Insertion Sort

An insertion sort of books

10
 Insertion Sort
 Sort an array partitions – that is, divides the
array into two parts.
 One part is sorted and initially contains just first
element in the array
 The second part contains the remaining elements.
 The algorithm removes the first element from
the unsorted part and inserts it into its proper
sorted position within the sorted part.
 An insertion sort is more efficient when an
array is sorted.

11
public class Sort
{
public static void main(String [] args)
{
int [] tsnom = {3, 1, 2, 7, 5};
System.out.println(“Before Insertion Sort");
print(tsnom);
insertionSort(tsnom);
System.out.println(“After Insertion Sort");
print(tsnom);
}
public static void insertionSort (int[] ts)
{
for (int x=1; x < ts.length; x++) Java Code for
{
int temp=ts[x];
Insertion Sort
for (int y=x;y>0 && temp < ts[y-1];y--)
ts[y]=ts[y-1];
ts[y]=temp;
}
}
public static void print(int[] ts)
{
for(int x=0; x< ts.length; x++)
System.out.print(ts[x]);
System.out.println();
}
}

12
 Insertion Sort
int temp=ts[x];
3 1 2 7 5 int y=x;
[0] [1] [2] [3] [4] for (; y>0 && temp < ts[y-1];y--)
ts[y]=ts[y-1];
ts[y]=temp

x=1 x=3
y=1 1 2 3 7 5 y=3
3 1 2 7 5 y=0 [0] [1] [2] [3] [4]
[0] [1] [2] [3] [4]
X=1 1 2 3 7 5
temp=1 temp=7 [0] [1] [2] [3] [4]
1<3 1 3 2 7 5 X=3 7<3 
 [0] [1] [2] [3] [4]

1 2 3 7 5 x=4
y=4
X=2 [0] [1] [2] [3] [4]
1 3 2 7 5 y=2
y=3
[0] [1] [2] [3] [4] y=1

temp=5 1 2 3 5 7
1 3 3 7 5 5<7  [0] [1] [2] [3] [4]
X=2 X=4 5<3 
temp=2 [0] [1] [2] [3] [4]
2<3 
2<1 1 2 3 7 5
 1 2 3 5 7
[0] [1] [2] [3] [4]
[0] [1] [2] [3] [4] 13
 Insertion Sort
int temp=ts[x];
int y=x;
for (; y>0 && temp < ts[y-1];y--)
ts[y]=ts[y-1];
ts[y]=temp

3 1 2 7 5 1 2 3 7 5
[0] [1] [2] [3] [4] [0] [1] [2] [3] [4]

X=1 1 3 2 7 5
temp=1 temp=7
1<3 [0] [1] [2] [3] [4] x=3
temp=7
7<3

1 3 2 7 5
[0] [1] [2] [3] [4]
1 2 3 7 5
temp=2
[0] [1] [2] [3] [4]

x=2 1 2 3 7 5
temp=2 [0] [1] [2] [3] [4] temp=5
2<3 x=4
2<1  temp=5 1 2 3 5 7

5<7 [0] [1] [2] [3] [4]
5<3
14
Insertion Sort of Chain of Linked Nodes

A chain of integers sorted into ascending order

15
Insertion Sort of Chain of Linked Nodes

During the traversal of a chain to locate the

insertion point, save a reference to the node
before the current one.
16
Insertion Sort of Chain of Linked Nodes

Efficiency of
insertion sort of
a chain is O(n2)
Breaking a chain of nodes into two pieces as
the first step in an insertion sort:
(a) the original chain; (b) the two pieces 17
Efficiency of Insertion Sort

 Best time efficiency is O(n)

 Worst time efficiency is O(n2)
 Average time efficiency is O(n2)
 If array is closer to sorted order
 Less work the insertion sort does
 More efficient the sort is

 Insertion sort is acceptable for small

array sizes
18
SELECTION SORT

19
Selection Sort
 Rearranges the object by interchanging
value.
 The sort locates the smallest/largest value in
the array.
 Task: rearrange books on shelf by height
 Shortest book on the left
 Approach:
 Look at books, select shortest book
 Swap with first book
 Look at remaining books, select shortest
 Swap with second book
 Repeat … 20
Selection Sort

Before and after exchanging shortest book and

the first book.
21
Selection Sort

A selection sort of an array

of integers into ascending
order.
(method 1:choose the smallest value)

22
Before Sort L1 L2 L3 L4 Sorted List

12 12 12 12 12 12

19 19 19 19 19 19

33 33 22 22 22 22

26 26 26 26 26 26

22 22 33 33 33 33

Max=33 Max=26 Max=22 Max=19

A selection sort of an array of

integers into ascending order.
(method 2: choose the biggest value) 23
 Selection Sort
public void selectionSort(Object array[]) {
int minIndex;
Object temp;

for(int x = 0; x < array.length -1 ; x++) {

minIndex = x; //initialization the index of smallest value
for(int y = x + 1; y < array.length; y++){
//find the index of the smallest value in the list
if(array[y] < array[minIndex])
minIndex = y;
}
//move the smallest value into correct position (swap)
temp = array[x];
array[x] = array[minIndex];
array[minIndex] = temp; }
}//end selectionSort
24
The Efficiency of Selection Sort

 Iterative method for loop executes n – 1

times
 For each of n – 1 calls, inner loop executes
n – 2 times
 (n – 1) + (n – 2) + …+ 1 = n(n – 1)/2 = O(n2)

 Recursive selection sort performs same

operations
 Also O(n2)

25
Efficiency of Selection Sort

 Best time efficiency is O(n2)

 Worst time efficiency is O(n2)
 Average time efficiency is O(n2)
 Sufficient when to sort a small array.

26
SHELL SORT

27
Shell Sort

 A variation of the insertion sort

 But faster than O(n2)
 Done by sorting subarrays of equally
spaced indices
 Instead of moving to an adjacent location
an element moves several locations
away
 Results in an almost sorted array
 This array sorted efficiently with ordinary
insertion sort 28
Shell Sort

List (Before Sort) 81 94 11 96 12 35 17 95 28 58 41 75 15

Step 1:
Divide list – distance of 5 horizontal
81 94 11 96 12

35 17 95 28 58

41 75 15
Sort using insertion sort vertical

35 17 11 28 12

41 75 15 96 58

81 94 95

Combine the list horizontal

35 17 11 28 12 41 75 15 96 58 81 94 95
29
 Step 2:
Shell Sort Divide list – distance of 3

35 17 11

28 12 41

75 15 96

58 81 94

95

Sort using insertion sort

28 12 11

35 15 41

58 17 94

75 81 96

95

Combine the list

28 12 11 35 15 41 58 17 94 75 81 96 95
30
 Shell Sort

Step 3:
Divide list – distance of 1
28 12 11 35 15 41 58 17 94 75 81 96 95

Sort using insertion sort

11 12 15 17 28 35 41 58 75 81 94 95 96

•Combine the list – sorted list

11 12 15 17 28 35 41 58 75 81 94 95 96

31
Efficiency of Shell Sort

 Best time efficiency is O(n)

 Worst time efficiency is O(n1.5)
 Average time efficiency is O(n1.5)

32
MERGE SORT

33
 Merge Sort

 Divide an array into halves

 Sort the two halves
 Merge them into one sorted array

 Referred to as a divide and conquer

algorithm
 This is often part of a recursive algorithm
 However recursion is not a requirement

34
 Merge Sort
// Sorts the array elements a[first] through a[last] recursively.
Algorithm mergeSort(a, first, last)
if (first < last)
{
mid = (first + last)/2
mergeSort(a, first, mid)
mergeSort(a, mid+1, last)
Merge the sorted halves a[first..mid] and a[mid+1..last]
}

35
 Merge Sort

The effect of the recursive calls and the merges

36
during a merge sort.
 Merge Sort

 Efficiency of the merge sort

 Merge sort is O(n log n) in all cases
 It's need for a temporary array is a
disadvantage
 Merge sort in the Java Class Library
 The class Arrays has sort routines that
uses the merge sort for arrays of objects
public static void sort(Object[] a);
public static void sort(Object[] a, int first, int last);

37
public class MergeSort
{
public static void main(String [] args)
{
int [] tsnom = {34,8,64,51,32,21,30};
System.out.println(“Before Merge Sort");
print(tsnom);
mergeSorting(tsnom);
System.out.println(“After Merge Sort");
print(tsnom);
}
private static void mergeSorting(int []a, int[]tsSem, int left, int right)
{
if (left < right)
{
int mid = (left + right)/2;
mergeSorting(a,tsSem,left,mid); // left side
mergeSorting(a,tsSem,mid + 1, right); // right side
merge(a,tsSem,left,mid + 1, right);
}
}
public static void mergeSorting(int []a)
{ Java Code for Merge
int [] tsSem=new int[a.length]; Sort …… continued
mergeSorting(a,tsSem,0,a.length-1);
} next slide

38
 private static void merge(int [] a, int [] tsSem, int posleft,
int posright, int lastright)
{
int lastleft=posright - 1;
int possem=posleft;
int bilelemen=lastright - posleft + 1;

while(posleft <= lastleft && posright <= lastright)

{
if (a[posleft]<=a[posright])
tsSem[possem++]=a[posleft++];
else
tsSem[possem++]=a[posright++];
}
while(posleft <=lastleft)
tsSem[possem++]=a[posleft++];
while(posright<=lastright)
tsSem[possem++]=a[posright++];
for(int i=0;i<bilelemen;i++,lastright--)
a[lastright]=tsSem[lastright];
}//merge

public static void print(int[]a) Java Code for

{ Merge Sort
for (int i=0;i<a.length;i++)
System.out.print("\t"+ a[i]);
}//print

} // class MergeSort
39
 Merge Sort
 Merge Sort has two level:-
a) Divide the list into halves

L=0
R=6 2 5 1 9 3 8 7
mid=(0+6)/2 [0] [1] [2] [3] [4] [5] [6]
=3 L=4
R=6
L=0 mid=(4+6)/2
R=3 =5
mid=(0+3)/2
2 5 1 9 3 8 7
[0] [1] [2] [3] L=2 L=4 [4] [5] [6]
=1
R=3 R=5
mid=(2+3)/2 mid=(4+5)/2
L=0 =2 =4
R=1
3 8 7
mid=(0+1)/2 2 5 1 9 [6]
=0 [0] [1] [2] [3] [4] [5]
L=6
R=6
8
5 1 9 3 [5]
2 [4]
[0] [1] [2] [3]
L=5
L=4
L=0 L=1 L=2 L=3
R=4
R=5 First example
R=1 R=2 R=3
R=0 40
 Merge Sort

2) Merge/combine the list from bottom

1 2 3 5 7 8 9
[0] [1] [2] [3] [4] [5] [6]

1 2 5 9 3 8 7
[0] [1] [2] [3] [4] [5] [6]

3 8 7
2 5 1 9 [6]
[0] [1] [2] [3] [4] [5]

8
5 1 9 3 [5]
2 [4]
[0] [1] [2] [3]
First example
41
 Merge Sort

L=0
R=6 34 8 64 51 32 21 30
mid=(0+6)/2 [0] [1] [2] [3] [4] [5] [6]
=3
L=4
R=6
L=0 mid =(4+6)/2
R=3 =5
mid =(0+3)/2
34 8 64 51 32 21 30
[0] [1] [2] [3] Ki=4 [4] [5] [6]
=1
Ki=2 Ka=5
Ka=3 Tgh=(4+5)/2
L=0 Tgh=(2+3)/2 =4
R=1 =2 32 21 30
mid =(0+1)/2
34 8 64 51 [6]
[0] [1] [2] [3] [4] [5]
=0
L=6
R=6
21
L=0 8 64 51 32 [5]
R=0 34 [4]
[0] [1] [2] [3]
L=5 Second
L=4
L=1 L=2 L=3 R=5
R=1 R=2 R=3 R=4 example42
 Merge Sort

L=0
R=6
mid=(0+6)/2
8 21 30 32 34 51 64
=3 [0] [1] [2] [3] [4] [5] [6]
L=4
R=6
L=0 mid=(4+6)/2
R=3 =5
mid=(0+3)/2
8 34 51 64 21 30 32
[0] [1] [2] [3] Ki=4 [4] [5] [6]
=1 Ki=2
Ka=5
Ka=3
Tgh=(2+3)/2 Tgh=(4+5)/2
L=0 =4
=2
R=1 21 32 30
mid=(0+1)/2
8 34 51 64 [6]
[0] [1] [2] [3] [4] [5]
=0
L=6
R=6
21
L=0 8 64 51 32 [5]
R=0 34 [4]
[0] [1] [2] [3] Second
L=5
L=4 example
L=1 L=2 L=3 R=5
R=4 43
R=1 R=2 R=3
 Efficiency of Merge Sort

 Best time efficiency is O(n log n)

 Worst time efficiency is O(n log n)
 Average time efficiency is O(n log n)

44
QUICK SORT

45
 Quick Sort

 Divides the array into two pieces

 Not necessarily halves of the array
 An element of the array is selected as the
pivot
 Referred to as a divide and conquer
algorithm
 Elements are rearranged so that:
 Elements in positions before pivot are less
than the pivot
 Elements after the pivot are greater than the
46
pivot
 Quick Sort

A partition of an array during a quick sort

47
 Quick Sort

// Sorts the array elements a[first] through a[last] recursively.

Algorithm quickSort(a, first, last)

if (first < last)
{ Choose a pivot
Partition the array about the pivot
pivotIndex = index of pivot
quickSort(a, first, pivotIndex-1) // sort Smaller
quickSort(a, pivotIndex+1, last) // sort Larger
}

48
 Quick Sort

 Quick sort is O(n log n) in the

average case and best case
 O(n2) in the worst case
 Worst case can be avoided by
careful choice of the pivot

49
 Quick Sort

 Quick sort rearranges the elements in an

array during partitioning process
 After each step in the process
 One element (the pivot) is placed in its correct
sorted position
 The elements in each of the two sub arrays
 Remain in their respective subarrays
 The class Arrays in the Java Class Library
uses quick sort for arrays of primitive types
50
 Quick Sort
Quick Sort
 Quick Sort has two level:-
a) Divide the list into halves (the first number is a pivot)
Quick Sort
5 2 1 9 3 8 7

2 1 3 9 8 7

1 3 8 7

51
 Quick Sort
Quick Sort
b) Merge/combine the list from bottom

1 2 3 5 7 8 9

1 2 3 7 8 9

1 3 7 8

52
public class QuickSort
{
public static void main(String [] args)
{
int [] tsnom = {33,17,6,21,56,29};

System.out.println(“Before Quick Sort");

print(tsnom);
quickSorting(tsnom,0,tsnom.length-1);
System.out.println(“After Quick Sort");
print(tsnom);
}
private static void quickSorting(int []a, int first, int last)
{
int indeksPivot;

if (first < last)

{
indeksPivot=choosePivot(a,first,last);
quickSorting(a,first,indeksPivot-1);
quickSorting(a,indeksPivot+1,last); Java Code for Quick
}//if
}//quickSorting
Sort….. Continued
next slide

53
 //choosePivot method for chosing the first number and sort

private static int choosePivot(int []a,int first, int last)

{
int p=first;
int pivot=a[first];

//for sort, left is smaller then pivot and right for greater then pivot
for (int i=first + 1; i <=last; i++)
{
if (a[i] < pivot)
{
a[p]=a[i];
a[i]=a[p+1];
a[p+1]=pivot;
p++;
}
}
return p;
}//choosePivot
Java Code for
public static void print(int[]a) Quick Sort
{
for (int i=0;i<a.length;i++)
System.out.print("\t"+ a[i]);
System.out.println("\n");
}//print
}

54
 Quick Sort
5 6 7 8 9
List of Number

Pivot = 5 and the value of pivot

is smaller than other numbers. pivot unknown
5 6 7 8 9

pivot unknown
5 6 7 8 9

pivot unknown
5 6 7 8 9

pivot unknown
5 6 7 8 9

pivot
After first partition 5 6 7 8 9

4 comparisons and 0 changing number Worst case for quick sort55

 Quick Sort
Quick Sort

5 6 7 8 9
[0] [1] [2] [3] [4]
<5 >5

6 7 8 9
<6 >6

7 8 9
<7 >7

8 9
<8 >8

56
 Efficiency of Quick Sort

 Best time efficiency is O(n log n)

 Worst time efficiency is O(n2)
 Average time efficiency is O(n log n)

57
HEAP SORT

58
 Reprise: The ADT Heap

 A complete binary tree

 Nodes contain Comparable objects
 In a maxheap
 Object in each node ≥ objects in
descendants
 Note contrast of uses of the word "heap"
 The ADT heap
 The heap of the operating system from
which memory is allocated when new
executes 59
 Reprise: The ADT Heap

 Interface used for implementation of

maxheap
public interface MaxHeapInterface
{ public void add(Comparable newEntry);
public Comparable removeMax();
public Comparable getMax();
public boolean isEmpty();
public int getSize();
public void clear();
} // end MaxHeapInterface

60
Using an Array to Represent a Heap

Figure-1: (a) A complete binary tree with its nodes

numbered in level order; (b) its representation as an array.
61
Using an Array to Represent a Heap

 When a binary tree is complete

 Can use level-order traversal to store data
in consecutive locations of an array
 Enables easy location of the data in a
node's parent or children
 Parent of a node at i is found at i/2
(unless i is 1)
 Children of node at i found at indices
2i and 2i + 1
62
 Adding an Entry

Figure-2: The steps in adding 85 to the

maxheap of Figure-1(a)
63
 Adding an Entry

 Begin at next available position for a leaf

 Follow path from this leaf toward root
until find correct position for new entry
 As this is done
 Move entries from parent to child
 Makes room for new entry

64
 Adding an Entry

Figure-3: A revision of steps shown in

Figure-2 to avoid swaps. 65
 Adding an Entry

Figure-4: An array representation of the

steps in Figure-3 … continued → 66
 Adding an Entry

An array representation of the steps in

67
Figure-3.
 Adding an Entry

 Algorithm for adding new entry to a heap

Algorithm add(newEntry)
if (the array heap is full)
Double the size of the array
newIndex = index of next available array location
parentIndex = newIndex/2 // index of parent of available
location
while (newEntry > heap[parentIndex])
{ heap[newIndex] = heap[parentIndex]
// move parent to available location
// update indices
newIndex = parentIndex
parentIndex = newIndex/2
} // end while
68
Removing the Root

Figure-5: The steps to remove the entry in the root of the

maxheap of Figure-3(d) 69
 Removing the Root

Figure-6: The steps that transform a semiheap into

70
a heap without swaps.
 Removing the Root

 To remove a heap's root

 Replace the root with heap's last child
 This forms a semiheap
 Then use the method reheap
 Transforms the semiheap to a heap

71
 Creating a Heap

Figure-7: The steps in adding 20, 40, 30,

72
10, 90, and 70 to a heap.
 Creating a Heap

More efficient to use

reheap than to
use add

Figure-8: The steps in creating a heap by

using reheap. 73
 Heapsort

 Possible to use a heap to sort an array

 Place array items into a maxheap
 Then remove them
 Items will be in descending order
 Place them back into the original array and
they will be in order

74
 Heapsort

Figure-9: A trace of heapsort (a – c)

75
 Heapsort

Figure-9: A trace of heapsort (d – f)

76
 Heapsort

Figure-9: A trace of heapsort (g – i)

77
 Heapsort

Figure-9: A trace of heapsort (j – l)

78
 Heapsort

 Implementation of heapsort
public static void heapSort(Comparable[] array, int n)
{ // create first heap
for (int index = n/2; index >= 0; index--)
reheap(array, index, n-1);
swap(array, 0, n-1);
for (int last = n-2; last > 0; last--) Efficiency is O(n log n).
{ reheap(array, 0, last); However, quicksort is
swap(array, 0, last); usually a better choice.
} // end for
} // end heapSort

private static void reheap(Comparable[] heap, int first, int last)

{ < Statements from Segment 27.11, replacing rootIndex with first and
lastIndex with last. >
} // end reheap
79
 Heapsort
 Creating a heap

1 5

5 2 1 9 3 8 7 2 2 3 1

4 6 7
9 5 3 8 7

reheap(2)

1 5

2 9 3 1
5 9 1 2 3 8 7
4 6 7
2 5 3 8 7

reheap(3) 80
 Heap Sort
• Creating a heap

1 5

2 9 3 8
5 9 8 2 3 1 7
4 6 7
2 5 3 1 7

reheap(1)

1 9

2 5 3 8
9 5 8 2 3 1 7
4 6 7
2 5 3 1 7 81
 Heap Sort
• Creating a heap

1 9

2 5 3 8
9 5 8 2 3 1 7
4 6 7
2 5 3 1 7

remove

1 7

2 5 3 8
7 5 8 2 3 1 9
4 6
2 5 3 1

82
 Heap Sort

• Creating a heap

1 7

2 5 3 8
7 5 8 2 3 1 9
4 6
2 5 3 1
reheap(1)

1 8

2 5 3 7
8 5 7 2 3 1 9
4 6
2 5 3 1
remove
83
 Heap Sort

 Creating a heap

1 1

2 5 3 7
1 5 7 2 3 8 9
4
2 5 3

reheap(1)

84
 Heap Sort

• Creating a heap

1 7

2 5 3 1
7 5 1 2 3 8 9
4
2 5 3

remove

1 3

2 5 3 1
3 5 1 2 7 8 9
4
2
reheap(1) 85
 Heap Sort

• Creating a heap

1 5

2 3 3 1
5 3 1 2 7 8 9
4
2
remove

1 2

2 3 3 1
2 3 1 5 7 8 9

reheap(1)
86
 Heap Sort

• Creating a heap

1 3

2 2 3 1
3 2 1 5 7 8 9

remove

1 1

2 2
1 2 3 5 7 8 9

reheap(1)

87
 Heap Sort

• Creating a heap

1
2

2 1
2 1 3 5 7 8 9

remove

1
1

1 2 3 5 7 8 9
remove

1 2 3 5 7 8 9
88
Efficiency of Heap Sort

 Best time efficiency is O(n log n)

 Worst time efficiency is O(n log n)
 Average time efficiency is O(n log n)

89
Comparing the Algorithms
Best Average Worst
Case Case Case

Insertion sort O(n) O(n2) O(n2)

Selection sort O(n2) O(n2) O(n2)
Shell sort O(n) O(n1.5) O(n1.5)
Merge sort O(n log n) O(n log n) O(n log n)
Quick sort O(n log n) O(n log n) O(n2)
Heap sort O(n log n) O(n log n) O(n log n)

The time efficiencies of three sorting algorithms,

expressed in Big Oh notation. 90
References
 Data Structures and Abstractions with Java . Authors: Frank
M. Carrano & Walter Savitch . Chapter 11, 12 and 27.

 Data Structures with Java . Authors : Hubbard J.R. & Huray

A. . Chapter 15

91