Advanced Sorting Algorithms

Advanced Algorithms:
Sorting
Damian Gordon

Advanced Algorithms
• Sorting Algorithms
– Insertion Sort
– Shell Sort
– Merge Sort
– Quick Sort

PROGRAM BubbleSort:
Integer Age[8] <- {44,23,42,33,16,54,34,18};
FOR Outer-Index IN 0 TO N-1
DO FOR Index IN 0 TO N-2
DO IF (Age[Index+1] < Age[Index])
THEN Temp_Value <- Age[Index+1];
Age[Index+1] <- Age[Index];
Age[Index] <- Temp_Value;
ENDIF;
ENDFOR;
ENDFOR;
END.
Sorting: Bubble Sort

PROGRAM SelectionSort:
Integer Age[8] <- {44,23,42,33,16,54,34,18};
FOR Outer-Index IN 0 TO N-1
MinValueLocation <- Outer-Index;
FOR Index IN Outer-Index+1 TO N-1
DO IF (Age[Index] < Age[MinValueLocation])
THEN MinValueLocation <- Index;
ENDIF;
ENDFOR;
IF (MinValueLocation != Outer-Index)
THEN Swap(Age[Outer-Index], Age[MinValueLocation]);
ENDIF;
ENDFOR;
END.
Sorting: Selection Sort

Insertion Sort
• Insertion Sort works by taking the first two elements of the list,
sorting them, then taking the third one, and sorting that into
the first two, then taking the fourth element and sorting that
into the first three, etc.

Insertion Sort
• Let’s look at an example:

Insertion Sort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7

Insertion Sort
23 44 42 33 16 54 34 18
0 1 2 3 4 5 6 7

Insertion Sort
23 42 44 33 16 54 34 18
0 1 2 3 4 5 6 7

Insertion Sort
23 33 42 44 16 54 34 18
0 1 2 3 4 5 6 7

Insertion Sort
16 23 33 42 44 54 34 18
0 1 2 3 4 5 6 7

Insertion Sort
16 23 33 34 42 44 54 18
0 1 2 3 4 5 6 7

Insertion Sort
16 18 23 33 34 42 44 54
0 1 2 3 4 5 6 7

Insertion Sort
• How do we move the elements into their correct position
(we’ll call this the “Insertion Sort move”)?

Insertion Sort
• We store 34 in CURRENT.
16 23 33 42 44 54 34 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• Next we move what value is in the previous position to 34 into
that location.
16 23 33 42 44 54 34 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• Next we move what value is in the previous position to 34 into
that location.
16 23 33 42 44 54 54 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• And we do that again.
16 23 33 42 44 44 54 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• And again.
16 23 33 42 42 44 54 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• And then we move CURRENT into the correct position.
16 23 33 42 42 44 54 18
0 1 2 3 4 5 6 7
Current: 34

Insertion Sort
• And then we move CURRENT into the correct position.
16 23 33 34 42 44 54 18
0 1 2 3 4 5 6 7

Insertion Sort
• The element being added in each time is just to the left of the
sorted list.
• So, if the next element is called CURRENT, the largest element
in the sorted list is CURRENT – 1.
• So we’ll know if the next element is largest if it is bigger than
CURRENT – 1.
Sorted sub-list
Next
element
CURRENT

Insertion Sort
• Structured English:
FOR each element from the second TO the end of the list
DO Remember the current position and value
WHILE the previous element is bigger than current
DO Move the previous element into current’s position
END WHILE
We’ve reached the position in the list that current should be
Put it in
END FOR
NOTE: The Previous
Element is the end of
the sorted sub-list

PROGRAM InsertionSort:
Integer Array[8] <- {44,23,42,33,16,54,34,18};
FOR Index IN 1 TO N
DO current = Array[index];
pos = index;
WHILE (pos > 0 and Array[pos – 1] > current)
DO Array[pos] <- Array[pos - 1];
pos = pos - 1;
ENDWHILE;
Array[pos] = current;
ENDFOR;
END.
Insertion Sort
NOTE: If you have
reached the start of
the list, STOP!

Insertion Sort
• Complexity of Insertion Sort
– Best-case scenario complexity = O(N)
– Average complexity = O(N2)
– Worst-case scenario complexity = O(N2)

Shell Sort
• ShellSort is an extension of InsertionSort that was developed
by Donald Shell.
• Shell observed that the process of doing the Insertion Sort
move, particularly if that have to be moved from one side of
the array to other, is computationally expensive.

Shell Sort
• Instead of sorting the whole list, ShellSort first picks ever Nth
element, sorts those elements (0, N, 2N, 3N,…), then sorts (1,
N+1, 2N+1, 3N+1,…), then (2, N+2, 2N+2, 3N+2,…), and so on.
• Once that’s done, let’s sort every Mth element (where M is N
DIV 2), so we’ll sort (0, M, 2M, 3M,…), then sorts (1, M+1,
2M+1, 3M+1,…), then (2, M+2, 2M+2, 3M+2,…), and so on.
• And keep doing this until we get to 1.

Donald L. Shell
• Born: March 1, 1924
• Died: November 2, 2015
• Shell, D.L. (1959) "A High-
Speed Sorting Procedure“,
Communications of the ACM,
2(7), pp. 30–32.

Shell Sort
• So, for example. Age = [44, 23, 42, 33, 18, 54, 34, 16].

Shell Sort
• Let’s pick every 4th element:

Shell Sort
– First group: 44, 18

Shell Sort
– Second group: 23, 54

Shell Sort
– Third group: 42, 34

Shell Sort
– Fourth group: 33, 16

Shell Sort
– Fourth group: 33, 16
Sort these using
Insertion Sort

Shell Sort
– First group: 44, 18 18, 44
– Second group: 23, 54 23, 54
– Third group: 42, 34 34, 42
– Fourth group: 33, 16 16, 33
Sort these using
Insertion Sort

Shell Sort
– First group: 44, 18 18, 44
– Second group: 23, 54 23, 54
– Third group: 42, 34 34, 42
– Fourth group: 33, 16 16, 33
Sort these using
Insertion Sort
The data is not sorted, but a lot of the big numbers have been moved to the end of
the list, and a lot of the smaller numbers have been moved to the start.

Shell Sort
• Now let’s do every 2nd element:

Shell Sort
– First Group: 18, 34, 44, 42

Shell Sort
– First Group: 18, 34, 44, 42
– Second Group: 23, 16, 54, 33

Shell Sort
– First Group: 18, 34, 44, 42
– Second Group: 23, 16, 54, 33
Sort these using
Insertion Sort

Shell Sort
– First Group: 18, 34, 44, 42 18, 34, 42, 44
– Second Group: 23, 16, 54, 33 16, 23, 33, 54
Sort these using
Insertion Sort

Shell Sort
– First Group: 18, 34, 44, 42 18, 34, 42, 44
– Second Group: 23, 16, 54, 33 16, 23, 33, 54
Sort these using
Insertion Sort
The data is almost completely sorted now.

Shell Sort
• Finally do one more Insertion Sort with all elements

Shell Sort
• This will be very fast since the data is almost sorted

Shell Sort
• This will be very fast since the data is almost sorted
• Age = [16, 18, 23, 33, 34, 42, 44, 54].

Shell Sort
• Let’s look at the PseudoCode in two parts:
– 1) The Insertion Sort code as before, but modified not to sort all of
the elements, but rather every Nth element
– 2) The Shell Sort that calls the Insertion Sort code for each Nth
elements, and then N/2, and N/4, and so on until 1.

MODULE GapInsertionSort(Array, StartPos, Gap):
FOR Index IN StartPos TO N INCREMENT BY Gap
DO current = Array[index];
pos = index;
WHILE (pos > Gap and Array[pos – Gap] > current)
DO Array[pos] <- Array[pos - Gap];
pos = pos - Gap;
ENDWHILE;
Array[pos] = current;
ENDFOR;
END.
Shell Sort

MODULE ShellSort(Array):
GapSize <- Length(Array) DIV 2;
WHILE (GapSize > 0)
DO
FOR StartPos IN 0 TO GapSize
DO GapInsertionSort(Array, StartPos, GapSize);
ENDFOR;
GapSize <- GapSize DIV 2;
ENDWHILE;
END.
Shell Sort
We are reducing the
Gap in half each time
For each of the Nth
Element, each N+1th
Element, N+2th, etc.
The main loop will
keep going until the
Gap is 1.

Shell Sort
• Complexity of Shell Sort
– Average complexity = O(N * log2(N))
– Worst-case scenario complexity = O(N * log2(N))

Merge Sort
• Merge Sort was developed by John von Neumann in 1945.

John von Neumann
• Born: December 28, 1903
• Died: February 8, 1957
• Born in Budapest, Austria-Hungary
• A mathematician who made major
contributions to set theory,
functional analysis, quantum
mechanics, ergodic theory,
continuous geometry, economics
and game theory, computer
science, numerical analysis,
hydrodynamics and statistics.

Merge Sort
• Merge Sort used a “divide-and-conquer” strategy.
• It’s a two-step process:
– 1. Keep splitting the array in half until you end up with sub-arrays of
one item (which are sorted by definition).
– 2. Successively merge each sub-array together, and sort with each
merge.

Merge Sort

Merge Sort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7

Merge Sort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7

Merge Sort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7

Merge Sort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1
42 33
2 3

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1
42 33
2 3
33 42
2 3

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1
42 33
2 3
33 42
2 3
16 54
4 5
16 54
4 5

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1
42 33
2 3
33 42
2 3
16 54
4 5
16 54
4 5
34 18
6 7

Merge Sort
44 23 42 33
0 1 2 3
16 54 34 18
4 5 6 7
44 23
0 1
23 44
0 1
42 33
2 3
33 42
2 3
16 54
4 5
16 54
4 5
34 18
6 7
18 34
6 7

Merge Sort
23 44
0 1
33 42
2 3
16 54
4 5
18 34
6 7

Merge Sort
23 44
0 1
33 42
2 3
16 54
4 5
18 34
6 7
23 44
0 1
33 42
2 3

Merge Sort
23 44
0 1
33 42
2 3
16 54
4 5
18 34
6 7
23 44
0 1
33 42
2 3
23 33
0 1
42 44
2 3

Merge Sort
23 44
0 1
33 42
2 3
16 54
4 5
18 34
6 7
23 44
0 1
33 42
2 3
23 33
0 1
42 44
2 3
16 54
4 5
18 34
6 7

Merge Sort
23 44
0 1
33 42
2 3
16 54
4 5
18 34
6 7
23 44
0 1
33 42
2 3
23 33
0 1
42 44
2 3
16 54
4 5
18 34
6 7
16 18
4 5
34 54
6 7

Merge Sort
23 33
0 1
42 44
2 3
16 18
4 5
34 54
6 7

Merge Sort
23 33
0 1
42 44
2 3
16 18
4 5
34 54
6 7
23 33
0 1
42 44
2 3
16 18
4 5
34 54
6 7

Merge Sort
23 33
0 1
42 44
2 3
16 18
4 5
34 54
6 7
23 33
0 1
42 44
2 3
16 18
4 5
34 54
6 7
16 18
0 1
23 33
2 3
34 42
4 5
44 54
6 7

PROGRAM MainMergeSort:
Array = [44,23,42,33,16,54,34,18];
MergeSort(Array);
PRINT Array;
END.
Merge Sort

PROGRAM MergeSort(Array):
IF (length(Array) > 1)
THEN MidPoint = len(Age)//2
LeftHalf = Age[:MidPoint]
RightHalf = Age[MidPoint:]
Merge Sort
Keep recursively
splitting the array
until you get down
sub-arrays of one
element.
Continued 

MergeSort(LeftHalf)
MergeSort(RightHalf)
LCounter = 0
RCounter = 0
MainCounter = 0
Merge Sort
Recursively call
MergeSort for each half
of the array. After the
splitting gets down to
one element the
recursive calls will pop
off the stack to merge
the sub-arrays together.
Continued 
 Continued

WHILE (LCounter < len(LeftHalf) AND
RCounter < len(RightHalf))
DO IF LeftHalf[LCounter] < RightHalf[RCounter]
THEN Age[MainCounter] = LeftHalf[LCounter];
LCounter = LCounter + 1;
ELSE Age[MainCounter] = RightHalf[RCounter];
RCounter = RCounter + 1;
ENDIF;
MainCounter = MainCounter + 1;
ENDWHILE;
Merge Sort
Continued 
 Continued
Keep comparing each
element of the left and
right sub-array, writing
the smaller element
into the main array

WHILE LCounter < len(LeftHalf)
DO Age[MainCounter] = LeftHalf[LCounter];
LCounter = LCounter + 1;
MainCounter = MainCounter + 1;
ENDWHILE;
WHILE Rcounter < len(RightHalf)
DO Age[MainCounter] = RightHalf[RCounter]
RCounter = RCounter + 1
MainCounter = MainCounter + 1
ENDWHILE;
ENDIF;
Merge Sort
 Continued
After the comparisons
are done, write either
the rest of the left array
or the right array into
the main array that

Merge Sort
• Complexity of Merge Sort
– Worst-case scenario complexity = O(N * log2(N))

QuickSort
• Quicksort was developed by Tony Hoare in 1959 and is still a
commonly used algorithm for sorting.

Charles Antony Richard Hoare
• Born: January 11, 1934
• Hoare's most significant work
includes: his sorting and selection
algorithm (Quicksort and
Quickselect), Hoare logic, the
formal language Communicating
Sequential Processes (CSP) used to
specify the interactions between
concurrent processes, structuring
computer operating systems using
the monitor concept.

QuickSort
• The key idea behind Quicksort is to pick a random value from
the array, and starting from either side of the array, swap
elements that are lower than the value in the right of the array
with elements of the left of the array that are larger than the
value, until we reach the point where the random value should
be, then we put the random value in its place.
• We recursively do this process with the sub-arrays on either
side of the random value ( called the “pivot”).

QuickSort

QuickSort
44 23 42 33 16 54 34 18
0 1 2 3 4 5 6 7

QuickSort
44 23 42 33 16 18 34 54
0 1 2 3 4 5 6 7

QuickSort
34 23 42 33 16 18 44 54
0 1 2 3 4 5 6 7

QuickSort
34 23 42 33 16 18 44 54
0 1 2 3 4 5 6 7
Pivot value is
now in its correct
position.

QuickSort
34 23 42 33 16 18 44 54
0 1 2 3 4 5 6 7
Now repeat this
process with this
sub-array

QuickSort
34 23 42 33 16 18 44 54
0 1 2 3 4 5 6 7
Now repeat this
process with this
sub-array
…and this one

QuickSort
34 23 42 33 16 18 44 54
0 1 2 3 4 5 6 7
THIS IS ALREADY
SORTED!!!

QuickSort
34 23 42 33 16 18
0 1 2 3 4 5
44 54
6 7

QuickSort
34 23 18 33 16 42
0 1 2 3 4 5
44 54
6 7

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
44 54
6 7

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
Now repeat this
process with this
sub-array
44 54
6 7

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
Now repeat this
process with this
sub-array
…and this one
44 54
6 7

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
Now repeat this
process with this
sub-array THIS IS ALREADY
SORTED
44 54
6 7

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
44 54
6 7
No swaps will occur on this
pass, as 16 is in the right
place

QuickSort
16 23 18 33 34 42
0 1 2 3 4 5
44 54
6 7
…but as 16 is swapped into
itself, the rest of the sub-
array will be sorted.

QuickSort
16 18 23 33 34 42
0 1 2 3 4 5
44 54
6 7

QuickSort
16 18 23 33 34 42
0 1 2 3 4 5
44 54
6 7
SORTED!

PROGRAM QuickSort(Array, First, Last):
IF (First < Last)
THEN Pivot = Partition(Array, First, Last);
QuickSort(Array, First, Pivot - 1):
QuickSort(Array, Pivot + 1, Last):
ENDIF;
END.
QuickSort

PROGRAM Partition(Array, First, Last):
PivotVal = Array[First];
Finished = False;
LeftPointer = First + 1;
RightPointer = Last;
QuickSort
We randomly select
the pivot, in this case
we select the first
element. Since the
array isn’t sorted yet,
the value of the first
element could have
any value
Continued 

WHILE NOT(Finished)
DO
WHILE (LeftPointer <= RightPointer) AND
(Age[LeftPointer] <= PivotVal)
DO LeftPointer = LeftPointer + 1
ENDWHILE;
WHILE (Age[RightPointer] >= PivotVal) AND
(RightPointer >= LeftPointer)
DO RightPointer = RightPointer - 1
ENDWHILE;
QuickSort
Continued 
 Continued
Keep moving left
until we find a
value that is less
than the pivot, or
we reach the Right
Pointer.
Keep moving right
until we find a
value that is
greater than the
pivot, or we reach
the left Pointer.

IF (LeftPointer < RightPointer)
THEN Finished = False;
ELSE SWAP(Age[LeftPointer], Age[RightPointer]);
ENDIF;
SWAP(Age[First], Age[RightPointer]);
RETURN RightPointer;
END Partition.
QuickSort
 Continued
We’ve a value
greater than the
pivot to the left,
and one less to the
right, swap them
Put the pivot in its
correct position

QuickSort
• Complexity of Quick Sort
– Best-case scenario complexity = O(N * log2(N))
– Worst-case scenario complexity = O(N2)

Advanced Sorting Algorithms

More Related Content

What's hot

Viewers also liked

Similar to Advanced Sorting Algorithms

More from Damian T. Gordon

Recently uploaded

Advanced Sorting Algorithms