KEMBAR78
Quick Sort , Merge Sort , Heap Sort | PDF
Eng: Mohammed Hussein1
Republic of Yemen
THAMAR UNIVERSITY
Faculty of Computer Science&
Information System
Outlines :
Eng: Mohammed Hussein
2
 Recursive algorithms
 There are two classes of Sorting Algorithms:
 O(n log n):
 Quick Sort
 Merge Sort
 Heap Sort
Recursive algorithms
 The rules for designing a recursive algorithm:
 First, determine the base case.
 Then determine the general case.
 Combine the base case and the general cases into an algorithm
 Each recursive call must reduce the size of the problem and move it
toward the base case.
 The base case, when reached, must terminate without a call to the
recursive algorithm; that is, it must execute a return.
3 Eng: Mohammed Hussein
Recursive Algorithm. (Example 1)
 Computing the factorial function F(n)
 Factorial (n) =
RecursiveAlgorithm
----------------------------------
Factorial (n)
if n=0
then return 1
return Factorial(n-1) * n
Iterative Algorithm
-----------------------------------
Factorial (n)
fact ← 1
for i ← 2 to n do
fact =fact * i
return fact
1 if n=0
n * Factorial (n) otherwise
4 Eng: Mohammed Hussein
Recursive Algorithm (Example 2)
 Computing Fibonacci sequence
 
   
0 if =0
1 if =1
1 2 otherwise
n
Fibonacci n n
Fibonacci n Fibonacci n


 
   
RecursiveAlgorithm
----------------------------------
Fibonacci (n)
if n=0 then
return 0
if n=1 then
return 1
return Fibonacci (n-1) + Fibonacci (n-2)
Iterative Algorithm
-----------------------------------
Fibonacci (n)
if n=0 return 0
if n=1 return 1
fib1 ← 0
fib2 ← 1
for i ← 2 to n do
fib ← fib1 + fib2
fib1 ← fib2
fib2 ← fib
return fib
5 Eng: Mohammed Hussein
Recursive Algorithm Analysis
 When an algorithm contains a recursive call to itself, its running
time can often be described by a recurrence equation or
recurrence
 It describes the overall running time on a problem of size n in
terms of running time on smaller inputs.
6 Eng: Mohammed Hussein
Recurrence Equation Analysis
 The conquer step of merge-sort consists of merging two sorted sequences, each with
n/2 elements and implemented by means of a doubly linked list, takes at most bn steps,
for some constant b.
 Likewise, the basis case (n < 2) will take at b most steps.
 Therefore, if we let T(n) denote the running time of merge-sort:
 We can therefore analyze the running time of merge-sort by finding a closed form
solution to the above equation.
 That is, a solution that has T(n) only on the left-hand side.






2if)2/(2
2if
)(
nbnnT
nb
nT
7 Eng: Mohammed Hussein
Quick sort
 The Quick sort steps are:
1. Select an element, called a pivot, from the list.
2. The partition step is a process that divides an unsorted array of elements into
two smaller arrays and the pivot element.The elements to the left of the pivot are
all smaller than the pivot and the elements to the right of the pivot are all larger
than the pivot.After this partitioning, the pivot is in its final position.
3. Recursively sort the sub-list of lesser elements and the sub-list of greater elements.
4. The base case of the recursion are lists of size zero or one, which never need to be
sorted.
8
Eng: Mohammed Hussein
Quick sort is a divide and conquer
algorithm.
Quick sort first divides a large list
into two smaller sub-lists: the low
elements and the high elements.
Quick sort
 The quick sort is performed by partition step, which divide the array
into two sets.
 To explain the algorithm we begin with unsorted array then we select
one element of the array to be the pivot (‫.)المحور‬
 The pivot should be the middle value of the array like in this example 17
or 34 are the best pivots
 quick sort algorithm face one difficulty which is selecting a correct
pivot.
9
Eng: Mohammed Hussein
Quick sort examples
10
Eng: Mohammed Hussein
 In this example the first pivot is 4.
 When the array elements became
like: [smaller <= 4 <= larger].
 We call again the Partition function
recursively, which divide the array
before and after of the first pivot.
 At the end all array have been
divided and all pivots collected in
the final array.
Quick sort code: Partition step function
11 Eng: Mohammed Hussein
 If the size less than 2 so it is sorted.
 Select pivot randomly.
 The first (Lower) and last (Upper) elements in an
array.
 The while loops that runs until the upper and lower
are equal.
 When L=U then we call partition ()recursively until
all elements are sorted.
U=(L+1,size-L-1)
Quick sort code
Eng: Mohammed Hussein12
 Inside this while loop
Elements that are < pivot .
 Inside this while loop
Elements that are > pivot .
Partition step
We move forward until we
find a value => pivot
We move backward until we
find a value <= pivot
13
Eng: Mohammed Hussein
Smaller < =pivot <= larger
Swap the values
Partition step: example
Eng: Mohammed Hussein14
 First, pivot in this example is (19). Each time we divide each array into
two arrays according to a pivot.
 Second, the pivot (23) of the right array and the pivot (5) of the left array.
 Last, the pivot (35) of the right array and the pivot (3) of the left array.
Merge sort
 To merge two sorted arrays, we index both arrays starting at zero,
where the smallest element is located.
 Comparing the elements at each index, we choose the smaller
element, put it into the array that we are merging into.
 Increment the index of the smaller element.
 By this method, we continually select the next smallest element from
the two arrays and merge them into one sorted array.
15 Eng: Mohammed Hussein
Merge sort
 Conceptually, a merge sort works
as follows:
 Recursion divide the unsorted list
into n sub lists, each containing 1
element (a list of 1 element is
considered sorted).
 Repeatedly Merge sub lists to
produce new sub lists until there is
only 1 sub list remaining. (This will
be the sorted list.)
16 Eng: Mohammed Hussein
Merging step
Merging step is the center future of merge algorithm.
Merging step takes two sorted array and merging them in one
order array.
which the first element in array is the smallest one.
17
Eng: Mohammed Hussein
Merging step
 Suppose if we have unsorted array like
 We can think this array as 8 sorted array like
 We merge each pair into larger array using merging step
18 Eng: Mohammed Hussein
Merging step
Eng: Mohammed Hussein19
 We start by considering each element of the array to be a sorted array
of length 1.Then we merge each pair of elements into an array of
length 2.
 Then pairs of arrays of length 2 can be merged into arrays of length
4, and so on until we reach the size of the array.
 The stages of the sort for an array of length 8 that is merge sorted by
three merging step passes.
For merge steps we need to allocate
new array for each merge, like this
example we need 7 array to sort this
array.Thus, west implement of merge
sort with fewer allocation.
Merge sort
• The outer loop start at one and doubles the index for each loop until size of
the array is reached. In side the inner loop, we call the merge function to
perform the merging steps.
1. &A[j]= pointer to start of the first array to be merged.
2. j= the start index of each arrays to be merged.
3. i= length of the first array (end index of first array).
4. Min(2i,size-j)=is the size of both arrays with the second array and based on
the pointers of first array.
• The second array may be shorter than i at the end of the arrayA.
20
Eng: Mohammed Hussein
Merge step functions
21 Eng: Mohammed Hussein
Merge step functions
 First while loop runs two
elements in two arrays to be
merged and copy each smallest
elements into temp array.
 Temp is dynamically allocated, so
in this case we need to allocate
every time new array to perform
merge steps.Thus, led to west
memory.
22 Eng: Mohammed Hussein
Merge step functions
 The two while loops run to
copy the remaining of largest
elements from first or second
arrays into temp array.
 This final for loop runs to
copy the merge element from
the temp array backed to the
original array (A).
23 Eng: Mohammed Hussein
Heap sort
 After removing the largest item, it reconstructs the heap, removes
the largest remaining item, and places it in the next open position
from the end of the partially sorted array.This is repeated until there
are no items left in the heap and the sorted array is full.
 Elementary implementations require two arrays - one to hold the
heap and the other to hold the sorted elements.
24 Eng: Mohammed Hussein
Heap sort begins by building a
heap out of the data set, and
then removing the largest item
and placing it at the end of the
partially sorted array.
Heap sort steps
Eng: Mohammed Hussein25
 The heap sort algorithm consists of two primary steps:
 First, we construct a heap from the elements.
 Second, we repeatedly take the largest element of the heap and swap
it with the end until we fully sort the array.
Heap sort code functions
26 Eng: Mohammed Hussein
AddElement(int,int*,int)
RemoveRoot(int*,int)
Main ()
SwapWithChild(int, int*, int)
Left(int)
Right(int)
Swap(int &, int &)
Parent(int)
SwapWithParent(int,int*)
Swap(int &, int &)
Add element to the heap
Eng: Mohammed Hussein27
 The procedure for adding an element into the heap runs as follows:
1. Expand the size of the heap so that it contains the next element of the
array. Put the new element in its place by exchanging it with its parent
while it is larger than its parent.
2. Sort the elements of the heap, starting from the last index to the first.
 In the image below shows what this looks like after nine elements have
been sorted.The last nine sorted elements are green, while the first
six gray elements are still in heap order.
Remove element from the heap
Eng: Mohammed Hussein28
 The procedure for removing a single element from the heap runs as
follows:
1. Swap the root element to the end of the heap and shrink the size of
the heap to remove it from the heap.At this point, the root element
has been put into its proper sorted position and the new root may
violate the heap property.
2. Fix the new root, swap it with its larger child node as long as one of
the child nodes is larger.
Example of Heap sort: Add Element
Eng: Mohammed Hussein29
Example of Heap sort: Add Element
Eng: Mohammed Hussein30
Example of Heap sort: Remove Root
Eng: Mohammed Hussein31
Example of Heap sort: Remove Root
Eng: Mohammed Hussein32
Heap: Add Element
33 Eng: Mohammed Hussein
Heap: Remove Root
34 Eng: Mohammed Hussein
Divide-and-Conquer algorithms
Eng: Mohammed Hussein35
 Heap sort, Merge sort and Quick sort.
 Why do we need multiple sorting algorithms?
 Different methods work better in different applications.
1. Heap sort uses close to the right number of comparisons but needs to
move data around quite a bit. It can be done in a way that uses very little
extra memory. It's probably good when memory is tight, and you are
sorting many small items that come stored in an array.
2. Merge sort is good for data that's too big to have in memory at once,
because its pattern of storage access is very regular. It also uses even
fewer comparisons than heap sort, and is especially suited for data stored
as linked lists.
3. Quick sort also uses few comparisons (somewhat more than the other
two). Like heap sort it can sort "in place" by moving data in an array.
Divide-and-Conquer
 The whole problem we want to solve may too big to understand or solve at
once. We break it up into smaller pieces, solve the pieces separately, and
combine the separate pieces together.
 Divide-and conquer is a general algorithm design paradigm:
 Divide: divide the input data S in two or more disjoint subsets S1, S2,
…
 Recur: solve the subproblems recursively
 Conquer: combine the solutions for S1, S2, …, into a solution for S
 The base case for the recursion are subproblems of constant size
 Analysis can be done using recurrence equations
36 Eng: Mohammed Hussein
Reference
 Wikipedia, the free encyclopedia
 XoaX.net
37 Eng: Mohammed Hussein

Quick Sort , Merge Sort , Heap Sort

  • 1.
    Eng: Mohammed Hussein1 Republicof Yemen THAMAR UNIVERSITY Faculty of Computer Science& Information System
  • 2.
    Outlines : Eng: MohammedHussein 2  Recursive algorithms  There are two classes of Sorting Algorithms:  O(n log n):  Quick Sort  Merge Sort  Heap Sort
  • 3.
    Recursive algorithms  Therules for designing a recursive algorithm:  First, determine the base case.  Then determine the general case.  Combine the base case and the general cases into an algorithm  Each recursive call must reduce the size of the problem and move it toward the base case.  The base case, when reached, must terminate without a call to the recursive algorithm; that is, it must execute a return. 3 Eng: Mohammed Hussein
  • 4.
    Recursive Algorithm. (Example1)  Computing the factorial function F(n)  Factorial (n) = RecursiveAlgorithm ---------------------------------- Factorial (n) if n=0 then return 1 return Factorial(n-1) * n Iterative Algorithm ----------------------------------- Factorial (n) fact ← 1 for i ← 2 to n do fact =fact * i return fact 1 if n=0 n * Factorial (n) otherwise 4 Eng: Mohammed Hussein
  • 5.
    Recursive Algorithm (Example2)  Computing Fibonacci sequence       0 if =0 1 if =1 1 2 otherwise n Fibonacci n n Fibonacci n Fibonacci n         RecursiveAlgorithm ---------------------------------- Fibonacci (n) if n=0 then return 0 if n=1 then return 1 return Fibonacci (n-1) + Fibonacci (n-2) Iterative Algorithm ----------------------------------- Fibonacci (n) if n=0 return 0 if n=1 return 1 fib1 ← 0 fib2 ← 1 for i ← 2 to n do fib ← fib1 + fib2 fib1 ← fib2 fib2 ← fib return fib 5 Eng: Mohammed Hussein
  • 6.
    Recursive Algorithm Analysis When an algorithm contains a recursive call to itself, its running time can often be described by a recurrence equation or recurrence  It describes the overall running time on a problem of size n in terms of running time on smaller inputs. 6 Eng: Mohammed Hussein
  • 7.
    Recurrence Equation Analysis The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b.  Likewise, the basis case (n < 2) will take at b most steps.  Therefore, if we let T(n) denote the running time of merge-sort:  We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation.  That is, a solution that has T(n) only on the left-hand side.       2if)2/(2 2if )( nbnnT nb nT 7 Eng: Mohammed Hussein
  • 8.
    Quick sort  TheQuick sort steps are: 1. Select an element, called a pivot, from the list. 2. The partition step is a process that divides an unsorted array of elements into two smaller arrays and the pivot element.The elements to the left of the pivot are all smaller than the pivot and the elements to the right of the pivot are all larger than the pivot.After this partitioning, the pivot is in its final position. 3. Recursively sort the sub-list of lesser elements and the sub-list of greater elements. 4. The base case of the recursion are lists of size zero or one, which never need to be sorted. 8 Eng: Mohammed Hussein Quick sort is a divide and conquer algorithm. Quick sort first divides a large list into two smaller sub-lists: the low elements and the high elements.
  • 9.
    Quick sort  Thequick sort is performed by partition step, which divide the array into two sets.  To explain the algorithm we begin with unsorted array then we select one element of the array to be the pivot (‫.)المحور‬  The pivot should be the middle value of the array like in this example 17 or 34 are the best pivots  quick sort algorithm face one difficulty which is selecting a correct pivot. 9 Eng: Mohammed Hussein
  • 10.
    Quick sort examples 10 Eng:Mohammed Hussein  In this example the first pivot is 4.  When the array elements became like: [smaller <= 4 <= larger].  We call again the Partition function recursively, which divide the array before and after of the first pivot.  At the end all array have been divided and all pivots collected in the final array.
  • 11.
    Quick sort code:Partition step function 11 Eng: Mohammed Hussein  If the size less than 2 so it is sorted.  Select pivot randomly.  The first (Lower) and last (Upper) elements in an array.  The while loops that runs until the upper and lower are equal.  When L=U then we call partition ()recursively until all elements are sorted. U=(L+1,size-L-1)
  • 12.
    Quick sort code Eng:Mohammed Hussein12  Inside this while loop Elements that are < pivot .  Inside this while loop Elements that are > pivot .
  • 13.
    Partition step We moveforward until we find a value => pivot We move backward until we find a value <= pivot 13 Eng: Mohammed Hussein Smaller < =pivot <= larger Swap the values
  • 14.
    Partition step: example Eng:Mohammed Hussein14  First, pivot in this example is (19). Each time we divide each array into two arrays according to a pivot.  Second, the pivot (23) of the right array and the pivot (5) of the left array.  Last, the pivot (35) of the right array and the pivot (3) of the left array.
  • 15.
    Merge sort  Tomerge two sorted arrays, we index both arrays starting at zero, where the smallest element is located.  Comparing the elements at each index, we choose the smaller element, put it into the array that we are merging into.  Increment the index of the smaller element.  By this method, we continually select the next smallest element from the two arrays and merge them into one sorted array. 15 Eng: Mohammed Hussein
  • 16.
    Merge sort  Conceptually,a merge sort works as follows:  Recursion divide the unsorted list into n sub lists, each containing 1 element (a list of 1 element is considered sorted).  Repeatedly Merge sub lists to produce new sub lists until there is only 1 sub list remaining. (This will be the sorted list.) 16 Eng: Mohammed Hussein
  • 17.
    Merging step Merging stepis the center future of merge algorithm. Merging step takes two sorted array and merging them in one order array. which the first element in array is the smallest one. 17 Eng: Mohammed Hussein
  • 18.
    Merging step  Supposeif we have unsorted array like  We can think this array as 8 sorted array like  We merge each pair into larger array using merging step 18 Eng: Mohammed Hussein
  • 19.
    Merging step Eng: MohammedHussein19  We start by considering each element of the array to be a sorted array of length 1.Then we merge each pair of elements into an array of length 2.  Then pairs of arrays of length 2 can be merged into arrays of length 4, and so on until we reach the size of the array.  The stages of the sort for an array of length 8 that is merge sorted by three merging step passes. For merge steps we need to allocate new array for each merge, like this example we need 7 array to sort this array.Thus, west implement of merge sort with fewer allocation.
  • 20.
    Merge sort • Theouter loop start at one and doubles the index for each loop until size of the array is reached. In side the inner loop, we call the merge function to perform the merging steps. 1. &A[j]= pointer to start of the first array to be merged. 2. j= the start index of each arrays to be merged. 3. i= length of the first array (end index of first array). 4. Min(2i,size-j)=is the size of both arrays with the second array and based on the pointers of first array. • The second array may be shorter than i at the end of the arrayA. 20 Eng: Mohammed Hussein
  • 21.
    Merge step functions 21Eng: Mohammed Hussein
  • 22.
    Merge step functions First while loop runs two elements in two arrays to be merged and copy each smallest elements into temp array.  Temp is dynamically allocated, so in this case we need to allocate every time new array to perform merge steps.Thus, led to west memory. 22 Eng: Mohammed Hussein
  • 23.
    Merge step functions The two while loops run to copy the remaining of largest elements from first or second arrays into temp array.  This final for loop runs to copy the merge element from the temp array backed to the original array (A). 23 Eng: Mohammed Hussein
  • 24.
    Heap sort  Afterremoving the largest item, it reconstructs the heap, removes the largest remaining item, and places it in the next open position from the end of the partially sorted array.This is repeated until there are no items left in the heap and the sorted array is full.  Elementary implementations require two arrays - one to hold the heap and the other to hold the sorted elements. 24 Eng: Mohammed Hussein Heap sort begins by building a heap out of the data set, and then removing the largest item and placing it at the end of the partially sorted array.
  • 25.
    Heap sort steps Eng:Mohammed Hussein25  The heap sort algorithm consists of two primary steps:  First, we construct a heap from the elements.  Second, we repeatedly take the largest element of the heap and swap it with the end until we fully sort the array.
  • 26.
    Heap sort codefunctions 26 Eng: Mohammed Hussein AddElement(int,int*,int) RemoveRoot(int*,int) Main () SwapWithChild(int, int*, int) Left(int) Right(int) Swap(int &, int &) Parent(int) SwapWithParent(int,int*) Swap(int &, int &)
  • 27.
    Add element tothe heap Eng: Mohammed Hussein27  The procedure for adding an element into the heap runs as follows: 1. Expand the size of the heap so that it contains the next element of the array. Put the new element in its place by exchanging it with its parent while it is larger than its parent. 2. Sort the elements of the heap, starting from the last index to the first.  In the image below shows what this looks like after nine elements have been sorted.The last nine sorted elements are green, while the first six gray elements are still in heap order.
  • 28.
    Remove element fromthe heap Eng: Mohammed Hussein28  The procedure for removing a single element from the heap runs as follows: 1. Swap the root element to the end of the heap and shrink the size of the heap to remove it from the heap.At this point, the root element has been put into its proper sorted position and the new root may violate the heap property. 2. Fix the new root, swap it with its larger child node as long as one of the child nodes is larger.
  • 29.
    Example of Heapsort: Add Element Eng: Mohammed Hussein29
  • 30.
    Example of Heapsort: Add Element Eng: Mohammed Hussein30
  • 31.
    Example of Heapsort: Remove Root Eng: Mohammed Hussein31
  • 32.
    Example of Heapsort: Remove Root Eng: Mohammed Hussein32
  • 33.
    Heap: Add Element 33Eng: Mohammed Hussein
  • 34.
    Heap: Remove Root 34Eng: Mohammed Hussein
  • 35.
    Divide-and-Conquer algorithms Eng: MohammedHussein35  Heap sort, Merge sort and Quick sort.  Why do we need multiple sorting algorithms?  Different methods work better in different applications. 1. Heap sort uses close to the right number of comparisons but needs to move data around quite a bit. It can be done in a way that uses very little extra memory. It's probably good when memory is tight, and you are sorting many small items that come stored in an array. 2. Merge sort is good for data that's too big to have in memory at once, because its pattern of storage access is very regular. It also uses even fewer comparisons than heap sort, and is especially suited for data stored as linked lists. 3. Quick sort also uses few comparisons (somewhat more than the other two). Like heap sort it can sort "in place" by moving data in an array.
  • 36.
    Divide-and-Conquer  The wholeproblem we want to solve may too big to understand or solve at once. We break it up into smaller pieces, solve the pieces separately, and combine the separate pieces together.  Divide-and conquer is a general algorithm design paradigm:  Divide: divide the input data S in two or more disjoint subsets S1, S2, …  Recur: solve the subproblems recursively  Conquer: combine the solutions for S1, S2, …, into a solution for S  The base case for the recursion are subproblems of constant size  Analysis can be done using recurrence equations 36 Eng: Mohammed Hussein
  • 37.
    Reference  Wikipedia, thefree encyclopedia  XoaX.net 37 Eng: Mohammed Hussein