Sorting and Hashing Algorithm full pdfs.

The Sorting Problem
 Input:
 A sequence of n numbers <a1, a2, . . . , an>
 Output:
 A reordering <a1′, a2′, . . . , an′> of the input sequence such that
a1′ ≤ a2′ ≤ · · · ≤ an′

Why Study Sorting Algorithms?
 There are a variety of situations that are encountered:
 Do the keys randomly ordered?
 Are all keys distinct?
 How large is the set of keys to be ordered?
 Need guaranteed performance?
 Various algorithms are better suited to some of these situations

Some Definitions
 Internal Sort
 The data to be sorted is all stored in the computer’s main
memory.
 External Sort
 Some of the data to be sorted might be stored in some
external, slower device.

Insertion Sort
 Idea: like sorting a hand of playing cards
 Start with an empty left hand and the cards facing down on
the table.
 Remove one card at a time from the table, and insert it into
the correct position in the left hand
 compare it with each of the cards already in the hand, from
right to left
 The cards held in the left hand are sorted
 these cards were originally the top cards of the pile on the
table

To insert 12, make a room for it by
moving first 36 and then 24.
Insertion Sort

Insertion Sort
5 2 4 6 1 3
input array
left sub-array right sub-array
at each iteration, the array is divided in two sub-arrays:
sorted unsorted

Insertion Sort
Alg.: Insertion-Sort(A)
for j = 2 to n
do key = A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i = j - 1
while i > 0 and A[i] > key
do A[i + 1] = A[i]
i = i – 1
A[i + 1] = key
 Insertion sort – sorts the elements in place
a8
a7
a6
a5
a4
a3
a2
a1
1 2 3 4 5 6 7 8
key

Loop Invariant for Insertion Sort
Alg.: Insertion-Sort(A)
for j = 2 to n
do key = A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i = j - 1
while i > 0 and A[i] > key
do A[i + 1] = A[i]
i = i – 1
A[i + 1] = key
Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order

Proving Loop Invariants
 Proving loop invariants works like induction
 Initialization (base case):
 It is true prior to the first iteration of the loop
 Maintenance (inductive step):
 If it is true before an iteration of the loop, it remains true before
the next iteration
 Termination:
 When the loop terminates, the invariant gives a useful property that
helps show that the algorithm is correct
 Stop the induction when the loop terminates

 Initialization:
 Just before the first iteration, j = 2:
the subarray A[1 . . j-1] = A[1], (the
element originally in A[1]) – is sorted

 Maintenance:
 the while inner loop moves A[j -1], A[j -2], A[j -3], and so on,
by one position to the right until the proper position for key
(which has the value that started out in A[j]) is found
 At that point, the value of key is placed into this position.

 Termination:
 The outer for loop ends when j = n + 1  j-1 = n
 Replace n with j-1 in the loop invariant:
 the subarray A[1 . . n] consists of the elements originally
in A[1 . . n], but in sorted order
 The entire array is sorted!
j
j - 1
Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order

Time complexity of Insertion Sort
 The worst case time complexity of Insertion sort is O(n2)
 when the elements are in reverse sorted order
 The average case time complexity of Insertion sort is O(n2)
 The time complexity of the best case is O(n).
 when the elements are in sorted order
 The space complexity is O(1)

Bubble sort
 Compare each element (except the last one) with its neighbor to
the right
 If they are out of order, swap them
 This puts the largest element at the very end
 The last element is now in the correct and final place
 Compare each element (except the last two) with its neighbor to
the right
 If they are out of order, swap them
 This puts the second largest element next to last
 The last two elements are now in their correct and final places
 Compare each element (except the last three) with its neighbor to
the right
 Continue as above until no unsorted elements on the left

"Bubbling Up" the Largest Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using pair-wise
comparisons and swapping
5
12
35
42
77 101
1 2 3 4 5 6

 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using pair-wise
5
12
35
42
77 101
1 2 3 4 5 6
Swap
42 77

5
12
35
77
42 101
1 2 3 4 5 6
Swap
35 77

5
12
77
35
42 101
1 2 3 4 5 6
Swap
12 77

5
77
12
35
42 101
1 2 3 4 5 6
No need to swap

5
77
12
35
42 101
1 2 3 4 5 6
Swap
5 101

Reducing the Number of Comparisons
12
35
42
77 101
1 2 3 4 5 6
5
77
12
35
42 5
1 2 3 4 5 6
101
5
42
12
35 77
1 2 3 4 5 6
101
42
5
35
12 77
1 2 3 4 5 6
101
42
35
5
12 77
1 2 3 4 5 6
101

 On the nth “bubble up”, we only need to do MAX-n
comparisons.
 For example:
 This is the 4th “bubble up”
 MAX is 6
 Thus we have 2 comparisons to do
42
5
35
12 77
1 2 3 4 5 6
101
Reducing the Number of Comparisons

BubbleSort(A: Arr_Type)
iterations, index: integer
iterations = n – 1
loop
exitif(iterations = 0)
index = 1
loop
exitif(index > iterations)
if(A[index] > A[index + 1]) then
swap(A[index], A[index + 1])
endif
index = index + 1
endloop
iterations = iterations - 1
endloop
endprocedure
Inner
loop
Outer
loop
Algorithm

Already Sorted Collections?
 What if the collection was already sorted?
 What if only a few elements were out of place and after a
couple of “bubble ups,” the collection was sorted?
 We want to be able to detect this and “stop early”!
42
35
12
5 77
1 2 3 4 5 6
101

Using a Boolean “Flag”
 Use a boolean variable to determine if any swapping occurred
during the “bubble up.”
 If no swapping occurred, then the collection is already sorted!
 This boolean “flag” needs to be reset after each “bubble up”

flag: Boolean
flag = true
iterations = n – 1
loop
exitif ((iterations = 0) OR NOT(flag))
index = 1
flag = false
loop
exitif(index > iterations)
if(A[index] > A[index + 1]) then
swap(A[index], A[index + 1])
flag = true
endif
index = index + 1
endloop
iterations = iterations - 1
endloop
Revised Algorithm

An Animated Example
67
45
23 14 6 33
98 42
1 2 3 4 5 6 7 8
iterations
index
7
n 8 flag true

An Animated Example
67
45
23 14 6 33
98 42
1 2 3 4 5 6 7 8
iteration
index
7
1
n 8 flag false

An Animated Example
67
45
23 14 6 33
98 42
1 2 3 4 5 6 7 8
iteration
index
7
1
n 8 flag false
Swap

An Animated Example
67
45
98 14 6 33
23 42
1 2 3 4 5 6 7 8
iteration
index
7
1
n 8 flag true
Swap

An Animated Example
67
45
98 14 6 33
23 42
1 2 3 4 5 6 7 8
7
2
8 did_swap true
flag
iterations
index
n

An Animated Example
67
45
98 14 6 33
23 42
1 2 3 4 5 6 7 8
7
2
8
Swap
did_swap true
flag
iterations
index
n

An Animated Example
67
98
45 14 6 33
23 42
1 2 3 4 5 6 7 8
7
2
8
Swap
did_swap true
flag
iterations
index
n

An Animated Example
67
98
45 14 6 33
23 42
1 2 3 4 5 6 7 8
7
3
8 true
flag
iterations
index
n

An Animated Example
67
98
45 14 6 33
23 42
1 2 3 4 5 6 7 8
7
3
8
Swap
true
iterations
index
n flag

An Animated Example
67
14
45 98 6 33
23 42
1 2 3 4 5 6 7 8
7
3
8
Swap
did_swap true
flag
iterations
index
n

An Animated Example
67
14
45 98 6 33
23 42
1 2 3 4 5 6 7 8
7
4
8 true
iterations
index
n flag

An Animated Example
67
14
45 98 6 33
23 42
1 2 3 4 5 6 7 8
7
4
8
Swap
true
iterations
index
n flag

An Animated Example
67
14
45 6 98 33
23 42
1 2 3 4 5 6 7 8
7
4
8
Swap
true
iterations
index
n flag

An Animated Example
67
14
45 6 98 33
23 42
1 2 3 4 5 6 7 8
7
5
8 true
iterations
index
n flag

An Animated Example
67
14
45 6 98 33
23 42
1 2 3 4 5 6 7 8
7
5
8
Swap
true
iterations
index
n flag

An Animated Example
98
14
45 6 67 33
23 42
1 2 3 4 5 6 7 8
7
5
8
Swap
true
flag
iterations
index
n

An Animated Example
98
14
45 6 67 33
23 42
1 2 3 4 5 6 7 8
7
6
8 true
iterations
index
n flag

An Animated Example
98
14
45 6 67 33
23 42
1 2 3 4 5 6 7 8
7
6
8
Swap
true
iterations
index
n flag

An Animated Example
33
14
45 6 67 98
23 42
1 2 3 4 5 6 7 8
7
6
8
Swap
true
iterations
index
n flag

An Animated Example
33
14
45 6 67 98
23 42
1 2 3 4 5 6 7 8
7
7
8 true
iterations
index
n flag

An Animated Example
33
14
45 6 67 98
23 42
1 2 3 4 5 6 7 8
7
7
8
Swap
true
iterations
index
n flag

An Animated Example
33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
7
7
8
Swap
true
iterations
index
n flag

After First Pass of Outer Loop
33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
7
8
8
Finished first “Bubble Up”
true
iterations
index
n flag

The Second “Bubble Up”
33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
6
1
8 false
iterations
index
n flag

33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
6
1
8 false
No Swap
iterations
index
n flag

33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
6
2
8 false
iterations
index
n flag

33
14
45 6 67 42
23 98
1 2 3 4 5 6 7 8
6
2
8 false
Swap
iterations
index
n flag

33
45
14 6 67 42
23 98
1 2 3 4 5 6 7 8
6
2
8 true
Swap
iterations
index
n flag

33
45
14 6 67 42
23 98
1 2 3 4 5 6 7 8
6
3
8 true
iterations
index
n flag

33
45
14 6 67 42
23 98
1 2 3 4 5 6 7 8
6
3
8 true
Swap
iterations
index
n flag

33
6
14 45 67 42
23 98
1 2 3 4 5 6 7 8
6
3
8 true
Swap
iterations
index
n flag

33
6
14 45 67 42
23 98
1 2 3 4 5 6 7 8
6
4
8 true
iterations
index
n flag

33
6
14 45 67 42
23 98
1 2 3 4 5 6 7 8
6
4
8 true
No Swap
iterations
index
n flag

33
6
14 45 67 42
23 98
1 2 3 4 5 6 7 8
6
5
8 true
iterations
index
n flag

33
6
14 45 67 42
23 98
1 2 3 4 5 6 7 8
6
5
8 true
Swap
iterations
index
n flag

67
6
14 45 33 42
23 98
1 2 3 4 5 6 7 8
6
5
8 true
Swap
iterations
index
n flag

67
6
14 45 33 42
23 98
1 2 3 4 5 6 7 8
6
6
8 true
iterations
index
n flag

67
6
14 45 33 42
23 98
1 2 3 4 5 6 7 8
6
6
8 true
Swap
iterations
index
n flag

42
6
14 45 33 67
23 98
1 2 3 4 5 6 7 8
6
6
8 true
Swap
iterations
index
n flag

After Second Pass of Outer Loop
42
6
14 45 33 67
23 98
1 2 3 4 5 6 7 8
6
7
8 true
Finished second “Bubble Up”
iterations
index
n flag

The Third “Bubble Up”
42
6
14 45 33 67
23 98
1 2 3 4 5 6 7 8
5
1
8 false
iterations
index
n flag

42
6
14 45 33 67
23 98
1 2 3 4 5 6 7 8
5
1
8 false
Swap
iterations
index
n flag

42
6
23 45 33 67
14 98
1 2 3 4 5 6 7 8
5
1
8 true
Swap
iterations
index
n flag

42
6
23 45 33 67
14 98
1 2 3 4 5 6 7 8
5
2
8 true
iterations
index
n flag

42
6
23 45 33 67
14 98
1 2 3 4 5 6 7 8
5
2
8 true
Swap
iterations
index
n flag

42
23
6 45 33 67
14 98
1 2 3 4 5 6 7 8
5
2
8 true
Swap
iterations
index
n flag
42
45 33
14
5
2
8

42
23
6 45 33 67
14 98
1 2 3 4 5 6 7 8
5
3
8 true
iterations
index
n flag

42
23
6 45 33 67
14 98
1 2 3 4 5 6 7 8
5
3
8 true
No Swap
iterations
index
n flag

42
23
6 45 33 67
14 98
1 2 3 4 5 6 7 8
5
4
8 true
iterations
index
n flag

42
23
6 45 33 67
14 98
1 2 3 4 5 6 7 8
5
4
8 true
Swap
iterations
index
n flag

42
23
6 33 45 67
14 98
1 2 3 4 5 6 7 8
5
4
8 true
Swap
iterations
index
n flag

42
23
6 33 45 67
14 98
1 2 3 4 5 6 7 8
5
5
8 true
iterations
index
n flag

42
23
6 33 45 67
14 98
1 2 3 4 5 6 7 8
5
5
8 true
Swap
iterations
index
n flag

45
23
6 33 42 67
14 98
1 2 3 4 5 6 7 8
5
5
8 true
Swap
iterations
index
n flag

After Third Pass of Outer Loop
45
23
6 33 42 67
14 98
1 2 3 4 5 6 7 8
5
6
8 true
Finished third “Bubble Up”
iterations
index
n flag

The Fourth “Bubble Up”
45
23
6 33 42 67
14 98
1 2 3 4 5 6 7 8
4
1
8 false
iterations
index
n flag

45
23
6 33 42 67
14 98
1 2 3 4 5 6 7 8
4
1
8 false
Swap
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
1
8 true
Swap
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
2
8 true
iterations
index
n flag
6
4
8 true

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
2
8 true
No Swap
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
3
8 true
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
3
8 true
No Swap
iterations
index
n flag
4
3
8 true

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
4
8 true
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
4
8 true
No Swap
iterations
index
n flag
4
8

After Fourth Pass of Outer Loop
45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
4
5
8 true
Finished fourth “Bubble Up”
iterations
index
n flag

The Fifth “Bubble Up”
45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
1
8 false
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
1
8 false
No Swap
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
2
8 false
iterations
index
n flag
14
6
3
8 false

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
2
8 false
No Swap
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
3
8 false
iterations
index
n flag

45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
3
8 false
No Swap
iterations
index
n flag

After Fifth Pass of Outer Loop
45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
4
8 false
Finished fifth “Bubble Up”
iterations
index
n flag

Finished “Early”
45
23
14 33 42 67
6 98
1 2 3 4 5 6 7 8
3
4
8 false
We didn’t do any swapping,
so all of the other elements
must be correctly placed.
We can “skip” the last two
passes of the outer loop.
iterations
index
n flag

Bubble Sort
Alg.: Bubble-Sort(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j] < A[j -1]
then exchange A[j]  A[j-1]
1
3
2
9
6
4
8
i = 1 j
i

Bubble Sort: Analysis
 The biggest items “bubble up” to the end of the array, as the
algorithm progresses.
 Efficiency:
 if n is number of items,
 n-1 comparisons in first pass, n-2 in second pass, so on and so forth.
 The formula is : (n-1) + (n-2) +…+ 1 = n×(n-1)/2.
 hence runs in O(n2) time.

Time complexity of Bubble Sort
 The worst case time complexity of Bubble sort is O(n2)
 The average case time complexity of Bubble sort is O(n2)
 The time complexity of the best case is O(n).
 when thr elements are in sorted order

Selection Sort
 Idea:
 Find the smallest/ largest element in the array
 Exchange it with the element in the first/ last position
 Find the second smallest/ largest element and exchange it
with the element in the second position
 Continue until the array is sorted

Selection Sort
2
6
4
3
1
5
Comparison
Data Movement
Sorted

2
6
4
3
1
5
Comparison
Data Movement
Sorted
Selection Sort

2
6
4
3
1
5
Comparison
Data Movement
Sorted

Largest
Selection Sort

6
2
4
3
1
5
Comparison
Data Movement
Sorted
Selection Sort

6
2
4
3
1
5
Comparison
Data Movement
Sorted

Largest
Selection Sort

6
5
4
3
1
2
Comparison
Data Movement
Sorted
Selection Sort

6
5
4
3
1
2
Comparison
Data Movement
Sorted

Largest
Selection Sort

6
5
4
3
2
1
Comparison
Data Movement
Sorted
Selection Sort

6
5
4
3
2
1
Comparison
Data Movement
Sorted
DONE!
Selection Sort

Alg.: Selection-Sort(A)
n ← length[A]
for j ← 1 to n-1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]
Selection Sort
Alg.: Selection-Sort(A)
n ← length[A]
for j ← n downto 2
do largest ← 1
for i ← 2 to j
do if A[i] > A[largest]
then largest ← i
exchange A[j] ↔ A[largest]

Time complexity of Selection Sort
 The worst case time complexity of Bubble sort is O(n2)
 The average case time complexity of Bubble sort is O(n2)
 The time complexity of the best case is O(n2).
 when thr elements are in sorted order
The number of swaps in Selection Sort are as follows:
 Worst case: O(n)
 Average Case: O(n)
 Best Case: O(1)

Divide and Conquer
 Recursive in structure
 Divide the problem into sub-problems that are similar to the
original but smaller in size
 Conquer the sub-problems by solving them recursively. If
they are small enough, just solve them in a straightforward
manner.
 Combine the solutions to create a solution to the original
problem

An Example: Merge Sort
Sorting Problem: Sort a sequence of n elements into non-decreasing
order.
 Divide: Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
 Conquer: Sort the two subsequences recursively using merge
sort.
 Combine: Merge the two sorted subsequences to produce the
sorted answer.

Mergesort
 Divide it in two at the midpoint
 Conquer each side in turn (by recursively sorting)
 Merge two halves together
8 2 9 4 5 3 1 6

Merging
 The key point to Merge Sort is merging two sorted lists into one.
 Suppose, there are two lists X (x1  x2  …  xm) and Y(y1  y2  …  yn)
the resulting list is Z(z1  z2  …  zm+n)
 Example:
L1 = {3, 8, 9} L2 = {1, 5, 7}
merge(L1, L2) = {1, 3, 5, 7, 8, 9}

Merging (cont.)
54
23
10
3 75
25
5
1
X: Y:
Result:

Merging (cont.)
54
23
10 75
25
5
3
1
X: Y:
Result:

Merging (cont.)
54
23 75
25
10
5
3
1
X: Y:
Result:

Merging (cont.)
54 75
25
23
10
5
3
1
X: Y:
Result:

Merging (cont.)
75
54
25
23
10
5
3
1
X: Y:
Result:

Merge-Sort (A, p, r)
Input: a sequence of n numbers stored in array A
Output: an ordered sequence of n numbers
MergeSort (A, p, r) // sort A[p...r] by divide & conquer
1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]
Initial Call: MergeSort(A, 1, n)

Merge Sort Example
0
4
86
35
58
15
86
6
99

Merge Sort Example
0
4
86
35
58
15
86
6
99
15
86
6
99 0
4
86
35
58

Merge Sort Example
0
4
86
35
58
15
86
6
99
15
86
6
99 0
4
86
35
58
15
86
6
99 35
58 0
4
86

Merge Sort Example
0
4
86
35
58
15
86
6
99
15
86
6
99 0
4
86
35
58
15
86
6
99 35
58 0
4
86
99 6 86 15 58 35 86 0
4

Merge Sort Example
0
4
86
35
58
15
86
6
99
15
86
6
99 0
4
86
35
58
15
86
6
99 35
58 0
4
86
99 6 86 15 58 35 86 0
4
4 0

Merge Sort Example
99 6 86 15 58 35 86 4
0
4 0
Merge

Merge Sort Example
86
15
99
6 35
58 86
4
0
99 6 86 15 58 35 86 4
0
Merge

Merge Sort Example
99
86
15
6 86
58
35
4
0
86
15
99
6 35
58 86
4
0
Merge

Merge Sort Example
99
86
86
58
35
15
6
4
0
99
86
15
6 86
58
35
4
0
Merge

Merge Sort Example
99
86
86
58
35
15
6
4
0

Merge Algorithm
void merge(int *arr, int beg, int mid, int end) {
int temp[end - beg + 1];
int i = beg, j = mid+1, k = 1;
while(i <= mid && j <= end) {
if(arr[i] <= arr[j]) {
temp[k] = arr[i];
k += 1; i += 1;
}
else {
temp[k] = arr[j];
k += 1; j += 1;
}
}
// add elements left in the first interval
while(i <= mid) {
temp[k] = arr[i];
k += 1; i += 1;
}
// add elements left in the second interval
while(j <= end) {
temp[k] = arr[j];
k += 1; j += 1;
}
// copy temp to original interval
for(i = beg; i <= end; i++)
arr[i] = temp[i - beg];
}

Implementing Merge Sort
Basic way to implement merge sort:
 Merging is done with a temporary array of the same size as the
input array
 Pro: Faster
 Con: The memory requirement is doubled.

Merge Sort Analysis
The Double Memory Merge Sort runs O(n log n) for all cases,
because of its Divide and Conquer approach.
T(n) = 2T(n/2) + n = O(n logn)

QuickSort
 The quick sort uses divide and conquer to gain the same
advantages as the merge sort, while not using additional
storage.
 As a trade-off, it is possible that the list may not be divided in
half.
 When this happens, it has been seen that performance is
diminished.

174
Quick Sort
 Fastest known sorting algorithm in practice
 Average case: O(n log n)
 Worst case: O(n2)
 But the worst case can be made exponentially unlikely.
 Another divide-and-conquer recursive algorithm, like merge sort.

QuickSort
 A quick sort first selects a value, which is called the pivot value.
 Although there are many different ways to choose the pivot value, it
is simply used the first/ last item in the list
 The role of the pivot value is to assist with splitting the list.
 The actual position where the pivot value belongs in the final sorted
list, commonly called the split point, will be used to divide the list for
subsequent calls to the quick sort.

176
Quick Sort: Main Idea
1. If the number of elements in S is 0 or 1, then return (base case).
2. Pick any element v in S (called the pivot).
3. Partition the elements in S except v into two disjoint groups:
1. S1 = {x S – {v} | x  v}
2. S2 = {x S – {v} | x  v}
4. Return {QuickSort(S1) + v + QuickSort(S2)}

QuickSort
Divide-and-Conquer:
 Divide: partition A[p..r] into two subarrays A[p..q-1] and A[q+1..r]
such that each element of A[p..q-1] is ≤ A[q], and each element
of A[q+1..r] is ≥ A[q].
 Compute q as part of this partitioning.
 Conquer: sort the subarrays A[p..q-1] and A[q+1..r] by recursive
calls to Quicksort.
 Combine: the partitioning and recursive sorting leave us with a
sorted A[p..r]

Partitioning (Quicksort)
• A key step in the Quicksort algorithm is partitioning the array
– choose some (any) number q in the array to use as a pivot
– partition the array into three parts:
q
numbers less
than q
numbers greater than
or equal to q
q

Partitioning
 Choose an array value (say, the first/ last) to use as the pivot
 Starting from the left end, find the first element that is greater
than or equal to the pivot
 Searching backward from the right end, find the first element
that is less than the pivot
 Interchange (swap) these two elements
 Repeat, searching from where we left off, until done

Another way: Partition method
int partition(int a[], int left, int right) {
int p = a[left], l = left + 1, r = right;
int temp;
while (l < r) {
while (l < right && a[l] < p) l++;
while (r > left && a[r] >= p) r--;
if (l < r) {
temp = a[l];
a[l] = a[r];
a[r] = temp;
}
}
a[left] = a[r];
a[r] = p;
return r;
}
right moves to the left until
value that should be to left
of pivot...
left moves to the right until
value that should be to right
of pivot...

Quicksort method
void quicksort(int a[], int left, int right) {
if (left < right) {
int p = partition(a, left, right);
quicksort(a, left, p - 1);
quicksort(a, p + 1, right);
}
}

6 5 9 12 3 4
0 1 2 3 4 5
quickSort(a, 0, 5 )

quickSort(a,0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
pivot= ?
Partition Initialization...

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
pivot=6

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left
pivot=6

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
of pivot...

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
of pivot...

quickSort(a, 0, 5)
partition(a, 0, 5)
pivot=6
6 5 9 12 3 4
0 1 2 3 4 5
left right

quickSort(a, 0, 5)
6 5 9 12 3 4
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
swap arr[left] and arr[right]

quickSort(a, 0, 5)
6 5 4 12 3 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
repeat right/left scan
UNTIL left & right cross

quickSort(a, 0, 5)
6 5 4 12 3 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
of pivot...

quickSort(arr,0,5)
6 5 4 12 3 9
0 1 2 3 4 5
partition(arr,0,5)
left right
pivot=6

quickSort(a, 0, 5)
6 5 4 12 3 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
of pivot...

quickSort(a, 0, 5)
6 5 4 12 3 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
repeat right/left scan
UNTIL left & right cross

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left right
pivot=6
of pivot...

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left
right
pivot=6

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left
right
pivot=6
right & left CROSS!!!

quickSort(a, 0, 5)
6 5 4 3 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left
right
pivot=6
1 - Swap pivot and arr[right]

quickSort(a, 0, 5)
3 5 4 6 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left
right
pivot=6

quickSort(a, 0, 5)
3 5 4 6 12 9
0 1 2 3 4 5
partition(a, 0, 5)
left
right
pivot=6
2 - Return new location of pivot to caller
return 3

quickSort(a, 0, 5) 3 5 4 6 12 9
0 1 2 3 4 5
Recursive calls to quickSort()
using partitioned array...
pivot
position

quickSort(a, 0, 5)
3 5 4 6 12 9
0 1 2 3 4 5
quickSort(a, 0, 2)
3 5 4
0 1 2
quickSort(a, 4, 5)
12 9
4 5
6
3

quickSort(a, 0, 5)
quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
quickSort(a, 0, 5)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left
quickSort(a, 0, 5)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left right
quickSort(a, 0, 5)

of pivot...
quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left
quickSort(a, 0, 5)
right

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left right
quickSort(a, 0, 5)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left
right
quickSort(a, 0, 5)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
partition(a, 0, 2)
left
right
2 - Return new location of pivot to caller
return 0
quickSort(a, 0, 5)

quickSort(a, 0, 2)
3 5 4 6
0 1 2 3
quickSort(a, 4, 5)
12 9
4 5
quickSort(a, 0, 5)
Recursive calls to quickSort()
using partitioned array...

quickSort(a, 0, 2) quickSort(a, 4, 5)
12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
Base case triggered...
halting recursion.

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
1 2
Base Case: Return
6
3

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
partition(a, 1, 2)

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
partition(a, 1, 2)
left

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
partition(a, 1, 2)
left right
of pivot...

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
partition(a, 1, 2)
left
right

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
6
1 2
3
partition(a, 1, 2)
left
right
right & left CROSS!

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
5 4
1 2
partition(a, 1, 2)
left
right
right & left CROSS!
1- swap pivot and arr[right]
6
3

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3
partition(a, 1, 2)
left
right
right & left CROSS!
1- swap pivot and a[right]

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3
partition(a, 1, 2)
left
right
right & left CROSS!
1- swap pivot and arr[right]
2 – return new position of pivot
return 2

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3
Base Case: Return

12 9
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3
partition(a, 4, 5)

9 12
4 5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3
partition(a, 4, 5)

9
12
4
5
quickSort(a, 0, 5)
quickSort(a, 0, 0)
3
0
quickSort(a, 1, 2)
4 5
6
1 2
3 quickSort(a, 4, 4)
Base Case: Return

Example for Quick Sort
R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 left right
26 5 37 1 61 11 59 15 48 19 0 9
11 5 19 1 15 26 59 61 48 37 0 4
1 5 11 19 15 26 59 61 48 37 0 1
1 5 11 15 19 26 59 61 48 37 3 4
1 5 11 15 19 26 48 37 59 61 6 9
1 5 11 15 19 26 37 48 59 61 6 7
1 5 11 15 19 26 37 48 59 61 9 9
1 5 11 15 19 26 37 48 59 61
{ }
{ } { }
{ } { } { }
{ }
{ }
{
} {
}

Quicksort Analysis
 Assume that keys are random, uniformly distributed.
 Best case running time: O(n log2n)
 Worst case running time?
 Recursion:
 Partition splits array in two sub-arrays:
 one sub-array of size 0
 the other sub-array of size n-1
 Quicksort each sub-array
 Depth of recursion tree? O(n)
 Number of accesses per partition? O(n)

Quicksort Analysis
 Assume that keys are random, uniformly distributed.
 Best case running time: O(n log2n)
 Worst case running time: O(n2)!!!
 What can we do to avoid worst case?

Improved Pivot Selection
Pick median value of three elements from data array:
data[0], data[n/2], and data[n-1].
Use this median value as pivot.

The Heap Data Structure
 Def: A heap is a nearly complete binary tree with the following
two properties:
 Structural property: all levels are full, except possibly the last
one, which is filled from left to right
 Order (heap) property: for any node x
Parent(x) ≥ x
Heap
5
7
8
4
2
From the heap property, it
follows that:
“The root is the maximum
element of the heap!”
A heap is a binary tree that is filled in order

Array Representation of Heaps
 A heap can be stored as an array A.
 Root of tree is A[1]
 Left child of A[i] = A[2*i]
 Right child of A[i] = A[2*i + 1]
 Parent of A[i] = A[ i/2 ]
 Heapsize[A] ≤ length[A]
 The elements in the subarray
A[(n/2+1) .. n] are leaves

Heap Types
 Max-heaps (largest element at root), have the max-heap
property:
 for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
 Min-heaps (smallest element at root), have the min-heap
property:
 for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]

Adding/Deleting Nodes
 New nodes are always inserted at the bottom level (left to right)
 Nodes are removed from the bottom level (right to left)

Operations on Heaps
 Maintain/Restore the max-heap property
 Max-Heapify
 Create a max-heap from an unordered array
 Build-Max-Heap
 Sort an array in place
 Heapsort
 Priority queues

Maintaining the Heap Property
 Suppose a node is smaller than a child
 Left and Right subtrees of i are max-heaps
 To eliminate the violation:
 Exchange with larger child
 Move down the tree
 Continue until node is not smaller than
children

Example
Max-Heapify(A, 2, 10)
A[2] violates the heap property
A[2]  A[4]
A[4] violates the heap property
A[4]  A[9]
Heap property restored

Maintaining the Heap Property
 Assumptions:
 Left and Right
subtrees of i are
max-heaps
 A[i] may be smaller
than its children
Alg: Max-Heapify(A, i, n)
1. l ← left(i)
2. r ← right(i)
3. if l ≤ n and A[l] > A[i]
4. then largest ←l
5. else largest ←i
6. if r ≤ n and A[r] > A[largest]
7. then largest ←r
8. if largest  i
9. then exchange A[i] ↔ A[largest]
10. Max-Heapify(A, largest, n)

Max-Heapify Running Time
 Intuitively:
 It traces a path from the root to a leaf (longest path length: h)
 At each level, it makes exactly two comparisons.
 Total number of comparisons is 2h
 Running time is O(h) or O(log2n)
 Running time of Max-Heapify is O(log2n)
 Can be written in terms of the height of the heap, as being O(h)
 Since the height of the heap is log2n

Building a Heap
Alg: Build-Max-Heap(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do Max-Heapify(A, i, n)
 Convert an array A[1 … n] into a max-heap (n = length[A])
 The elements in the subarray A[(n/2+1) .. n] are leaves
 Apply MAX-HEAPIFY on elements between 1 and n/2
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
7
8
14
10
9
16
2
3
1
4
A:

Example: A 7
8
14
10
9
16
2
3
1
4
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
10
9 3
1
2 3
4 5 6 7
8 9 10
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
16
7
1
4
10
9 3
1
2 3
4 5 6 7
8 9 10
8
2 4
14
7
1
16
10
9 3
1
2 3
4 5 6 7
8 9 10
i = 5 i = 4 i = 3
i = 2 i = 1

Running Time of Build-Max-Heap
 Running time: O(nlog2n)
 This is not an asymptotically tight upper bound
Alg: Build-Max-Heap(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do Max-Heapify(A, i, n)
O(log2n)
O(n)

Heapsort
 Goal:
 Sort an array using heap representations
 Idea:
 Build a max-heap from the array
 Swap the root (the maximum element) with the last element
in the array
 “Discard” this last node by decreasing the heap size
 Call Max-Heapify on the new root
 Repeat this process until only one node remains

Example: A=[7, 4, 3, 1, 2]
Max-Heapify(A, 1, 4) Max-Heapify(A, 1, 3) Max-Heapify(A, 1, 2)
Max-Heapify(A, 1, 1)

Alg: Heapsort(A)
1. Build-Max-Heap(A)
2. for i ← length[A] downto 2
3. do exchange A[1] ↔ A[i]
4. Max-Heapify(A, 1, i - 1)
 Running time: O(nlog2n) --- Can be shown to be O(nlog2n)
O(n)
O(log2n)
n-1 times

Radix sort
 If integers are in a larger range then do bucket sort on each digit
 Start by sorting with the low-order digit using a STABLE bucket
sort.
 Then, do the next-lowest, and so on

Radix Sort
 Extra information: every integer can be represented by at most k digits
 d1d2…dk where di are digits in base r
 d1: most significant digit
 dk: least significant digit

Radix Sort
 Algorithm
 sort by the least significant digit first (counting sort)
=> Numbers with the same digit go to same bin
 reorder all the numbers: the numbers in bin 0 precede the numbers
in bin 1, which precede the numbers in bin 2, and so on
 sort by the next least significant digit
 continue this process until the numbers have been sorted on all k
digits

Radix sort characteristics
 Each sorting step can be performed via bucket sort, and is thus
O(N).
 If the numbers are all b bits long, then there are b sorting
steps.
 Hence, radix sort is O(bN).

Radix sort
// Function to get the largest element from an array
int getMax(int arr[], int n) {
int max = arr[0];
for (int i = 1; i < n; i++)
if (arr[i] > max)
max = arr[i];
return max;
}

Radix sort
// Counting sort
void countingSort(int arr[], int n, int place) {
int i;
int output[n + 1];
int count[10] = {0};
// Calculate count of elements
for(i=0; i<n; i++)
count[(arr[i] / place) % 10]++;
// Calculate cummulative count
for(i=1; i<10; i++)
count[i] += count[i-1];
// Place the elements in sorted order
for(i=n-1; i>=0; i--) {
output[count[(arr[i] / place) % 10] - 1] = arr[i];
count[(arr[i] / place) % 10]--;
}
for(i=0; i<n; i++)
arr[i] = output[i];
}

Radix sort
// Main function to implement radix sort
void radixsort(int arr[], int n) {
int max = getMax(arr, n); // Get maximum element
// Apply counting sort to sort elements based on place value.
for (int place=1; max / place > 0; place *= 10)
countingSort(arr, n, place);
}
// Print an array
void printArray(int arr[], int n) {
for (int i=0; i<n; ++i)
printf("%d ", arr[i]);
printf("n");
}

Radix sort
// Code
int main() {
int arr[] = {121, 432, 564, 23, 1, 45, 788};
int n = sizeof(arr) / sizeof(arr[0]);
radixsort(arr, n);
printArray(arr, n);
}

Radix Sort
 Least-significant-digit-first
Example: 275, 087, 426, 061, 509, 170, 677, 503

Radix Sort
 Increasing the base r decreases the number of passes
 Running time
 k passes over the numbers (i.e. k counting sorts, with range being 0...r)
 each pass takes O(n+r)
 total: O(nk+rk)
 r and k are constants: O(n)

Sorting and Hashing Algorithm full pdfs.

More Related Content

Similar to Sorting and Hashing Algorithm full pdfs.

Recently uploaded

Sorting and Hashing Algorithm full pdfs.