KEMBAR78
L2_DatabAlgorithm Basics with Design & Analysis.pptx
Lecture 02
Divide and Conquer (BinarySearch &
Mergesort)
CSE373: Design and Analysis of Algorithms
A motivating Example of D&C Algorithm
Binary Search (recursive)
// Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1
Algorithm BinarySearch(A, first, last, x)
if last ≥ first then
mid = first + (last - first)/2
// If the element is present at the middle itself
if A[mid] = x then
return mid
// If element is smaller than mid, then it can only be present in left sub-array
else if A[mid] > x then
return BinarySearch(A, first, mid-1, x)
// Otherwise the element can only be present in the right sub-array
else
return BinarySearch(A, mid+1, last, x);
Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A
Retrieving an Item from Sorted List
• Find 84
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
Retrieving an Item from Sorted List
• Find 84
• Step 1
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
Retrieving an Item from Sorted List
• Find 84
• Step 1
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(0+14)/2
Retrieving an Item from Sorted List
• Find 84
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(0+14)/2
Retrieving an Item from Sorted List
• Find 84
• Step 2
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
Retrieving an Item from Sorted List
• Find 84
• Step 2
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(8+14)/2
Retrieving an Item from Sorted List
• Find 84
• Step 2
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(8+14)/2
Retrieving an Item from Sorted List
• Find 84
• Step 3
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
Retrieving an Item from Sorted List
• Find 84
• Step 3
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(8+10)/2
Retrieving an Item from Sorted List
• Find 84
• Step 3
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(8+10)/2
Retrieving an Item from Sorted List
• Find 84
• Step 4
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
Retrieving an Item from Sorted List
• Find 84
• Step 4
• 84 found at the midpoint
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
first last
mid
=(10+10)/2
Binary Search (recursive) Algorithm
// Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1
Algorithm BinarySearch(A, p, q, x)
if last ≥ first then
mid  (p+q)/2
// If the element is present at the middle itself
if A[mid] = x then
return mid
// If element is smaller than mid, then it can only be present in left sub-array
if A[mid] > x then
return BinarySearch(A, p, mid-1, x)
// Otherwise the element can only be present in the right sub-array
else
return BinarySearch(A, mid+1, q, x)
return -1 // We reach here when element is not present in A
Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A
Time: Θ(lg n), why?
Divide and Conquer (D&C)
• In general, has 3 steps:
– Divide the problem into independent sub-
problems that are similar to the original but
smaller in size
– Conquer the sub-problems by solving them
recursively. If they are small enough, just solve
them in a straightforward manner.
– Combine the solutions to create a solution to the
original problem (this step may be empty)
D&C Algorithm Example: Binary Search
Searching Problem: Search for item in a sorted sequence A of n elements
Divide: Divide the n-element input array into two subarray of ≈ n/2 elements
each:
m  (p+q)/2
Conquer: Search either of the subarrays recursively by calling BinarySearch on
the appropriate subarray:
if A[m] > x then
return BinarySearch(A, p, m-1, x)
else
return BinarySearch(A, m+1, q, x)
Combine: Nothing to be done
D&C Example: Merge Sort (Section 2.3)
Sorting Problem: Sort a sequence A of n elements into non-decreasing order:
MergeSort (A[p..r]) //sort A[p..r]
Divide: Divide the n-element input array into two subarray of ≈ n/2 elements
each [easy]:
q  (p+r)/2
Conquer: Sort the two subsequences recursively by calling merge sort on
each subsequence [easy]:
MergeSort (A[p .. q]) // A[p .. q] becomes sorted after this call
MergeSort (A[q+1 .. r]) //A[q+1..r] becomes sorted after this call
Combine: Merge the two sorted subsequences to produce the sorted
sequence [how?]
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Sorted Sorted
Unsorted
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15 18
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15 18
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15 18 43
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15 18 43
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
1 6 9 15 18 43 56 62
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
1 6 9 15 18 43 56 62
1 6 9 15 18 43 56 62
Left half
Right half
Minimum between first elements in both halves
Merging
Merging two sorted subsequeces
1 6 9 15 18 43 56 62
Merging two sorted subsequeces
Merge(A, p, q, r)
1 n1  q – p + 1
2 n2  r – q
3 for i  1 to n1
4 do L[i]  A[p + i – 1]
5 for j  1 to n2
6 do R[j]  A[q + j]
7 L[n1+1]  
8 R[n2+1]  
9 i  1
10 j  1
11 for k p to r
12 do if L[i]  R[j]
13 then A[k]  L[i]
14 i  i + 1
15 else A[k]  R[j]
16 j  j + 1
Sentinels, to avoid having to
check if either subarray is
fully copied at each step.
Input: Array containing
sorted subarrays A[p..q] and
A[q+1..r].
Output: Merged sorted
subarray in A[p..r].
Time complexity of Merge
Merge(A, p, q, r) //Let r-p+1 = n
1 n1  q – p + 1 //Θ(1)
2 n2  r – q //Θ(1)
3 for i  1 to n1 //Θ(q-p+1)
4 do L[i]  A[p + i – 1]
5 for j  1 to n2 //Θ(r-q)
6 do R[j]  A[q + j]
7 L[n1+1]  
8 R[n2+1]  
9 i  1
10 j  1
11 for k p to r //Θ(r-p+1) = Θ(n)
12 do if L[i]  R[j]
13 then A[k]  L[i]
14 i  i + 1
15 else A[k]  R[j]
16 j  j + 1
//Total time: Θ(n)
Input: Array containing
sorted subarrays A[p..q] and
A[q+1..r].
Output: Merged sorted
subarray in A[p..r].
Merge Sort (recursive/D&C version)
MergeSort (A, p, r) // sort A[p..r] via merge sort
1 if p < r
2 then q  (p+r)/2 //divide
3 MergeSort (A, p, q) //conquer
4 MergeSort (A, q+1, r) //conquer
5 Merge (A, p, q, r) //combine: merge A[p..q] with A[q+1..r]
Initial Call: MergeSort(A, 1, n)
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98
Merge
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98
23
Merge
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98
23 98
Merge
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
23 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
23 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
14
Merge
23 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
45
Merge
23 98 14
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
98 45
14
23
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
98 14
14
23 45
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
23 14
14 23
98 45
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
23 98 45
14
14 23 45
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
Merge
23 98 45
14
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
23 98 45
14
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6
23 98 45
14
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6
Merge
23 98 45
14
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6
6
Merge
23 98 45
14
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6
67
Merge
23 98 45
14 6
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
23 98 45
14 67
6
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
33
23 98 45
14 67
6
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
42
23 98 45
14 67
6 33
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 45 98
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 6 42
33
14 23 45 98 6
67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 6 33
14 23 45 98 6 33
67 42
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 6 42
33
14 23 45 98 6 33 42
67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 45 98 6 33 42 67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
23 45 98 33 42 67
14 6
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
23 45 98 6 42 67
6
14 33
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 45 98 6 42 67
6 14
23 33
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 98 6 42 67
6 14 23
45 33
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 98 6 33 67
6 14 23 33
45 42
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 98 6 33 42
6 14 23 33 42
45 67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 45 6 33 42
6 14 23 33 42 45
98 67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67
67
45
23 14 6 33
98 42
67
45
23 14 6 33
98 42
45
23 14
98
23
98 45 14
67
6 33 42
67
6 33 42
Merge
23 98 45
14 67
6 42
33
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
67
45
23 14 6 33
98 42
6 14 23 33 42 45 67 98
Analysis of Merge Sort
Statement Cost
MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n) [let]
1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r)
Analysis of Merge Sort
Statement Cost (time)
So T(n) = (1) ; when n = 1, and 2T(n/2) + (n) + 2(1)
; when n > 1
It’s a recurrence relation. Equivalent recurrence relation:
T(n) = (1) ; when n = 1, and 2T(n/2) + (n) ;
when n > 1
Equivalent recurrence relation:
T(n) = c if n = 1
= 2T(n/2) + cn if n > 1
MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n), to sort n elements
1 if p < r (1)
2 then q  (p+r)/2 //q ≈ n/2 (1)
3 MergeSort (A, p, q) T(n/2), to sort n/2
elements
4 MergeSort (A, q+1, r) T(n/2), to sort n/2 elements
5 Merge (A, p, q, r) (n)
Recurrence Relations (RR)
Equation or an inequality that characterizes a function by
its values on smaller inputs.
Recurrence relations arise when we analyze the running
time of iterative or recursive algorithms.
Ex: Divide and Conquer algorithms typically have r.r. of the form:
T(n) = (1) if n  c
T(n) = a T(n/b) + D(n) otherwise
Mthods to solve recurrence relations
•Substitution Method.
•Recursion-tree Method.
Substitution Method
Illustration of guessing solution of a r.r. (representing time
complexity of MergeSort) via substitution method:
T(n) = 2T(n/2) + cn
= 2(2T(n/4)+cn/2) + cn = 22
T(n/22
) + 2cn
= 22
(2T(n/8)+cn/4) + 2cn = 23
T(n/23
) + 3cn
…
= 2k
T(n/2k
) + kcn [guess the pattern from previous equations]
Let 2k
= n (so that we get T(n/2k
) = T(1) which is known to us)
⸫ T(n) = n T(n/n) + (lg n) cn
= n T(1) + (lg n) cn
= n T(1) + cn lg n
= cn + (lg n) cn which is (n lg n)
Recursion-tree Method
• Recursion trees can also be used to solve r.r.
Recursion Trees
•Show successive expansions of recurrences using trees.
•Keep track of the time spent on the subproblems of a divide and
conquer algorithm.
•Help organize the algebraic bookkeeping necessary to solve a
recurrence.
Recursion Tree – Example
Running time of Merge Sort:
T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
Rewrite the recurrence as
T(n) = c if n = 1
T(n) = 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort
For the original problem,
we have a cost of cn, plus
two subproblems each of
size (n/2) and running time
T(n/2).
cn
T(n/2) T(n/2)
Each of the size n/2 problems has
a cost of cn/2 plus two
subproblems, each costing T(n/4).
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of divide
and merge.
Cost of sorting
subproblems.
T(n) = 2T(n/2) + cn
Þ cn: parent node with
2 children: each T(n/2)
T(n/2) = 2T(n/4) + cn/2
Þ cn/2: parent node with
2 children: each T(n/4)
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
c c c c
c c
lg n
cn
cn
cn
cn
Total : cnlgn+cn
Counting Inversions Problem
• Given two ranked list of items, how can you compare
these two lists?
• Application: Recommendation systems try to match your
preferences (for books, movies, restaurants, etc.) with
those of other people in the internet
• Idea: represent one ranked list by <1,2, …, n> and another
by a permutation of the first list. Then count the number of
inversions (i.e. out-of-order pairs in the second list.
Merging & Counting Inversions
MergeAndCount(A, p, q, r)
1 n1  q – p + 1
2 n2  r – q
3 for i  1 to n1
4 do L[i]  A[p + i – 1]
5 for j  1 to n2
6 do R[j]  A[q + j]
7 L[n1+1]  
8 R[n2+1]  
9 i  1
10 j  1
11 cnt  0
12 for k p to r
13 do if L[i]  R[j]
14 then A[k]  L[i]
15 i  i + 1
16 else A[k]  R[j]
17 j  j + 1
18 cnt  cnt + n1-i+1
19 return cnt
Input: Array containing
sorted subarrays A[p..q] and
A[q+1..r].
Output: Merged sorted
subarray in A[p..r].
Counting Inversions
Statement Cost
So T(n) = (1) when n = 1, and 2T(n/2)
+ (n) when n > 1
CountInversions(A, p, r)
1 if p < r
2 then q  (p+r)/2
3 x  CountInversions (A, p, q)
4 y  CountInversions(A, q+1, r)
5 z  MergeAndCount(A, p, q, r)
6 return x+y+z

L2_DatabAlgorithm Basics with Design & Analysis.pptx

  • 1.
    Lecture 02 Divide andConquer (BinarySearch & Mergesort) CSE373: Design and Analysis of Algorithms
  • 2.
    A motivating Exampleof D&C Algorithm Binary Search (recursive) // Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1 Algorithm BinarySearch(A, first, last, x) if last ≥ first then mid = first + (last - first)/2 // If the element is present at the middle itself if A[mid] = x then return mid // If element is smaller than mid, then it can only be present in left sub-array else if A[mid] > x then return BinarySearch(A, first, mid-1, x) // Otherwise the element can only be present in the right sub-array else return BinarySearch(A, mid+1, last, x); Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A
  • 3.
    Retrieving an Itemfrom Sorted List • Find 84 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6
  • 4.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 1 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last
  • 5.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 1 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(0+14)/2
  • 6.
    Retrieving an Itemfrom Sorted List • Find 84 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(0+14)/2
  • 7.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 2 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last
  • 8.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 2 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(8+14)/2
  • 9.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 2 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(8+14)/2
  • 10.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 3 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last
  • 11.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 3 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(8+10)/2
  • 12.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 3 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(8+10)/2
  • 13.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 4 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last
  • 14.
    Retrieving an Itemfrom Sorted List • Find 84 • Step 4 • 84 found at the midpoint 8 2 1 3 4 6 5 7 10 9 11 12 14 13 0 64 14 13 25 33 51 43 53 84 72 93 95 97 96 6 first last mid =(10+10)/2
  • 15.
    Binary Search (recursive)Algorithm // Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1 Algorithm BinarySearch(A, p, q, x) if last ≥ first then mid  (p+q)/2 // If the element is present at the middle itself if A[mid] = x then return mid // If element is smaller than mid, then it can only be present in left sub-array if A[mid] > x then return BinarySearch(A, p, mid-1, x) // Otherwise the element can only be present in the right sub-array else return BinarySearch(A, mid+1, q, x) return -1 // We reach here when element is not present in A Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A Time: Θ(lg n), why?
  • 16.
    Divide and Conquer(D&C) • In general, has 3 steps: – Divide the problem into independent sub- problems that are similar to the original but smaller in size – Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner. – Combine the solutions to create a solution to the original problem (this step may be empty)
  • 17.
    D&C Algorithm Example:Binary Search Searching Problem: Search for item in a sorted sequence A of n elements Divide: Divide the n-element input array into two subarray of ≈ n/2 elements each: m  (p+q)/2 Conquer: Search either of the subarrays recursively by calling BinarySearch on the appropriate subarray: if A[m] > x then return BinarySearch(A, p, m-1, x) else return BinarySearch(A, m+1, q, x) Combine: Nothing to be done
  • 18.
    D&C Example: MergeSort (Section 2.3) Sorting Problem: Sort a sequence A of n elements into non-decreasing order: MergeSort (A[p..r]) //sort A[p..r] Divide: Divide the n-element input array into two subarray of ≈ n/2 elements each [easy]: q  (p+r)/2 Conquer: Sort the two subsequences recursively by calling merge sort on each subsequence [easy]: MergeSort (A[p .. q]) // A[p .. q] becomes sorted after this call MergeSort (A[q+1 .. r]) //A[q+1..r] becomes sorted after this call Combine: Merge the two sorted subsequences to produce the sorted sequence [how?]
  • 19.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43
  • 20.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 Sorted Sorted Unsorted
  • 21.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 Merging
  • 22.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 Left half Right half Minimum between first elements in both halves Merging
  • 23.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 Left half Right half Minimum between first elements in both halves Merging
  • 24.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 Left half Right half Minimum between first elements in both halves Merging
  • 25.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 Left half Right half Minimum between first elements in both halves Merging
  • 26.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 Left half Right half Minimum between first elements in both halves Merging
  • 27.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 Left half Right half Minimum between first elements in both halves Merging
  • 28.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 Left half Right half Minimum between first elements in both halves Merging
  • 29.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 Left half Right half Minimum between first elements in both halves Merging
  • 30.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 Left half Right half Minimum between first elements in both halves Merging
  • 31.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 18 Left half Right half Minimum between first elements in both halves Merging
  • 32.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 18 Left half Right half Minimum between first elements in both halves Merging
  • 33.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 18 43 Left half Right half Minimum between first elements in both halves Merging
  • 34.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 18 43 Left half Right half Minimum between first elements in both halves Merging
  • 35.
    Merging two sortedsubsequeces 6 18 56 62 1 9 15 43 1 6 9 15 18 43 56 62 Left half Right half Minimum between first elements in both halves Merging
  • 36.
    Merging two sortedsubsequeces 1 6 9 15 18 43 56 62 1 6 9 15 18 43 56 62 Left half Right half Minimum between first elements in both halves Merging
  • 37.
    Merging two sortedsubsequeces 1 6 9 15 18 43 56 62
  • 38.
    Merging two sortedsubsequeces Merge(A, p, q, r) 1 n1  q – p + 1 2 n2  r – q 3 for i  1 to n1 4 do L[i]  A[p + i – 1] 5 for j  1 to n2 6 do R[j]  A[q + j] 7 L[n1+1]   8 R[n2+1]   9 i  1 10 j  1 11 for k p to r 12 do if L[i]  R[j] 13 then A[k]  L[i] 14 i  i + 1 15 else A[k]  R[j] 16 j  j + 1 Sentinels, to avoid having to check if either subarray is fully copied at each step. Input: Array containing sorted subarrays A[p..q] and A[q+1..r]. Output: Merged sorted subarray in A[p..r].
  • 39.
    Time complexity ofMerge Merge(A, p, q, r) //Let r-p+1 = n 1 n1  q – p + 1 //Θ(1) 2 n2  r – q //Θ(1) 3 for i  1 to n1 //Θ(q-p+1) 4 do L[i]  A[p + i – 1] 5 for j  1 to n2 //Θ(r-q) 6 do R[j]  A[q + j] 7 L[n1+1]   8 R[n2+1]   9 i  1 10 j  1 11 for k p to r //Θ(r-p+1) = Θ(n) 12 do if L[i]  R[j] 13 then A[k]  L[i] 14 i  i + 1 15 else A[k]  R[j] 16 j  j + 1 //Total time: Θ(n) Input: Array containing sorted subarrays A[p..q] and A[q+1..r]. Output: Merged sorted subarray in A[p..r].
  • 40.
    Merge Sort (recursive/D&Cversion) MergeSort (A, p, r) // sort A[p..r] via merge sort 1 if p < r 2 then q  (p+r)/2 //divide 3 MergeSort (A, p, q) //conquer 4 MergeSort (A, q+1, r) //conquer 5 Merge (A, p, q, r) //combine: merge A[p..q] with A[q+1..r] Initial Call: MergeSort(A, 1, n)
  • 41.
    67 45 23 14 633 98 42
  • 42.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42
  • 43.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98
  • 44.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98
  • 45.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 Merge
  • 46.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 23 Merge
  • 47.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 23 98 Merge
  • 48.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 23 98
  • 49.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 23 98
  • 50.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 14 Merge 23 98
  • 51.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 45 Merge 23 98 14
  • 52.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 98 45 14 23
  • 53.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 98 14 14 23 45
  • 54.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 23 14 14 23 98 45
  • 55.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 23 98 45 14 14 23 45
  • 56.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 Merge 23 98 45 14 14 23 45 98
  • 57.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 23 98 45 14 14 23 45 98
  • 58.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 23 98 45 14 14 23 45 98
  • 59.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 Merge 23 98 45 14 14 23 45 98
  • 60.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 6 Merge 23 98 45 14 14 23 45 98
  • 61.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 67 Merge 23 98 45 14 6 14 23 45 98
  • 62.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 23 98 45 14 67 6 14 23 45 98
  • 63.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 14 23 45 98
  • 64.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 33 23 98 45 14 67 6 14 23 45 98
  • 65.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 42 23 98 45 14 67 6 33 14 23 45 98
  • 66.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 45 98
  • 67.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 6 42 33 14 23 45 98 6 67
  • 68.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 6 33 14 23 45 98 6 33 67 42
  • 69.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 6 42 33 14 23 45 98 6 33 42 67
  • 70.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 45 98 6 33 42 67
  • 71.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 23 45 98 33 42 67 14 6
  • 72.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 23 45 98 6 42 67 6 14 33
  • 73.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 45 98 6 42 67 6 14 23 33
  • 74.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 98 6 42 67 6 14 23 45 33
  • 75.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 98 6 33 67 6 14 23 33 45 42
  • 76.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 98 6 33 42 6 14 23 33 42 45 67
  • 77.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 45 6 33 42 6 14 23 33 42 45 98 67
  • 78.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67
  • 79.
    67 45 23 14 633 98 42 67 45 23 14 6 33 98 42 45 23 14 98 23 98 45 14 67 6 33 42 67 6 33 42 Merge 23 98 45 14 67 6 42 33 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 98
  • 80.
    67 45 23 14 633 98 42 6 14 23 33 42 45 67 98
  • 81.
    Analysis of MergeSort Statement Cost MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n) [let] 1 if p < r 2 then q  (p+r)/2 3 MergeSort (A, p, q) 4 MergeSort (A, q+1, r) 5 Merge (A, p, q, r)
  • 82.
    Analysis of MergeSort Statement Cost (time) So T(n) = (1) ; when n = 1, and 2T(n/2) + (n) + 2(1) ; when n > 1 It’s a recurrence relation. Equivalent recurrence relation: T(n) = (1) ; when n = 1, and 2T(n/2) + (n) ; when n > 1 Equivalent recurrence relation: T(n) = c if n = 1 = 2T(n/2) + cn if n > 1 MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n), to sort n elements 1 if p < r (1) 2 then q  (p+r)/2 //q ≈ n/2 (1) 3 MergeSort (A, p, q) T(n/2), to sort n/2 elements 4 MergeSort (A, q+1, r) T(n/2), to sort n/2 elements 5 Merge (A, p, q, r) (n)
  • 83.
    Recurrence Relations (RR) Equationor an inequality that characterizes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer algorithms typically have r.r. of the form: T(n) = (1) if n  c T(n) = a T(n/b) + D(n) otherwise Mthods to solve recurrence relations •Substitution Method. •Recursion-tree Method.
  • 84.
    Substitution Method Illustration ofguessing solution of a r.r. (representing time complexity of MergeSort) via substitution method: T(n) = 2T(n/2) + cn = 2(2T(n/4)+cn/2) + cn = 22 T(n/22 ) + 2cn = 22 (2T(n/8)+cn/4) + 2cn = 23 T(n/23 ) + 3cn … = 2k T(n/2k ) + kcn [guess the pattern from previous equations] Let 2k = n (so that we get T(n/2k ) = T(1) which is known to us) ⸫ T(n) = n T(n/n) + (lg n) cn = n T(1) + (lg n) cn = n T(1) + cn lg n = cn + (lg n) cn which is (n lg n)
  • 85.
    Recursion-tree Method • Recursiontrees can also be used to solve r.r. Recursion Trees •Show successive expansions of recurrences using trees. •Keep track of the time spent on the subproblems of a divide and conquer algorithm. •Help organize the algebraic bookkeeping necessary to solve a recurrence.
  • 86.
    Recursion Tree –Example Running time of Merge Sort: T(n) = (1) if n = 1 T(n) = 2T(n/2) + (n) if n > 1 Rewrite the recurrence as T(n) = c if n = 1 T(n) = 2T(n/2) + cn if n > 1 c > 0: Running time for the base case and time per array element for the divide and combine steps.
  • 87.
    Recursion Tree forMerge Sort For the original problem, we have a cost of cn, plus two subproblems each of size (n/2) and running time T(n/2). cn T(n/2) T(n/2) Each of the size n/2 problems has a cost of cn/2 plus two subproblems, each costing T(n/4). cn cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4) Cost of divide and merge. Cost of sorting subproblems. T(n) = 2T(n/2) + cn Þ cn: parent node with 2 children: each T(n/2) T(n/2) = 2T(n/4) + cn/2 Þ cn/2: parent node with 2 children: each T(n/4)
  • 88.
    Recursion Tree forMerge Sort Continue expanding until the problem size reduces to 1. cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 c c c c c c lg n cn cn cn cn Total : cnlgn+cn
  • 89.
    Counting Inversions Problem •Given two ranked list of items, how can you compare these two lists? • Application: Recommendation systems try to match your preferences (for books, movies, restaurants, etc.) with those of other people in the internet • Idea: represent one ranked list by <1,2, …, n> and another by a permutation of the first list. Then count the number of inversions (i.e. out-of-order pairs in the second list.
  • 90.
    Merging & CountingInversions MergeAndCount(A, p, q, r) 1 n1  q – p + 1 2 n2  r – q 3 for i  1 to n1 4 do L[i]  A[p + i – 1] 5 for j  1 to n2 6 do R[j]  A[q + j] 7 L[n1+1]   8 R[n2+1]   9 i  1 10 j  1 11 cnt  0 12 for k p to r 13 do if L[i]  R[j] 14 then A[k]  L[i] 15 i  i + 1 16 else A[k]  R[j] 17 j  j + 1 18 cnt  cnt + n1-i+1 19 return cnt Input: Array containing sorted subarrays A[p..q] and A[q+1..r]. Output: Merged sorted subarray in A[p..r].
  • 91.
    Counting Inversions Statement Cost SoT(n) = (1) when n = 1, and 2T(n/2) + (n) when n > 1 CountInversions(A, p, r) 1 if p < r 2 then q  (p+r)/2 3 x  CountInversions (A, p, q) 4 y  CountInversions(A, q+1, r) 5 z  MergeAndCount(A, p, q, r) 6 return x+y+z

Editor's Notes

  • #84 Talk about how mathematical induction works.