Data structure and problem solving ch03.ppt

Chapter 3
Introduction to Algorithms

2
Outline
• Sorting Algorithms
 Selection Sort
• Search Algorithms
 Sequential Search
 Binary Search
• Analysis of Algorithms
 Estimating the Efficiency
 Big-O Notation
 Running-Time Analysis
 Growth Rates of Functions
• Recursive Functions
 Factorials
 Towers of Hanoi

3
Selection Sort - 5 Element Array
• Pass 0:
• Scan the entire list from arr[0] to arr[4] and identify 20 at
index 1 as the smallest element.
• Exchange 20 with arr[0] = 50, the first element in the list.
Pass 0: Select 20 at index 1
Exchange arr[1] and arr[0]
50 40 75 35
pass = 0
20

4
• Pass 1:
• Scan the sublist 50, 40, 75, and 35.
• Exchange the smallest element 35 at index 4 with arr[1] = 50.
20 50 40 75
pass = 1
35

5
• Pass 2:
• Locate the smallest element in the sublist 40, 75, and 50.
20 35 75 50
pass = 2
40
N o exchange necessary

6
• Pass 3:
• Two elements remain to be sorted.
• Scan the sublist 75, 50 and exchange the smaller element with
arr[3].
• The exchange places 50 at index 4 in arr[3].
pass = 3
20 35 40 75 50

7
20 35 40
Sorted list
50 75

8
Selection Sort Algorithm
void selectionSort(int arr[], int n)
{
int smallIndex; // index of smallest element in the sublist
int pass, j;
int temp;
for (pass = 0; pass < n-1; pass++) // pass has the range 0 to n-2
{
// scan the sublist starting at index pass
smallIndex = pass;
// j traverses the sublist arr[pass+1] to arr[n-1]
for (j = pass+1; j < n; j++)
// if smaller element found, assign smallIndex to that position
if (arr[j] < arr[smallIndex])
smallIndex = j;
// if smallIndex and pass are not the same location,
// exchange the smallest item in the sublist with arr[pass]
if (smallIndex != pass)
{
temp = arr[pass];
arr[pass] = arr[smallIndex];
arr[smallIndex] = temp;
}
}
}

9
Search Algorithms
0 1 2 6 7 8
first last
Array arr
5
4
3
• Search algorithms start with a target value and employ some
strategy to visit the elements looking for a match.
 If target is found, the index of the matching element becomes the
return value.

10
6 4 2 9 5 10
index = seqSearch(arr, 0, 8, 3);
7
Index 0 1 2 3 4 5 6 7
3
target = 3
8
match at index = 5
return index 5
Search Algorithms - Sequential Search Algorithm
6 4 2 9 5 3 10 7
Index 0 1 2 3 4 5 6 7
target = 9
8
no match
return ind ex 8
index = seqS earch(arr, 0, 8, 9);

11
Search Algorithms - Sequential Search Algorithm
int seqSearch(const int arr[], int first, int last, int target)
{
int i = first;
// scan indices in the range first <= I < last;
// test for a match or index out of range.
while(i != last && arr[i] != target)
i++;
return i; // i is index of match or i = last if no match
}

12
Search Algorithms – Binary Search
Binary search is more efficient if the array is sorted.
Given a target value, the algorithm begins the search by selecting
the midpoint in the list.
Case 1:
A match occurs. The search is complete and mid is the index that
locates the target.
if (midValue == target)
// found match
return mid;
mid
first
target
Case 1: target = midvalue
Search is done
last-1 last

13
Case 2:
The value of target is less than midvalue and the search must
continue in the lower sublist. Reposition the index last to the
end of the sublist (last = mid).
// search the lower sublist
if (target < midvalue)
<reposition last to mid>
<search sublist arr[first]…arr[mid-1]
last-1
first
target
C ase 2: target < midvalue
Search lower sublist
mid-1 last

14
Case 3.
The value of target is greater than midvalue and the search
must continue in the upper sublist . Reposition the index first
to the front of the sublist (first = mid+1).
// search upper sublist
if (target > midvalue)
<reposition first to mid+1>
<search sublist arr[mid+1]…arr[last-1]>
C ase 3: target > mid value
S earch up p er sub list
last-1
new first = mid + 1
first
target
last

15
Illustrating the Binary Search - Successful Search
1. Search for target = 23
Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.
Since target = 23 > midvalue = 12, step 2 searches the
upper sublist with first = 5 and last = 9.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8 9

16
Step 2:
Indices first = 5, last = 9, mid = (5+9)/2 = 7.
Since target = 23 < midvalue = 33, step 3 searches the lower
sublist with first = 5 and last = 7.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8 9

17
Since target = midvalue = 23, a match is found at index mid = 6.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8 9

18
Illustrating the Binary Search - Unsuccessful Search
Search for target = 4.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8 9
sublist with first = 0 and last = 4.

19
sublist with first = 0 and last 2.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8

20
Since target = 4 > midvalue = 3, step 4 should search the
upper sublist with first = 2 and last =2.
However, since first >= last, the target is not in the list and
we return index last = 9.
mid
-7 3 5 8 12 16
arr
0 1 2 3 4 5
23 33 55
6 7 8 9

21
Binary Search Algorithm
int binSearch(const int arr[], int first, int last, int target)
{
int mid; // index of the midpoint
int midvalue; // object that is assigned arr[mid]
int origLast = last; // save original value of last
while (first < last) // test for nonempty sublist
{
mid = (first+last)/2;
midvalue = arr[mid];
if (target == midvalue)
return mid; // have a match
// determine which sublist to search
else if (target < midvalue)
last = mid; // search lower sublist. reset last
else
first = mid+1; // search upper sublist. Reset first
}
return origLast; // target not found
}

23
Mathematical Definition of f(n) = O(g(n))
• We say that f(n)=O(g(n)) if there are positive constants C
and K such that f(n)  C*g(n) when n  K.
• Here, we are concerned with n being positive real numbers
(usually positive integers)
• Examples:
 f(n) = n = O(n) with g(n) = n, C = 1, K = 1
 f(n) = 10n = O(n) with g(n)=n, C=10, K = 1
 Note that f(n) = 10n = O(n2
) with g(n) = n2
, C = 1, K = 10
 Also, f(n) = 10n = O(n2
) with g(n) = n2
, C = 2, K = 5
 Hence, g(n), C and K are not unique

24
Big-O notation
• g(n) is an upper bound for f(n)
• We say that f(n) is no larger than g(n)
• We say f(n) is of order of g(n)
• We say f(n) is Big-O of g(n), i.e., f(n) = O(g(n))
• We say the growth rate of f(n) is less than or equal to the
growth rate of g(n)

25
Graphic Representation of f(n) = O(g(n))
K

26
Any Algorithm that is O(n2
) is also O(n3
)
K
K’

27
Constant Time Algorithms
An algorithm is O(1) when its running time is independent of
the number of data items. The algorithm runs in constant
time.
The storing of the element involves a simple assignment
statement and thus has efficiency O(1).
front rear
Direct Insert at Rear

28
Linear Time Algorithms
An algorithm is O(n) when its running time is proportional to
the size of the list.
When the number of elements doubles, the number of
operations doubles.
Sequential Search for the Minimum Element in an A rray
32 46 8 12 3
minimum elem ent found
in the list after n comparisons
n = 5
1 2 3 4 5

29
Exponential Algorithms
• Algorithms with running time O(n2
) are quadratic.
 practical only for relatively small values of n.
• Whenever n doubles, the running time of the algorithm increases
by a factor of 4.
• Algorithms with running time O(n3
)are cubic.
 efficiency is generally poor; doubling the size of n
increases the running time eight-fold.

30
Logarithmic Time Algorithms
The logarithm of n, base 2, is commonly used when
analyzing computer algorithms.
Ex. log2(2) = 1
log2(75) = 6.2288
When compared to the functions n and n2
,
the function log2 n grows very slowly.
n
n2
log2n

31
Common Functions
Function Name
C or 1 Constant
log n Logarithmic
log2
n Log-squared
n Linear
n log n
n2
Quadratic
n3
Cubic
2n
Exponential

32
Growth Rates of Various Functions
O(n2
)
O(n log n) O(n)
O(log n)
O(1)
n
Running
Time O(2n
)

33
Example of Running Time
f(n) n 103 operations
105 operations
106 operations
log n .000010sec .000017sec .000020sec
n .001sec .1sec 1sec
n log n .01sec 1.7sec 20sec
n2
1sec 3hr 12days
n3
17min 32cent 30000cent
2n
10285
cent 101000
yrs 1010000
yrs
Assume 1 operation per microsecond (0.000001)

34
Examples
)
(
3 4
4
x
O
x 
2
3
30
,
log
8
,
3400
,
116
,
log
12
,
4
,
005
.
0 x
x
x
x
x
x x
Give the asymptotic notation (Big-O notation) of the following
functions and rearrange them in ascending order by rate of
growth. (Hint: )
x
x
x
x
x
x
x 4
,
005
.
0
,
30
,
log
8
,
3400
,
log
12
,
116 3
2
)
(
),
log
(
),
(
),
1
(
),
(log
),
4
(
),
( 2
3
x
O
x
x
O
x
O
O
x
O
O
x
O x
reorder


35
Let n = size of initial region to be searched.
For unsuccessful search (worst case)
Worst Case: keep dividing n by 2 until n = 0
The number of divisions will be: floor(log
2
n) + 1
So worstTime(n) is logarithmic in n.
Binary Search Running Time Analysis
The averageTime(n) is also logarithmic in n
Because, for an unsuccessful search,
the algorithm terminates only when n = 0.
For successful search:
worst case: keep dividing n by 2 until n = 1.
first = last – 1 = middle.
The worstTime(n) is logarithmic in n
The averagetime(n) is logarithmic in n also.
Then

37
Recursive Functions
• A function is recursive if it includes a call to itself.
• The usual example of a recursive functions is factorial
 n! = n x (n-1) x (n-2) x … x 2 x 1
 4! = 4 x 3 x 2 x 1 = 24
• For n > 1, we can calculate n! in terms of (n – 1)!
• The recursive definition is
 n! = 1 if n == 1 or n == 0
 n! = n x (n-1)! otherwise
• The factorial function is defined using the factorial function
 4! = 4 x 3!
 3! = 3 x 2!
 2! = 2 x 1!
 1! = 1 (calculate directly)
• We can then work back up:
 2! = 2 x 1 = 2
 3! = 3 x 2 = 6
 4! = 4 x 6 = 24

38
Recursive Computation of 4!

39
factorial.cpp
#include <iostream>
using namespace std;
int factorial(int n)
{
if ((n == 1) || (n == 0))
return 1;
else
return n * factorial(n - 1);
}
int main()
{
int m;
cout << “Enter number for factorial computation: ";
cin >> m;
cout << factorial(m) << endl;
return 0;
}
• Running-time of the worst case, worstTime(n), is linear in n. That is,
O(n) is the smallest upper bound of worstTime(n).

40
Iterative Functions
• An iterative function is one that has a loop statement.
• Any problem that can be solved recursively can also be solved iteratively.
// Precondition: n >= 0.
// Postcondition: The value returned is n!, the product of
// all integers between 1 and n, inclusive.
int factorial (int n)
{
int product = n;
if (n == 0)
return 1;
for (int i = n -1; i > 1; i--)
product = product * i;
return product;
}
• In general, worstTime(n) depends on only two things:
 the number of loop iterations as a function of n;
 the number of recursive calls as a function of n.

41
Converting From Decimal To Binary
• The input is a non-negative decimal integer.
• The output is the binary representation of
that integer.
• Example: Print the binary equivalent of 34
34 % 2 = 0 0
34 / 2 = 17
17 % 2 = 1 1
17 / 2 = 8
8 % 2 = 0 0
8 / 2 = 4
4 % 2 = 0 0
4 / 2 = 2
2 % 2 = 0 0
2 / 2 = 1 1
Read bits from bottom to top: 100010
The rightmost bit is 34 % 2 = 0;
The other bits are the binary equivalent of
34 / 2, which is 17.
Repeated division by 2

42
// Precondition: n >= 0.
// Postcondition: The binary equivalent of n has been printed.
void writeBinary (int n)
{
if (n == 0 || n == 1)
cout << n;
else
{
writeBinary (n / 2);
cout << n % 2;
}
}
In general, the rightmost bit has the value of n % 2;
the other bits are the binary equivalent of n / 2.
Recursive Solution
• For n > 0, the number of recursive calls is the number of times that n can be divided
by 2 until n = 1. So worstTime(n) is logarithmic in n.

43
Tower of Hanoi Problem
• Given 3 poles (a, b, c) and n disks of increasing size
(1, 2, 3, …, n), move the n disks from pole a to pole b.
use pole c for temporary storage
• Rules:
 Only one disk may be moved at any time.
 No disk may ever be placed on top of a smaller disk.
 Other than the prohibition of rule 2, the top disk on
any pole may be moved to either of the other poles.

44
Initially, with 4 Disks
1
2
3
4
A B C

45
Instead of trying to figure out where to move disk 1,
let’s look at the picture just before disk 4 is moved:
So we will be able to move 4 disks from one pole to
another if we are able to figure out how to move 3 disks
from one pole to another (aha!).
to move 3 disks …
Just before Disk 4 is Moved
1
2
4 3
A B C

46
If n = 1, move disk 1 from pole ‘A’ to pole ‘B’.
Otherwise,
1. move n – 1 disks from ‘A’ to ‘C’, with ‘B’ as a temporary.
2. move disk n from ‘A’ to ‘B’.
3. move n – 1 disks from ‘C’ to ‘B’, with ‘A’ as a temporary.
For the sake of generality, use variables instead of constants
for the poles:
orig = ‘A’
dest = ‘B’
temp = ‘C’
Thinking Recursively

47
Here is the strategy to move n disks from orig to dest:
If n = 1, move disk 1 from orig to dest.
otherwise,
1. move n-1 disks from orig to temp.
2. move disk n from orig to dest.
3. move n-1 disks from temp to dest
Thinking Recursively

48
// Precondition: n > 0.
// Postcondition: The steps needed to move n disks from
// pole orig to pole dest have been written out.
// Pole temp is used for temporary storage.
// The worstTime(n) is O(2 n
).
void move (int n, char orig, char dest, char temp)
{
if (n == 1)
cout << "Move disk 1 from " << orig << " to "
<< dest << endl;
else
{
move (n 1,
‑ orig, temp, dest);
cout << "Move disk " << n << " from " << orig
<< " to " << dest << endl;
move (n 1,
‑ temp, dest, orig) ;
}
}
Recursive Solution

49
move (n, …)
move (n-1, …) move (n-1, …)
move (n-2, …) move (n-2, …) move (n-2, …) move (n-2, …)
… … … … … … … …
move (1, …) move (1, …) …
Analysis: Running Time(n)  # of calls to move

50
There are n levels in this tree
The number of calls to move at level 0 is 1 = 2
0
The number of calls to move at level 1 is 2 = 2
1
The number of calls to move at level 2 is 4 = 2 2
…
The number of calls to move at level n-1 is 2
n-1
Analysis: Running Time(n)  # of calls to move

51
The total number of calls to move is:
1
2
2
2
...
4
2
1
1
0
1






 


 n
n
i
i
n
We can prove the following equation by mathematical induction.
• We conclude that worstTime(n) is O(2n
), and, because 2n
- 1 disks
must be moved, o(2n
) is the smallest upper bound of worstTime(n).
• Because n appears as the exponent in the estimate of worstTime(n), we
say that worstTime(n) is exponential in n.
•Whenever possible, avoid methods that take exponential time.

Data structure and problem solving ch03.ppt

More Related Content

Similar to Data structure and problem solving ch03.ppt

More from Ping261512

Recently uploaded

Data structure and problem solving ch03.ppt