Fundamentals of the analysis of algorithm efficiency

Chapter 2
Chapter 2
Fundamentals of the Analysis
Fundamentals of the Analysis
of Algorithm Efficiency
of Algorithm Efficiency
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

2-2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd
ed., Ch. 2
Analysis of algorithms
Analysis of algorithms
 Issues:
Issues:
• correctness
correctness
• time efficiency
time efficiency
• space efficiency
space efficiency
• optimality
optimality
 Approaches:
Approaches:
• theoretical analysis
theoretical analysis
• empirical analysis
empirical analysis

2-3
ed., Ch. 2
Theoretical analysis of time efficiency
Theoretical analysis of time efficiency
Time efficiency is analyzed by determining the number of
Time efficiency is analyzed by determining the number of
repetitions of the
repetitions of the basic operation
basic operation as a function of
as a function of input size
input size
 Basic operation
Basic operation: the operation that contributes the most
: the operation that contributes the most
towards the running time of the algorithm
towards the running time of the algorithm
T
T(
(n
n)
) ≈
≈ c
cop
opC
C(
(n
n)
)
running time execution time
for basic operation
or cost
Number of times
basic operation is
executed
input size
Note: Different basic operations may cost differently!

2-4
ed., Ch. 2
Input size and basic operation examples
Input size and basic operation examples
Problem
Problem Input size measure
Input size measure Basic operation
Basic operation
Searching for key in a
Searching for key in a
list of
list of n
n items
items
Number of list
Number of list’
’s items,
s items,
i.e.
i.e. n
n
Key comparison
Key comparison
Multiplication of two
matrices
matrices
Matrix dimensions or
Matrix dimensions or
total number of elements
total number of elements
numbers
numbers
Checking primality of
Checking primality of
a given integer
a given integer n
n
n
n’
’size = number of digits
size = number of digits
(in binary representation)
(in binary representation)
Division
Division
Typical graph problem
Typical graph problem #vertices and/or edges
#vertices and/or edges
Visiting a vertex or
Visiting a vertex or
traversing an edge
traversing an edge

2-5
ed., Ch. 2
Empirical analysis of time efficiency
Empirical analysis of time efficiency
 Select a specific (typical) sample of inputs
Select a specific (typical) sample of inputs
 Use physical unit of time (e.g., milliseconds)
Use physical unit of time (e.g., milliseconds)
or
or
Count actual number of basic operation
Count actual number of basic operation’
’s executions
s executions
 Analyze the empirical data
Analyze the empirical data

2-6
ed., Ch. 2
Best-case, average-case, worst-case
Best-case, average-case, worst-case
For some algorithms, efficiency depends on form of input:
For some algorithms, efficiency depends on form of input:
 Worst case: C
Worst case: Cworst
worst(
(n
n) – maximum over inputs of size
) – maximum over inputs of size n
n
 Best case: C
Best case: Cbest
best(
(n
n) – minimum over inputs of size
) – minimum over inputs of size n
n
 Average case: C
Average case: Cavg
avg(
(n
n) –
) – “
“average
average”
” over inputs of size
over inputs of size n
n
• Number of times the basic operation will be executed on typical input
Number of times the basic operation will be executed on typical input
• NOT the average of worst and best case
NOT the average of worst and best case
• Expected number of basic operations considered as a random variable
Expected number of basic operations considered as a random variable
under some assumption about the probability distribution of all possible
under some assumption about the probability distribution of all possible
inputs. So, avg = expected under uniform distribution.
inputs. So, avg = expected under uniform distribution.

2-7
ed., Ch. 2
Example: Sequential search
Example: Sequential search
 Worst case
Worst case
 Best case
Best case
 Average case
Average case
n key comparisons
1 comparisons
(n+1)/2, assuming K is in A

2-8
ed., Ch. 2
Types of formulas for basic operation
Types of formulas for basic operation’
’s count
s count
 Exact formula
Exact formula
e.g., C(
e.g., C(n
n) =
) = n
n(
(n
n-1)/2
-1)/2
 Formula indicating order of growth with specific
Formula indicating order of growth with specific
multiplicative constant
e.g., C(
e.g., C(n
n)
) ≈
≈ 0.5
0.5 n
n2
2
 Formula indicating order of growth with unknown
Formula indicating order of growth with unknown
e.g., C(
e.g., C(n
n)
) ≈
≈ cn
cn2
2

2-9
ed., Ch. 2
Order of growth
Order of growth
 Most important: Order of growth within a constant multiple
Most important: Order of growth within a constant multiple
as
as n
n→∞
→∞
 Example:
Example:
• How much faster will algorithm run on computer that is
How much faster will algorithm run on computer that is
twice as fast?
twice as fast?
• How much longer does it take to solve problem of double
How much longer does it take to solve problem of double
input size?
input size?

2-10
ed., Ch. 2
Values of some important functions as
Values of some important functions as n
n 
 


2-11
ed., Ch. 2
Asymptotic order of growth
Asymptotic order of growth
A way of comparing functions that ignores constant factors and
A way of comparing functions that ignores constant factors and
small input sizes
small input sizes (because?)
(because?)
 O(
O(g
g(
(n
n)): class of functions
)): class of functions f
f(
(n
n) that grow
) that grow no faster
no faster than
than g
g(
(n
n)
)
 Θ
Θ(
(g
g(
(n
f(
(n
n) that grow
) that grow at same rate
at same rate as
as g
g(
(n
n)
)
 Ω
Ω(
(g
g(
(n
f(
(n
n) that grow
) that grow at least as fast
at least as fast as
as g
g(
(n
n)
)

2-12
ed., Ch. 2
Big-oh
Big-oh

2-13
ed., Ch. 2
Big-omega
Big-omega

2-14
ed., Ch. 2
Big-theta
Big-theta

2-15
ed., Ch. 2
Establishing order of growth using the definition
Establishing order of growth using the definition
Definition:
Definition: f
f(
(n
n) is in O(
) is in O(g
g(
(n
n)), denoted f
)), denoted f(n)
(n) 
 O(g(n)),
O(g(n)), if order
if order
of growth of
of growth of f
f(
(n
n)
) ≤
≤ order of growth of
order of growth of g
g(
(n
n) (within
) (within
constant multiple), i.e., there exist positive constant
constant multiple), i.e., there exist positive constant c
c and
and
non-negative integer
non-negative integer n
n0
0 such that
such that
f
f(
(n
n)
) ≤
≤ c g
c g(
(n
n) for every
) for every n
n ≥
≥ n
n0
0
Examples:
Examples:
 10
10n
n is in O(
is in O(n
n2
2
)
)
 5
5n
n+20 is in O(
+20 is in O(n
n)
)

2-16
ed., Ch. 2

-notation
-notation
 Formal definition
Formal definition
• A function
A function t(n)
t(n) is said to be in
is said to be in 
(g(n)),
(g(n)), denoted
denoted t(n)
t(n) 


(g(n)),
(g(n)), if
if t(n)
t(n) is bounded below by some constant multiple
is bounded below by some constant multiple
of
of g(n)
g(n) for all large
for all large n
n, i.e.,
, i.e., if there exist some positive
if there exist some positive
constant c and some nonnegative integer
constant c and some nonnegative integer n
n0
0 such that
such that
t(n)
t(n) 
 cg(n) for all n
cg(n) for all n 
 n
n0
0
 Exercises:
Exercises: prove the following using the above definition
prove the following using the above definition
• 10
10n
n2
2

 
(
(n
n2
2
)
)
• 0.3
0.3n
n2
2
- 2n
- 2n 
 
(
(n
n2
2
)
)
• 0.1
0.1n
n3
3

 
(
(n
n2
2
)
)

2-17
ed., Ch. 2

-notation
-notation
 Formal definition
Formal definition
• A function
A function t(n)
t(n) is said to be in
is said to be in 
(g(n)),
(g(n)), denoted
denoted t(n)
t(n) 
 
(g(n)),
(g(n)),
if
if t(n)
t(n) is bounded both above and below by some positive
is bounded both above and below by some positive
constant multiples of
constant multiples of g(n)
g(n) for all large
for all large n
n, i.e.,
, i.e., if there exist some
if there exist some
positive constant c
positive constant c1
1 and c
and c2
2 and some nonnegative integer
and some nonnegative integer n
n0
0
such that
such that
c
c2
2 g(n)
g(n) 
 t(n)
t(n) 
 c
c1
1 g(n) for all n
g(n) for all n 
 n
n0
0
 Exercises:
Exercises: prove the following using the above definition
prove the following using the above definition
• 10
10n
n2
2

 
(
(n
n2
2
)
)
• 0.3
0.3n
n2
2
- 2n
- 2n 
 
(
(n
n2
2
)
)
• (1/2)n(n+1)
(1/2)n(n+1) 
 
(
(n
n2
2
)
)

2-18
ed., Ch. 2
(g(n)), functions that grow at least as fast as g(n)
(g(n)), functions that grow at the same rate as g(n)
O(g(n)), functions that grow no faster than g(n)
g(n)
>=
<=
=

2-19
ed., Ch. 2
Theorem
Theorem
 If
If t
t1
1(n)
(n) 
 O(g
O(g1
1(n))
(n)) and
and t
t2
2(n)
(n) 
 O(g
O(g2
2(n)),
(n)), then
then
t
t1
1(n) + t
(n) + t2
2(n)
(n) 
 O(max{g
O(max{g1
1(n), g
(n), g2
2(n)}).
(n)}).
• The analogous assertions are true for the
The analogous assertions are true for the 
-notation
-notation
and
and 
-notation.
-notation.
 Implication: The algorithm
Implication: The algorithm’
’s overall efficiency will be determined by
s overall efficiency will be determined by
the part with a larger order of growth, i.e., its least efficient part.
the part with a larger order of growth, i.e., its least efficient part.
• For example,
For example, 5n
5n2
2
+ 3nlogn
+ 3nlogn 
 O(n
O(n2
2
)
)
Proof. There exist constants c1, c2, n1, n2 such that
t
t1
1(n)
(n) 
 c
c1
1*
*g
g1
1(n),
(n), for all
for all n
n 
 n
n1
1
t
t2
2(n)
(n) 
 c
c2
2*
*g
g2
2(n),
(n), for all
for all n
n 
 n
n2
2
Define
Define c
c3
3 = c
= c1
1 + c
+ c2
2 and
and n
n3
3 = max{n
= max{n1
1,n
,n2
2}. Then
}. Then
t
t1
1(n) + t
(n) + t2
2(n)
(n) 
 c
c3
3*
*max{g
max{g1
1(n), g
(n), g2
2(n)},
(n)}, for all
for all n
n 
 n
n3
3

2-20
ed., Ch. 2
Some properties of asymptotic order of growth
Some properties of asymptotic order of growth
 f
f(
(n
n)
) 
 O(
O(f
f(
(n
n))
))
 f
f(
(n
n)
) 
 O(
O(g
g(
(n
n)) iff
)) iff g
g(
(n
n)
) 
(
(f
f(n))
(n))
 If
If f
f (
(n
n)
) 
 O(
O(g
g (
(n
n)) and
)) and g
g(
(n
n)
) 
 O(
O(h
h(
(n
n)) , then
)) , then f
f(
(n
n)
) 
 O(
O(h
h(
(n
n))
))
Note similarity with
Note similarity with a
a ≤
≤ b
b
 If
If f
f1
1(
(n
n)
) 
 O(
O(g
g1
1(
(n
n)) and
)) and f
f2
2(
(n
n)
) 
 O(
O(g
g2
2(
(n
n)) , then
)) , then
f
f1
1(
(n
n) +
) + f
f2
2(
(n
n)
) 
 O(max{
O(max{g
g1
1(
(n
n),
), g
g2
2(
(n
n)})
)})
Also,
Also, 
1
1
i
i
n
n 
(
(f
f(
(i
i)) =
)) = 
 (
(
1
1
i
i
n
n f
f(
(i
i))
))
Exercise: Can you prove these properties?
Exercise: Can you prove these properties?

2-21
ed., Ch. 2
Establishing order of growth using limits
Establishing order of growth using limits
lim
lim T
T(
(n
n)/
)/g
g(
(n
n) =
) =
0
0 order of growth of T
T(
(n)
n) < order of growth of g
g(
(n
n)
)
c
c > 0
> 0 order of growth of T
T(
(n)
n) = order of growth of g
g(
(n
n)
)
∞
∞ order of growth of T
T(
(n)
n) > order of growth of g
g(
(n
n)
)
Examples:
Examples:
• 10
10n
n vs.
vs. n
n2
2
• n
n(
(n
n+1)/2 vs.
+1)/2 vs. n
n2
2
n
n→∞
→∞

2-22
ed., Ch. 2
L
L’
’Hôpital
Hôpital’
’s rule and Stirling
s rule and Stirling’
’s formula
s formula
L
L’
’Hôpital
Hôpital’
’s rule: If
s rule: If lim
limn
n
 f
f(
(n
n) =
) = lim
limn
n
 g(n
g(n) =
) = 
 and
and
the derivatives
the derivatives f
f´,
´, g
g´ exist, then
´ exist, then
Stirling
Stirling’
’s formula:
s formula: n
n!
! 
 (2
(2
n
n)
)1/2
1/2
(
(n
n/e)
/e)n
n
f
f(
(n
n)
)
g
g(
(n
n)
)
lim
lim
n
n

=
f
f ´(
´(n
n)
)
g
g ´(
´(n
n)
)
lim
lim
n
n

Example: log
Example: log n
n vs.
vs. n
n
Example: 2
Example: 2n
n
vs.
vs. n
n!
!

2-23
ed., Ch. 2
Orders of growth of some important functions
Orders of growth of some important functions
 All logarithmic functions log
All logarithmic functions loga
a n
n belong to the same class
belong to the same class

(log
(log n
n)
) no matter what the logarithm
no matter what the logarithm’
’s base
s base a
a > 1 is
> 1 is
because
because
 All polynomials of the same degree
All polynomials of the same degree k
k belong to the same class:
belong to the same class:
a
ak
kn
nk
k
+
+ a
ak
k-1
-1n
nk
k-1
-1
+ … +
+ … + a
a0
0 
 
(
(n
nk
k
)
)
 Exponential functions
Exponential functions a
an
n
have different orders of growth for different
have different orders of growth for different a
a’
’s
s
 order
order log
log n <
n < order
order n
n

(
(
>0) < order
>0) < order a
an
n
< order
< order n
n! < order
! < order n
nn
n
a
n
n b
b
a log
/
log
log 

2-24
ed., Ch. 2
Basic asymptotic efficiency classes
Basic asymptotic efficiency classes
1
1 constant
constant
log
log n
n logarithmic
logarithmic
n
n linear
linear
n
n log
log n
n n-
n-log
log-n
-n
n
n2
2
quadratic
quadratic
n
n3
3
cubic
cubic
2
2n
n
exponential
exponential
n
n!
! factorial
factorial

2-25
ed., Ch. 2
Time efficiency of nonrecursive algorithms
Time efficiency of nonrecursive algorithms
General Plan for Analysis
General Plan for Analysis
 Decide on parameter
Decide on parameter n
n indicating
indicating input size
input size
 Identify algorithm
Identify algorithm’
’s
s basic operation
basic operation
 Determine
Determine worst
worst,
, average
average, and
, and best
best cases for input of size
cases for input of size n
n
 Set up a sum for the number of times the basic operation is
Set up a sum for the number of times the basic operation is
executed
executed
 Simplify the sum using standard formulas and rules (see
Simplify the sum using standard formulas and rules (see
Appendix A)
Appendix A)

2-26
ed., Ch. 2
Useful summation formulas and rules
Useful summation formulas and rules

l
l
i
i
n
n1 = 1+1+…+1 =
1 = 1+1+…+1 = n
n -
- l
l + 1
+ 1
In particular,
In particular, 
l
l
i
i
n
n1 =
1 = n
n - 1 + 1 =
- 1 + 1 = n
n 
 
(
(n
n)
)

1
1
i
i
n
n i
i = 1+2+…+
= 1+2+…+n
n =
= n
n(
(n
n+1)/2
+1)/2 
 n
n2
2
/2
/2 
 
(
(n
n2
2
)
)

1
1
i
i
n
n i
i2
2
= 1
= 12
2
+2
+22
2
+…+
+…+n
n2
2
=
= n
n(
(n
n+1)(2
+1)(2n
n+1)/6
+1)/6 
 n
n3
3
/3
/3 
 
(
(n
n3
3
)
)

0
0
i
i
n
n a
ai
i
= 1
= 1 +
+ a
a +…+
+…+ a
an
n
= (
= (a
an
n+1
+1
- 1)/(
- 1)/(a
a - 1) for any
- 1) for any a
a 
 1
1
In particular,
In particular, 
0
0
i
i
n
n 2
2i
i
= 2
= 20
0
+ 2
+ 21
1
+…+ 2
+…+ 2n
n
= 2
= 2n
n+1
+1
- 1
- 1 
 
(2
(2n
n
)
)

(
(a
ai
i ±
± b
bi
i ) =
) = 
a
ai
i ±
± 
b
bi
i 
ca
cai
i =
= c
c
a
ai
i 
l
l
i
i
u
ua
ai
i =
= 
l
l
i
i
m
ma
ai
i +
+ 
m
m+1
+1
i
i
u
ua
ai
i

2-27
ed., Ch. 2
Example 1: Maximum element
Example 1: Maximum element
T(n) = 
1
1
i
i
n-1
n-1 1 = n-1 =
1 = n-1 = 
(
(n
n)
) comparisons

2-28
ed., Ch. 2
Example 2: Element uniqueness problem
Example 2: Element uniqueness problem
T(n) = 
0
0
i
i
n-2
n-2 (
(
i+1
i+1
j
j
n-1
n-1 1)
=
= 
0
0
i
i
n-2
n-2 n-i-1 = (n-1+1)(n-1)/2
= 
( )
( ) comparisons
2
n

2-29
ed., Ch. 2
Example 3: Matrix multiplication
Example 3: Matrix multiplication
T(n) = 
0
0
i
i
n-1
n-1 
0
0
i
i
n-1
n-1 n
= 
0
0
i
i
n-1
n-1 
(
( )
= 
(
( ) multiplications
2
n
3
n

2-30
ed., Ch. 2
Example 4: Gaussian elimination
Example 4: Gaussian elimination
Algorithm
Algorithm GaussianElimination
GaussianElimination(
(A
A[0..
[0..n
n-
-1,0..
1,0..n
n])
])
//Implements Gaussian elimination on an
//Implements Gaussian elimination on an n-
n-by
by-
-(
(n
n+1) matrix
+1) matrix A
A
for
for i
i 
 0
0 to
to n
n -
- 2
2 do
do
for
for j
j 
 i
i + 1
+ 1 to
to n
n -
- 1
1 do
do
for
for k
k 
 i
i to
to n
n do
do
A
A[
[j
j,
,k
k]
] 
 A
A[
[j
j,
,k
k]
] -
- A
A[
[i
i,
,k
k]
] 
 A
A[
[j
j,
,i
i] /
] / A
A[
[i
i,
,i
i]
]
Find the efficiency class and a constant factor improvement.
Find the efficiency class and a constant factor improvement.
for
for i
i 
 0
0 to
to n
n -
- 2
2 do
do
for
for j
j 
 i
i + 1
+ 1 to
to n
n -
- 1
1 do
do
B
B 
 A
A[
[j,i
j,i] /
] / A
A[
[i
i,
,i
i]
]
for
for k
k 
 i
i to
to n
n do
do
A
A[
[j
j,
,k
k]
] 
 A
A[
[j
j,
,k
k]
] – A
– A[
[i,k
i,k]
] * B
* B

2-31
ed., Ch. 2
Example 5: Counting binary digits
Example 5: Counting binary digits
It cannot be investigated the way the previous examples are.
It cannot be investigated the way the previous examples are.
The halving game: Find integer i such that n/ ≤
≤ 1.
1.
Answer: i ≤ log n. So, T(n) =
Answer: i ≤ log n. So, T(n) = 
(
(log n) divisions.
Another solution: Using recurrence relations.
i
2

2-32
ed., Ch. 2
Plan for Analysis of Recursive Algorithms
Plan for Analysis of Recursive Algorithms
 Decide on a parameter indicating an input
Decide on a parameter indicating an input’
’s size.
s size.
 Identify the algorithm
Identify the algorithm’
’s basic operation.
s basic operation.
 Check whether the number of times the basic op. is executed
Check whether the number of times the basic op. is executed
may vary on different inputs of the same size. (If it may, the
may vary on different inputs of the same size. (If it may, the
worst, average, and best cases must be investigated separately.)
worst, average, and best cases must be investigated separately.)
 Set up a recurrence relation with an appropriate initial
Set up a recurrence relation with an appropriate initial
condition expressing the number of times the basic op. is
condition expressing the number of times the basic op. is
executed.
executed.
 Solve the recurrence (or, at the very least, establish its
Solve the recurrence (or, at the very least, establish its
solution
solution’
’s order of growth) by backward substitutions or
s order of growth) by backward substitutions or
another method.
another method.

2-33
ed., Ch. 2
Example 1: Recursive evaluation of
Example 1: Recursive evaluation of n
n!
!
Definition:
Definition: n
n ! = 1
! = 1 
 2
2 
 …
… 
(
(n-
n-1)
1) 
 n
n for
for n
n ≥
≥ 1 and 0! = 1
1 and 0! = 1
Recursive definition of
Recursive definition of n
n!:
!: F
F(
(n
n) =
) = F
F(
(n-
n-1)
1) 
 n
n for
for n
n ≥
≥ 1 and
1 and
F
F(
(0) = 1
0) = 1
Size:
Size:
Basic operation:
Basic operation:
Recurrence relation:
Recurrence relation:
n
multiplication
M(n) = M(n-1) + 1
M(0) = 0

2-34
ed., Ch. 2
Solving the recurrence for M(
Solving the recurrence for M(n
n)
)
M(
M(n
n) = M(
) = M(n
n-1) + 1, M(0) = 0
-1) + 1, M(0) = 0
M(n) = M(n-1) + 1
= (M(n-2) + 1) + 1 = M(n-2) + 2
= (M(n-3) + 1) + 2 = M(n-3) + 3
…
= M(n-i) + i
= M(0) + n
= n
The method is called backward substitution.

2-35
ed., Ch. 2
Example 2: The Tower of Hanoi Puzzle
Example 2: The Tower of Hanoi Puzzle
Recurrence for number of moves:
Recurrence for number of moves:
M(n) = 2M(n-1) + 1

2-36
ed., Ch. 2
Solving recurrence for number of moves
Solving recurrence for number of moves
M(
M(n
n) = 2M(
) = 2M(n
n-1) + 1, M(1) = 1
-1) + 1, M(1) = 1
M(n) = 2M(n-1) + 1
= 2(2M(n-2) + 1) + 1 = 2^2*M(n-2) + 2^1 + 2^0
= 2^2*(2M(n-3) + 1) + 2^1 + 2^0
= 2^3*M(n-3) + 2^2 + 2^1 + 2^0
= …
= 2^(n-1)*M(1) + 2^(n-2) + … + 2^1 + 2^0
= 2^(n-1) + 2^(n-2) + … + 2^1 + 2^0
= 2^n - 1

2-37
ed., Ch. 2
Tree of calls for the Tower of Hanoi Puzzle
Tree of calls for the Tower of Hanoi Puzzle

2-38
ed., Ch. 2
Example 3: Counting #bits
Example 3: Counting #bits
A( ) = A( ) + 1, A( ) = 1 (using the Smoothness Rule)
= (A( ) + 1) + 1 = A( ) + 2
= A( ) + i
= A( ) + k = k + 0
=
k
2 1
2 
k 0
2
2
2 
k
n
2
log
2
2 
k
i
k
2
k
k
2
A(n) = A( ) + 1, A(1) = 0
 
2
/
n

2-39
ed., Ch. 2
Smoothness Rule
Smoothness Rule
 Let f(n) be a
Let f(n) be a nonnegative
nonnegative function defined on the set of natural
function defined on the set of natural
numbers. f(n) is call
numbers. f(n) is call smooth
smooth if it is
if it is eventually
eventually nondecreasing
nondecreasing and
and
f(2n)
f(2n) ∈
∈ Θ
Θ (f(n))
(f(n))
• Functions that do not grow too fast, including
Functions that do not grow too fast, including logn
logn,
, n
n,
, nlogn
nlogn, and
, and
n
n

where
where 
>=0 are smooth.
>=0 are smooth.
 Smoothness rule
Smoothness rule
Let T(n) be an
Let T(n) be an eventually nondecreasing
eventually nondecreasing function and
function and f(n) be a
f(n) be a
smooth function
smooth function. If
. If
T(n)
T(n) ∈
∈ Θ
Θ (f(n))
(f(n)) for values of n that are powers of b,
for values of n that are powers of b,
where b>=2, then
where b>=2, then
T(n)
T(n) ∈
∈ Θ
Θ (f(n)) for any n.
(f(n)) for any n.

2-40
ed., Ch. 2
Fibonacci numbers
Fibonacci numbers
The Fibonacci numbers:
The Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, …
0, 1, 1, 2, 3, 5, 8, 13, 21, …
The Fibonacci recurrence:
The Fibonacci recurrence:
F(
F(n
n) = F(
) = F(n
n-1) + F(
-1) + F(n
n-2)
-2)
F(0) = 0
F(0) = 0
F(1) = 1
F(1) = 1
General 2
General 2nd
nd
order
order linear homogeneous recurrence
linear homogeneous recurrence with
with
constant coefficients:
constant coefficients:
a
aX(
X(n
n) +
) + b
bX(
X(n
n-1) +
-1) + c
cX
X(n
(n-2)
-2) =
= 0
0

2-41
ed., Ch. 2
Solving
Solving a
aX(
X(n
n) +
) + b
bX(
X(n
n-1) +
-1) + c
cX(
X(n
n-2)
-2) =
= 0
0
 Set up the characteristic equation (quadratic)
Set up the characteristic equation (quadratic)
ar
ar2
2
+
+ br
br +
+ c
c =
= 0
0
 Solve to obtain roots
Solve to obtain roots r
r1
1 and
and r
r2
2
 General solution to the recurrence
General solution to the recurrence
if
if r
r1
1 and
and r
r2
2 are two distinct real roots: X(
are two distinct real roots: X(n
n) =
) = α
αr
r1
1
n
n
+
+ β
βr
r2
2
n
n
if
if r
r1
1 =
= r
r2
2 =
= r
r are two equal real roots: X(
are two equal real roots: X(n
n) =
) = α
αr
rn
n
+
+ β
βnr
nr n
n
 Particular solution can be found by using initial conditions
Particular solution can be found by using initial conditions

2-42
ed., Ch. 2
Application to the Fibonacci numbers
Application to the Fibonacci numbers
F(
F(n
n) = F(
) = F(n
n-1) + F(
-1) + F(n
n-2) or F(
-2) or F(n
n) - F(
) - F(n
n-1) - F(
-1) - F(n
n-2) = 0
-2) = 0
Characteristic equation:
Characteristic equation:
Roots of the characteristic equation:
Roots of the characteristic equation:
General solution to the recurrence:
General solution to the recurrence:
Particular solution for F(0) =0, F(1)=1:
Particular solution for F(0) =0, F(1)=1:
2
r - r -1 = 0
2
/
)
5
1
(
2
,
1 

r
n
n
r
r 2
1 

 

1
2
1 


 r
r 

0

 


2-43
ed., Ch. 2
Computing Fibonacci numbers
Computing Fibonacci numbers
1.
1. Definition-based recursive algorithm
Definition-based recursive algorithm
2.
2. Nonrecursive definition-based algorithm
Nonrecursive definition-based algorithm
3.
3. Explicit formula algorithm
Explicit formula algorithm
4.
4. Logarithmic algorithm based on formula:
Logarithmic algorithm based on formula:
F
F(
(n
n-1)
-1) F
F(
(n
n)
)
F
F(
(n
n)
) F
F(
(n
n+1)
+1)
0 1
0 1
1 1
1 1
=
=
n
n
for
for n
n≥
≥1,
1, assuming an efficient way of computing matrix powers.
assuming an efficient way of computing matrix powers.

2-44
ed., Ch. 2
Important Recurrence Types
Important Recurrence Types
 Decrease-by-one recurrences
Decrease-by-one recurrences
• A decrease-by-one algorithm solves a problem by exploiting a relationship
A decrease-by-one algorithm solves a problem by exploiting a relationship
between a given instance of size
between a given instance of size n
n and a smaller size
and a smaller size n – 1
n – 1.
.
• Example:
Example: n!
n!
• The recurrence equation for investigating the time efficiency of such
The recurrence equation for investigating the time efficiency of such
algorithms typically has the form
T(n) = T(n-1) + f(n)
T(n) = T(n-1) + f(n)
 Decrease-by-a-constant-factor recurrences
Decrease-by-a-constant-factor recurrences
• A decrease-by-a-constant-factor algorithm solves a problem by dividing its
A decrease-by-a-constant-factor algorithm solves a problem by dividing its
given instance of size
given instance of size n
n into several smaller instances of size
into several smaller instances of size n/b
n/b, solving
, solving
each of them recursively, and then, if necessary, combining the solutions to
each of them recursively, and then, if necessary, combining the solutions to
the smaller instances into a solution to the given instance.
the smaller instances into a solution to the given instance.
• Example:
Example: binary search.
binary search.
• The recurrence equation for investigating the time efficiency of such
The recurrence equation for investigating the time efficiency of such
T(n) = aT(n/b) + f (n)
T(n) = aT(n/b) + f (n)

2-45
ed., Ch. 2
Decrease-by-one Recurrences
Decrease-by-one Recurrences
 One (constant) operation reduces problem size by one.
One (constant) operation reduces problem size by one.
T(
T(n
n) = T(
) = T(n
n-1) +
-1) + c
c T(1) =
T(1) = d
d
Solution:
Solution:
 A pass through input reduces problem size by one.
A pass through input reduces problem size by one.
T(
T(n
n) = T(
) = T(n
n-1) +
-1) + c n
c n T(1) =
T(1) = d
d
Solution:
Solution:
T(n) = (n-1)c + d linear
T(n) = [n(n+1)/2 – 1] c + d quadratic

2-46
ed., Ch. 2
Decrease-by-a-constant-factor recurrences –
Decrease-by-a-constant-factor recurrences –
The Master Theorem
The Master Theorem
T
T(
(n
n) =
) = aT
aT(
(n/b
n/b) +
) + f
f (
(n
n),
), where
where f
f (
(n
n)
) ∈
∈ Θ
Θ(
(n
nk
k
)
) , k>=0
, k>=0
1.
1. a < b
a < bk
k
T
T(
(n
n)
) ∈
∈ Θ
Θ(
(n
nk
k
)
)
2.
2. a = b
a = bk
k
T
T(
(n
n)
) ∈
∈ Θ
Θ(
(n
nk
k
log
log n
n )
)
3.
3. a > b
a > bk
k
T
T(
(n
n)
) ∈
∈ Θ
Θ(
(n
nlog
log a
a
)
)
4.
4. Examples:
Examples:
1.
1. T(n) = T(n/2) + 1
T(n) = T(n/2) + 1
2.
2. T(n) = 2T(n/2) + n
T(n) = 2T(n/2) + n
3.
3. T(n) = 3T(n/2) + n
T(n) = 3T(n/2) + n
4.
4. T(n) = T(n/2) + n
T(n) = T(n/2) + n Θ(nlog
2
3
)
Θ(log n)
Θ(nlog n)
b
Θ(n)

Fundamentals of the analysis of algorithm efficiency

More Related Content

Similar to Fundamentals of the analysis of algorithm efficiency

Recently uploaded

Fundamentals of the analysis of algorithm efficiency

Editor's Notes