1
What is analgorithm?
What is an algorithm?
An
An algorithm
algorithm is a sequence of unambiguous instructions
is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of
output for any legitimate input in a finite amount of
time.
time.
“computer”
problem
algorithm
input output
2.
What is analgorithm?
What is an algorithm?
• is any well-defined computational procedure
is any well-defined computational procedure
that takes some value, or set of values, as input
that takes some value, or set of values, as input
and produces some value, or set of values, as
and produces some value, or set of values, as
output.
output.
• is thus a sequence of computational steps that
is thus a sequence of computational steps that
transform the input into the output.
transform the input into the output.
• is a tool for solving a well - specified
is a tool for solving a well - specified
computational problem.
computational problem.
• Any special method of solving a certain kind of
Any special method of solving a certain kind of
problem (Webster Dictionary)
problem (Webster Dictionary)
2
3.
What is aproblem?
What is a problem?
Definition
Definition
• A mapping/relation between a set of input instances
A mapping/relation between a set of input instances
(domain) and an output set (range)
(domain) and an output set (range)
Problem Specification
Problem Specification
• Specify what a typical input instance is
Specify what a typical input instance is
• Specify what the output should be in terms of the input
Specify what the output should be in terms of the input
instance
instance
Example: Sorting
Example: Sorting
• Input: A sequence of N numbers a
Input: A sequence of N numbers a1
1…a
…an
n
• Output: the permutation (reordering) of the input
Output: the permutation (reordering) of the input
sequence such that a
sequence such that a1
1
a
a2
2
…
…
a
an
n .
.
3
4.
Types of Problems
Typesof Problems
Search: find X in the input satisfying property Y
Search: find X in the input satisfying property Y
Structuring: Transform input X to satisfy property Y
Structuring: Transform input X to satisfy property Y
Construction: Build X satisfying Y
Construction: Build X satisfying Y
Optimization: Find the best X satisfying property Y
Optimization: Find the best X satisfying property Y
Decision: Does X satisfy Y?
Decision: Does X satisfy Y?
4
5.
What do weanalyze about
What do we analyze about algorithms
algorithms
Correctness
Correctness
• Does the input/output relation match algorithm requirement?
Does the input/output relation match algorithm requirement?
Amount of work done
Amount of work done (aka complexity)
(aka complexity)
• Basic operations to do task
Basic operations to do task
Amount of space used
Amount of space used
• Memory used
Memory used
5
6.
What do weanalyze about
What do we analyze about algorithms
algorithms
Simplicity, clarity
Simplicity, clarity
• Verification and implementation.
Verification and implementation.
Optimality
Optimality
• Is it impossible to do better?
Is it impossible to do better?
6
7.
7
Euclid’s Algorithm
Euclid’s Algorithm
Problem:Find gcd(
Problem: Find gcd(m,n
m,n), the greatest common divisor of two
), the greatest common divisor of two
nonnegative, not both zero integers
nonnegative, not both zero integers m
m and
and n
n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of equality
Euclid’s algorithm is based on repeated application of equality
gcd(
gcd(m,n
m,n) = gcd(
) = gcd(n, m
n, m mod
mod n
n)
)
until the second number becomes 0, which makes the problem
until the second number becomes 0, which makes the problem
trivial.
trivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
8.
8
Two descriptions ofEuclid’s algorithm
Two descriptions of Euclid’s algorithm
Step 1 If
Step 1 If n
n = 0, return
= 0, return m
m and stop; otherwise go to Step 2
and stop; otherwise go to Step 2
Step 2
Step 2 Divide
Divide m
m by
by n
n and assign the value fo the remainder to
and assign the value fo the remainder to r
r
Step 3 Assign the value of
Step 3 Assign the value of n
n to
to m
m and the value of
and the value of r
r to
to n.
n. Go to
Go to
Step 1.
Step 1.
while
while n
n ≠ 0
≠ 0 do
do
r ← m
r ← m mod
mod n
n
m← n
m← n
n ← r
n ← r
return
return m
m
9.
9
Other methods forcomputing gcd(
Other methods for computing gcd(m,n
m,n)
)
Consecutive integer checking algorithm
Consecutive integer checking algorithm
Step 1 Assign the value of min{
Step 1 Assign the value of min{m,n
m,n} to
} to t
t
Step 2
Step 2 Divide
Divide m
m by
by t.
t. If the remainder is 0, go to Step 3;
If the remainder is 0, go to Step 3;
otherwise, go to Step 4
otherwise, go to Step 4
Step 3
Step 3 Divide
Divide n
n by
by t.
t. If the remainder is 0, return
If the remainder is 0, return t
t and stop;
and stop;
otherwise, go to Step 4
otherwise, go to Step 4
Step 4 Decrease
Step 4 Decrease t
t by 1 and go to Step 2
by 1 and go to Step 2
10.
10
Other methods forgcd(
Other methods for gcd(m,n
m,n) [cont.]
) [cont.]
Middle-school procedure
Middle-school procedure
Step 1 Find the prime factorization of
Step 1 Find the prime factorization of m
m
Step 2 Find the prime factorization of
Step 2 Find the prime factorization of n
n
Step 3 Find all the common prime factors
Step 3 Find all the common prime factors
Step 4 Compute the product of all the common prime factors
Step 4 Compute the product of all the common prime factors
and return it as gcd
and return it as gcd(m,n
(m,n)
)
Is this an algorithm?
Is this an algorithm?
11.
11
Sieve of Eratosthenes
Sieveof Eratosthenes
Input:
Input: Integer
Integer n
n ≥ 2
≥ 2
Output: List of primes less than or equal to
Output: List of primes less than or equal to n
n
for
for p
p ← 2
← 2 to
to n
n do
do A
A[
[p
p] ←
] ← p
p
for
for p
p ← 2
← 2 to
to
n
n
do
do
if
if A
A[
[p
p]
]
0 //
0 //p
p hasn’t been previously eliminated from the list
hasn’t been previously eliminated from the list
j
j ←
← p
p*
* p
p
while
while j
j ≤
≤ n
n do
do
A
A[
[j
j]
] ← 0
← 0 //mark element as eliminated
//mark element as eliminated
j
j ←
← j
j + p
+ p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
13
Two main issuesrelated to algorithms
Two main issues related to algorithms
How to design algorithms
How to design algorithms
How to analyze algorithm efficiency
How to analyze algorithm efficiency
14.
Algorithm design techniques/strategies
Algorithmdesign techniques/strategies
An algorithm design technique (or “strategy” or
An algorithm design technique (or “strategy” or
“paradigm”) is a general approach to solving problems
“paradigm”) is a general approach to solving problems
algorithmically that is applicable to a variety of problems
algorithmically that is applicable to a variety of problems
from different areas of computing.
from different areas of computing.
14
15.
15
Algorithm design techniques/strategies
Algorithmdesign techniques/strategies
Brute force
Brute force
Divide and conquer
Divide and conquer
Decrease and conquer
Decrease and conquer
Transform and conquer
Transform and conquer
Space and time tradeoffs
Space and time tradeoffs
Greedy approach
Greedy approach
Dynamic programming
Dynamic programming
Iterative improvement
Iterative improvement
Backtracking
Backtracking
Branch and bound
Branch and bound
16.
Algorithm design techniques/strategies
Algorithmdesign techniques/strategies
Brute force
Brute force is a straightforward approach to solving a
is a straightforward approach to solving a
problem, usually directly based on the problem statement
problem, usually directly based on the problem statement
and definitions of the concepts involved.
and definitions of the concepts involved.
The
The decrease-and-conquer
decrease-and-conquer technique is based on reducing
technique is based on reducing
the size of the input instance.
the size of the input instance.
Divide-and-Conquer
Divide-and-Conquer
• A problem is divided into several subproblems of the same type,
A problem is divided into several subproblems of the same type,
ideally of about equal size.
ideally of about equal size.
• The subproblems are solved separately.
The subproblems are solved separately.
• the solutions to the subproblems are combined to get a solution to
the solutions to the subproblems are combined to get a solution to
the original problem.
the original problem.
16
17.
Algorithm design techniques/strategies
Algorithmdesign techniques/strategies
Transform-and-Conquer
Transform-and-Conquer: Firstly, the problem instance is
: Firstly, the problem instance is
modified to be more Appropriate to solution. Then, in the
modified to be more Appropriate to solution. Then, in the
second or conquering stage, it is solved.
second or conquering stage, it is solved.
Dynamic programming
Dynamic programming is a technique for solving problems
is a technique for solving problems
with overlapping subproblems.
with overlapping subproblems.
The
The greedy approach
greedy approach suggests constructing a solution
suggests constructing a solution
through a sequence of steps until a complete solution to the
through a sequence of steps until a complete solution to the
problem is reached.
problem is reached.
Iterative improvement
Iterative improvement starts with some feasible solution
starts with some feasible solution
and proceeds to improve it by repeated applications of
and proceeds to improve it by repeated applications of
some simple step.
some simple step.
17
18.
18
Analysis of algorithms
Analysisof algorithms
How good is the algorithm?
How good is the algorithm?
• time efficiency
time efficiency
• space efficiency
space efficiency
Does there exist a better algorithm?
Does there exist a better algorithm?
• lower bounds
lower bounds
• optimality
optimality
19.
Properties as importantas performance
Properties as important as performance
Modularity
Modularity
Maintainability
Maintainability
Functionality
Functionality
Robustness and Reliability
Robustness and Reliability
User-friendliness
User-friendliness
19
20.
20
Important problem types
Importantproblem types
Sorting
Sorting
rearrange the items of a given list in non-decreasing order.
rearrange the items of a given list in non-decreasing order.
Searching
Searching
deals with f inding a given value, called a search key, in a given set
deals with f inding a given value, called a search key, in a given set
string processing
string processing: Eg, string matching
: Eg, string matching
graph problems
graph problems
graph-traversal algorithms
graph-traversal algorithms
shortest-path algorithms
shortest-path algorithms
21.
Important problem types
Importantproblem types
Combinatorial problems
Combinatorial problems: find a combinatorial object—
: find a combinatorial object—
such as a permutation, a combination, or a subset—that
such as a permutation, a combination, or a subset—that
satisfies certain constraints.
satisfies certain constraints.
Geometric problems
Geometric problems: deal with geometric objects such as
: deal with geometric objects such as
points, lines, and polygons.
points, lines, and polygons.
Numerical problems
Numerical problems: involve mathematical objects of
: involve mathematical objects of
continuous nature:
continuous nature:
solving equations and systems of equations,
solving equations and systems of equations,
computing definite integrals,
computing definite integrals,
evaluating functions, and
evaluating functions, and
…
….. so on.
.. so on.
21
Fundamental data structures
Fundamentaldata structures
Tree
Tree: is a connected acyclic graph
: is a connected acyclic graph
• Rooted Trees: for every two vertices in a tree, there always exists
Rooted Trees: for every two vertices in a tree, there always exists
exactly one simple path from one of these vertices to the other.
exactly one simple path from one of these vertices to the other.
Set and dictionary
Set and dictionary: an unordered collection (possibly
: an unordered collection (possibly
empty) of distinct items called elements of the set.
empty) of distinct items called elements of the set.
• The operations we need to perform for a set often are searching
The operations we need to perform for a set often are searching
for a given item, adding a new item, and deleting an item from
for a given item, adding a new item, and deleting an item from
the collection.
the collection.
• A data structure that implements these three operations is called
A data structure that implements these three operations is called
the dictionary.
the dictionary.
25
Editor's Notes
#7 Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.