CS8461 - Design and Analysis of Algorithms

CS8451
Design and Analysis of
Algorithms
Dr.K.Muthumanickam
Associate Professor/IT
Kongunadu College of Engineering and Technology
Tholur Patti, Thottiam Taluk
Trichy District – 621215, Tamilnadu

Design & Analysis of Algorithms
"algos" = Greek word for pain.
"algor" = Latin word for to be cold.
Why study this subject?
 Efficient algorithms lead to efficient programs.
 Efficient programs sell better.
 Efficient programs make better use of hardware.
 Programmers who write efficient programs are
preferred.

Objectives
To gain experiences in fundamental
techniques used for algorithm analysis and
the main methodologies used for the
design of efficient algorithms.
To study the most important computer
algorithms of current practical use.

Realtime Applications of
Algorithms
• There are many kinds of algorithm. Each have
their own applications in the real world.
• For example, programming languages are
completely based on algorithms which help
you design software like games, MS Office,
Google, etc. Moreover they help you build
things like robots!

UNIT – I - Introduction
Notion of an Algorithm – Fundamentals of
Algorithmic Problem Solving – Important
Problem Types – Fundamentals of the Analysis
of Algorithm Efficiency – Asymptotic Notations
and its properties – Analysis Framework -
Empirical analysis - Mathematical analysis for
Recursive and Non-recursive algorithms -
Visualization

What is an algorithm?
An algorithm is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of
time.
The notion of the algorithm.
“computer”
problem
algorithm
input output

Algorithm VS Pseudo code
• In computer science, an algorithm is an
effective method expressed as a finite list of
well-defined instructions for calculating a
function.
• Pseudo code describes how you would
implement an algorithm without getting into
syntactical details.

Difference between Algorithm and Program
S.
No
Algorithm Program
1 Algorithm is finite Program need not to be
finite
2 Algorithm is written
using natural
language or
algorithmic language
Programs are written
using a specific
programming
language

Important points
• The nonambiguity requirement (understanding)
of each step is important.
• The range of inputs for which an algorithm works
has to be specified carefully.
• The same algorithm can be represented in several
different ways.
• There may exist several algorithms for solving the
same problem. Works with different ideas and
different speed.

• A prime number is a whole number greater than
1 whose only factors are 1 and itself.
• A prime number has only two factors: 1 and
itself
• The number 1 is neither prime nor composite
• A composite number has more than two factors.
Factors
• "Factors" are the numbers you multiply together
to get another number:

Prime Factorization
• "Prime Factorization" is finding which prime
numbers multiply together to make the
original number.

Prime Factorization Using the
Factor Tree Method
• One way to find the prime factorization of a
number is to make a factor tree.
Procedure
• start by writing the number, and then writing
it as the product of two factors.
• We write the factors below the number and
connect them to the number with a small line
segment—a “branch” of the factor tree.

• If a factor is prime, circle it, and do not factor
that “branch” any further.
• If a factor is not prime, we repeat this process,
writing it as the product of two factors and
adding new branches to the tree.
• Continue the above steps until all the
branches end with a prime.
• When the factor tree is complete, the circled
primes give us the prime factorization.

Example:
• For example, let’s find the prime factorization
of 36.
• We can start with any factor pair such
as 3 and 12.
• We write 3 and 12 below 36 with branches
connecting them.

• The factor 3 is prime, so we circle it.
• The factor 12 is
composite, so we need
to find its factors.
• Let’s use 3 and 4. We
write these factors on
the tree under the 12.

• The factor 4 is
composite, and it factors
into 2⋅2.
• We write these factors
under the 4. Since 2 is
prime, we circle both 2s.
• The factor 3 is prime, so we circle it.

• The prime factorization is the product of the
circled primes. We generally write the prime
factorization in order from least to greatest.
2⋅2⋅3⋅3
• In cases like this, where some of the prime
factors are repeated, we can write prime
factorization in exponential form.
22⋅32

Class Work
• Find the prime factorization of 48 using the
factor tree method.

Class Work
• What is the prime factorization of 147 ?

Greatest Common Divisor (GCD)
• The gcd of two nonnegative integers m and n,
denoted gcd(m, n), is defined as the largest
integer that divides both m and n evenly, i.e.,
with a remainder of zero.

Methods to find gcd of two
numbers
(a) Middle School Procedure
1. Find the prime factorization of m.
2. Find the prime factorization of n.
3. Find all the common prime factors.
4. Compute the product of all the common
prime factors and return it as gcd(m, n).

Example
Input: m = 12, n = 14
Prime factor of 12 = 2*2*3
Prime factor of 14 = 2*7
GCD(12, 14) = 2
Input: m = 5, n = 10; GCD(5, 10) = ?

(b) Euclid’s algorithm
• In modern terms, Euclid’s algorithm is based
on applying repeatedly the equality
gcd(m, n) = gcd(n, m mod n),
where m mod n is the remainder of the
division of m by n, until m mod n is equal
to 0.

ALGORITHM Euclid(m, n)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero
integers m and n
//Output: Greatest common divisor of m and n
while n ≠ 0 do
r ← m mod n
m ← n
n ← r
return m

Euclid(67,29) 67 mod 29 = 9
Euclid(29,9) 29 mod 9 = 2
Euclid(9,2) 9 mod 2 = 1
Euclid(2,1) 2 mod 1 = 0
Euclid(1,0) outputs 1
Euclid(A,B)
If B=0 then return A
else return Euclid(B, A mod B)
Euclid’s Algorithm for GCD

Class Work
Find gcd(31415, 14142) by applying
Euclid’s algorithm
Find gcd(29, 9) by applying Euclid’s
algorithm

Problem. Find gcd(31415, 14142) by applying Euclid’s algorithm
gcd(31415, 14142) = gcd(14142, 3131)
= gcd(3131, 1618)
= gcd(1618, 1513)
= gcd(1513, 105)
= gcd(1513, 105)
= gcd(105, 43)
= gcd(43, 19)
= gcd(19, 5)
= gcd(5, 4)
= gcd(4, 1) = gcd(1, 0) = 1.

Fundamentals of Algorithm and Problem Solving
We now list and briefly discuss a sequence of steps
one typically goes through in designing and
analyzing an algorithm

Understanding the Problem
• Before designing an algorithm is to understand
completely the problem given.
• Read the problem’s description carefully and ask
questions if you have any doubts about the
problem
• Do a few small examples by hand, think about
special cases, and ask questions again if needed.

• An input to an algorithm specifies an instance
of the problem the algorithm solves.
• It is very important to specify exactly the set
of instances the algorithm needs to handle.
• If you fail to do this, your algorithm may work
correctly for a majority of inputs
but crash on some “boundary” value.

Ascertaining the Capabilities of
the Computational Device
• Once you completely understand a problem,
you need to ascertain the capabilities of the
computational device the algorithm is
intended for.
• Random-access machine (RAM) - Its central
assumption is that instructions are executed
one after another, one operation at a time.
Accordingly, algorithms designed to be
executed on such machines are called
sequential algorithms.

• The central assumption of the RAM model
does not hold for some newer computers that
can execute operations concurrently, i.e., in
parallel.
• Algorithms that take advantage of this
capability are called parallel algorithms.
• Aware of the speed and memory available on
a particular computer system.

Choosing between Exact and
Approximate Problem Solving
• The next principal decision is to choose
between solving the problem exactly or
solving it approximately.
• In the former case, an algorithm is called an
exact algorithm; in the latter case, an algorithm
is called an approximation algorithm.
• Why would one opt for an approximation
algorithm?

• First, there are important problems that
simply cannot be solved exactly for most of
their instances; examples include extracting
square roots.
• Second, available algorithms for solving a
problem exactly can be unacceptably slow
because of the problem’s intrinsic complexity.

Algorithm Design Techniques
• Now, with all the components of the
algorithmic problem solving in place, how do
you design an algorithm to solve a given
problem?
• What is an algorithm design technique?
An algorithm design technique is a general
approach to solving problems algorithmically
that is applicable to a variety of problems
from different areas of computing.

• Learning algorithm design techniques is of
utmost importance for the following reasons.
• First, they provide guidance for designing
algorithms for new problems.
• Second, algorithms are the cornerstone of
computer science.

Designing an Algorithm and Data
Structures
• Designing an algorithm for a particular
problem may be a challenging task.
• Some design techniques can be simply
inapplicable to the problem in question.
• Sometimes, several techniques need to
be combined, and there are algorithms that
are hard to pinpoint as applications
of the known design techniques.

• One should pay close attention to choosing
data structures appropriate for the operations
performed by the algorithm.
Algorithms+ Data Structures = Programs
• In the new world of object-oriented
programming, data structures remain crucially
important for both design and analysis
of algorithms.

Methods of Specifying an
Algorithm
• Once you have designed an algorithm, you need
to specify it in some fashion.
• These are the two options that are most widely
used nowadays for specifying algorithms.
(1) Natural language - Programming language with
which one usually talks to a computer.
• Writing clear description of algorithms
surprisingly difficult.

(2) Pseudocode - is a mixture of a natural language
and programming language like constructs.
• Pseudocode is usually more precise than natural
language
• In the earlier days of computing, the dominant
vehicle for specifying algorithms was a flowchart
- a method of expressing an algorithm by a
collection of connected geometric shapes
containing descriptions of the algorithm’s steps.
• This representation technique has proved to be
inconvenient for all but very simple algorithm

Proving an Algorithm’s
Correctness
• Once an algorithm has been specified, you
have to prove its correctness.
• That is, you have to prove that the algorithm
yields a required result for every legitimate
input in a finite amount of time.
• For some algorithms, a proof of correctness is
quite easy; for others, it can be
quite complex.
• A common technique for proving correctness
is to use mathematical induction.

Analyzing an Algorithm
• We usually want our algorithms to possess
several qualities.
• After correctness, the most important is
efficiency.
• There are two kinds of algorithm efficiency:
time efficiency, indicating how fast the
algorithm runs, and space efficiency,
indicating how much extra memory it uses.

• Another desirable characteristic of an
algorithm is simplicity
• Simplicity is an important algorithm
characteristic because simpler algorithms are
easier to understand and easier to program
and usually contain fewer bugs.
• Sometimes simpler algorithms are also more
efficient than more complicated alternatives.
Unfortunately, it is not always true

• Yet another desirable characteristic of an
algorithm is generality.
• There are, in fact, two issues here: generality
of the problem the algorithm solves and the
set of inputs it accepts.
• On the first issue, there are situations, where
designing a more general algorithm is
unnecessary or difficult or even impossible.
For example, it is unnecessary to sort a list of
n numbers to find its median.

• On the second issue, the set of inputs, your
main concern should be designing an
algorithm that can handle a set of inputs that
is natural for the problem at hand.
• If you are not satisfied with the algorithm’s
efficiency, simplicity, or generality, you must
return to the drawing board and redesign the
algorithm.

Coding an Algorithm
• Most algorithms are destined to be ultimately
implemented as computer programs.
• When implementing algorithms as programs to
be used in actual applications, you should
provide verifications.
• Implementing an algorithm correctly is necessary
but not sufficient
• Another important issue of algorithmic problem
solving is the question of whether or not every
problem can be solved by an algorithm.

Important Problem Types
The most important problem types are:
Sorting
Searching
String processing
Graph problems
Combinatorial problems
Geometric problems
Numerical problems

Sorting
• The sorting problem is to rearrange the items
of a given list in nondecreasing order.
• As a practical matter, we usually need to sort
lists of numbers, characters from an alphabet,
character strings, and, most important,
records similar to those maintained by schools
about their students, libraries about
their holdings, and companies about their
employees.

• A specially chosen piece of information is
called a key.
• Computer scientists often talk about sorting a
list of keys even when the list’s items are not
records but, say, just integers.
• computer scientists have discovered dozens of
different sorting algorithms.

• Although some algorithms are better than
others, there is no algorithm that would be
the best solution in all situations.
• Some of the algorithms are simple but
relatively slow, while others are faster but
more complex;
• some work better on randomly ordered
inputs, while others do better on almost-
sorted lists;
• some are suitable only for lists residing in the
fast memory, while others can
be adapted for sorting large files stored on a
disk; and so on.

Two properties of sorting algorithms are:
(1) A sorting algorithm is called stable if it
preserves the relative order of any two equal
elements in its input.
(2) The amount of extra memory the algorithm
requires.
An algorithm is said to be in-place if it does
not require extra memory, except, possibly,
for a few memory units.

Searching
• The searching problem deals with finding a
given value, called a search key, in a given set
• There are plenty of searching algorithms to
choose from.
• For searching, too, there is no single algorithm
that fits all situations best.
• Some algorithms work faster than others but
require more memory; some are very fast but
applicable only to sorted arrays; and so on.

• Unlike with sorting algorithms, there is no
stability problem, but different issues arise.
• Organizing very large data sets for efficient
searching poses special challenges with
important implications for real-world
applications.

String Processing
• A string is a sequence of characters from an
alphabet.
• Strings comprise letters, numbers, and special
characters; bit strings.
• string-processing algorithms have been
important for computer science for a long
time
• Searching for a given word in a text – String
matching

Graph Problems
• A graph can be thought of as a collection of
points called vertices, some of which are
connected by line segments called edges.
• Graphs can be used for modeling a wide
variety of applications, including
transportation, communication, social and
economic networks, project scheduling, and
games.

Examples:
(1) - The travelling salesman problem (TSP) is the
problem of finding the shortest tour through
n cities that visits every city exactly once.
(2) The graph-coloring problem seeks to assign
the smallest number of colors to the vertices
of a graph so that no two adjacent vertices
are the same color.

Combinatorial Problems
• These are problems ask us to find a
combinatorial object—such as a permutation,
a combination, or a subset—that satisfies
certain constraints.
• Some combinatorial problems can be solved
by efficient algorithms.

• Generally speaking, combinatorial problems
are the most difficult problems in computing,
from both a theoretical and practical
standpoint.
• First, the number of combinatorial objects
typically grows extremely fast with a
problem’s size even for moderate-sized
instances.
• Second, there are no known algorithms for
solving most such problems exactly in an
acceptable amount of time.

Geometric Problems
• Geometric algorithms deal with geometric
objects such as points, lines, and polygons.
• Today people are interested in geometric
algorithms with quite different applications in
mind, such as computer graphics, robotics,
and tomography.
• a technique for displaying a representation of a cross section
through a human body or other solid object using X-rays or
ultrasound.

Examples:
• The closest-pair problem is self-explanatory:
given n points in the plane, find the closest
pair among them.
• The convex-hull problem asks to find the
smallest convex polygon that would include all
the points of a given set.

Numerical Problems
• Numerical problems are problems that involve
mathematical objects of continuous nature:
solving equations and systems of equations,
computing definite integrals, evaluating
functions, and so on.
• These problems can be solved only
approximately

• Another principal difficulty is such problems
typically require manipulating real numbers,
which can be represented in a computer only
approximately.

CS8461 - Design and Analysis of Algorithms

More Related Content

What's hot

Similar to CS8461 - Design and Analysis of Algorithms

Recently uploaded

In this document

CS8461 - Design and Analysis of Algorithms