2-Algorithms and Complexit data structurey.pdf

Algorithms Design
and
Complexity Analysis

Algorithm
A well defined Sequence of statements which solves the problem logically
A well defined sequence of English statements can also be an algorithm
Algorithm does not belong to any particular language
A problem can have many algorithms
Algorithm always takes zero or more input and produces some desired output.
Solution
Steps
(Algorithm)
INPUT
DESIRED
OUTPUT

Input: Zero or more quantities are supplied.
Definiteness: Each instruction of the algorithm
should be clear and unambiguous.
Effectiveness: Statements should be basic
enough, not very complex
Finiteness: The execution must be finished
after finite number of steps
Output: Must provide desired output
An algorithm can be expressed in three ways:-
(i) In the form of pseudo code
(ii) In the form of a flowchart
(iii) In the form of a program in any specific
programming language
Properties of algorithms

4
A program is an implementation of an algorithm to be
run on a specific computer.
An algorithm is a more abstract way to solve a
problem; it does not deal with machine specific
details.
Program Vs. Algorithm

Program Vs. Software
Computer Program :
A well defined set of instructions written in any programming language to perform
some specific task.
Is Program itself a Software ?
NO, Program is small part of software.
A software generally comprises a group of programs (source codes),
Documentation, Test cases, Input and Output Description etc.

Program design using algorithm
Problem Algorithm Program
Analysis Particular
Language
Two phase process –
1. Problem analysis:
Analyze and understand the user defined problem.
Write algorithm using basic algorithmic constructs.
2. Implementation:
Translate the algorithm into a desired programming language.

Sub-algorithm
A sub-algorithm is a part of another algorithm.
The relationship between an algorithm and a sub-algorithm is similar to the relationship
between a main program (function) and a sub-program (sub-function) in a programming
language.
When the problem is very complex, it is divided into several independent sub-problems.
Each sub-problem can be solved independently using sub-algorithm.
The use of sub-algorithm allows efficient task management
✔ John solves problem 1 and Mary solves problem 2
Whole project is not completely stopped if one of the smaller problems cannot be solved
immediately.
✔ Solve the other problems
✔ Then research and solve the difficult part of the problem.
We may be able to reuse the solution to the smaller problem in another large problem
Sub-algorithms can be combined together to form the solution for the entire problem.

Approaches for designing algorithms
Greedy Approach –At each step, it selects the best possible solution.
✔ A greedy algorithm, as the name suggests, always makes the choice that seems to be
the best at that moment.
✔ This means that it makes a locally-optimal choice in the hope that this choice will lead
to a globally-optimal solution.
✔ Examples:. Shortest path algorithm, Kruskal's algorithm and Prim's algorithm for
finding minimum spanning trees.
Divide and Conquer –The divide and conquer approach involves the following steps at
each level.
✔ Divide − The original problem is divided into sub-problems.
✔ Conquer − The sub-problems are solved recursively.
✔ Combine − The solutions of the sub-problems are combined together to get the solution
of the original problem.
✔ Examples: Recursive Quick sort and merge sort etc.

Dynamic Programming (DP)- The approach of the Dynamic programming is similar
to divide and conquer.
✔ The difference is that for the recursive function calls with the same result, instead of
calling them again, a DP approach tries to store the result in a data structure in the form
of a table and retrieve the results from the table.
✔ Thus, the overall time complexity can be reduced in DP.
✔ Term “Dynamic” means this approach dynamically decides, whether to call a function or
to retrieve the values from the table for solving a sub-problem.
✔ Examples: 0-1 Knapsack, subset-sum problem, and recursive algorithm for Fibonacci
Series are some examples of dynamic programming.
Approaches for designing algorithms

A good algorithm is an efficient algorithm
What is an efficient algorithm?
An efficient algorithm means which has small running time and
also it takes less memory.
Hence, the time and space are two measures of efficiency for
an algorithm.
What is a Good Algorithm?

Algorithm analysis
The goal of the analysis of algorithms is to compare algorithms (or solutions) mainly
in terms of running time and memory requirement.
Assumption –
1. The algorithms to compared are correct.
2. The algorithm that probably takes lesser time is a better one, if the space requirement is
constant.
Can you suggest any approach to compare the algorithms?
Approach 1: Empirical analysis (Experimental Study) approach–
1. Implement the algorithm into any programming language.
2. Run the program with inputs of varying size and compositions.
3. Use functions like clock() , System.currentTimeMillis(),etc. to get an accurate measure
of the actual running time.

12
• Three basic Limitations:
1. It is necessary to implement the algorithm into a particular
programming language , which may be a difficult or time
consuming task.
2. Results may not be indicative of the running time on other
inputs that are not included in the experiment.
3. To compare two algorithms, the same hardware and software
environments must be used.
Limitations of Empirical analysis approach

Approach 2: Asymptotic analysis approach
Method :
✔Develop a high level description of an algorithm.
✔From this high level description, count the number of primitive
(basic) operations of the algorithm.
Advantages of this approach:
1. No need to implement algorithm on any system.
2. This approach considers all possible input instances as we
consider the worst case
3. This approach also evaluate the efficiency of the algorithm which
is independent of the platform we are using.

Constructs of algorithm
1. Identifying Numbers – Each statement can be assigned a identifying number.
2. Comment – Comment may be used to make algorithm easily readable and should not
assigned any step number.
3. Variable – provide meaningful variable names
4. Input/output – For taking input from user Read / Input construct and for displaying
/printing any message or any variable value Print/Write construct can be used.
5. Control Structure –a way to specify flow of control in algorithms
Mainly three types:
Sequential logic– Write numbered steps sequentially.
Selection logic – To select statement from many statements, If – else or its variations can
be used.
Iteration logic – To repeat some statements, loops like while, for and do….while can be
used.

Example: Find Largest Element in an array
Algorithm Find_Largest(x[], n)
1. let max = x[0]
2. for i = 1 to n-1
3. if x[i] > max then,
4. max = x[i]
5. Print max

Steps to analyze algorithms using approach 2
Primitive operations:
✔ Assigning a value to a variables, like X=10, X=Y, etc.
✔ Input output related operations, like Input, Display, Print.
✔ Arithmetic and logical operations, Expression evaluation, like C=A+B*2
✔ Control statements, like If-else, For, while,
✔ Function and return statements, like Find_Fact(), return X.
1. First identify primitive (basic ) operations in the algorithm.
2. Count all corresponding primitive operations that are executed by an
algorithm as follows:
✔ a+b+b*c+(a+b)/(a-b) 🡪 one step;
✔comments 🡪 zero steps;
✔while (<expr>)🡪 step count equal to the number of times <expr> is
executed.
✔for i=<expr> to <expr1> 🡪 step count equal to number of times
<expr1> is checked.
3. Find the step count function or time complexity function in term of n by
adding all the steps.

Time complexity: Example
Statements Cost/Exec. Freq. Total
1. Algorithm Sum(a[],n) 0 - 0
2. S = 0.0; 1 1 1
3. for i=1 to n do 1 n+1 n+1
4. s = s+a[i]; 1 n n
5. return s; 1 1 1
2n+3
Hence, step count function or growth rate function f(n) =2n+3

Find largest: Elements arranged in Descending order
Statements Cost/
Exec.
Freq. Total
1. Algorithm Find_Largest(x[], n) 0 - 0
2. max = x[0] 1 1 1
3. for i = 1 to n-1 1 n n
4. if x[i] > max then 1 n-1 n-1
5. max = x[i] 1 0 0
6. Print max 1 1 1
2n + 1
Hence, step count function or growth rate function f(n) =2n+1

Find largest: Elements arranged in ascending order
Statements Cost/
Exec.
Freq. Total
1. Algorithm Find_Largest(x[], n) 0 - 0
2. max = x[0] 1 1 1
3. for i = 1 to n-1 1 n n
4. if x[i] > max then 1 n-1 n-1
5. max = x[i] 1 n-1 n-1
6. Print max 1 1 1
3n
Hence, step count function or growth rate function f(n) =3n

Hence, step count function or growth rate function
f=2rows*cols+2rows+1

Find the step count function
1 Algorithm Sum(a[],n,m)
2 {
3 for i=1 to n do;
4 for j=1 to m do
5 s = s+a[i][j];
6 return s;
7 }

Hence, step count function or growth rate function f(n) = 2nm+2n+2
Statements C/E Freq. Total
1 Algorithm Sum(a[],n,m) 0 - 0
2 { 0 - 0
3 for i=1 to n do; 1 n+1 n+1
4 for j=1 to m do 1 n(m+1) n(m+1)
5 s = s+a[i][j]; 1 nm nm
6 return s; 1 1 1
7 } 0 - 0
2nm+2n+2

Time Complexity
Time complexity :Time requirement of an algorithm in terms of size of input.
In general, the time complexity of an algorithm can be analyzed in three cases:
Worst case
o Defines the input for which the algorithm takes maximum time.
o Input is the one for which the algorithm runs the slowest.
Best case
o Defines the input for which the algorithm takes least time.
o Input is the one for which the algorithm runs the fastest.
Average case
o Provides a prediction about the running time of the algorithm
o Assumes that the input is random

Conclusion
We have already seen that an algorithm can be represented in the
form of an expression.
That means
Worst case
o Defines the input for which the algorithm takes huge time.
o Input is the one for which the algorithm runs the slowest.
Best case
o Defines the input for which the algorithm takes lowest time.
o Input is the one for which the algorithm runs the fastest.
Average case
o Provides a prediction about the running time of the algorithm
o Assumes that the input is random

Time Complexity
Simple step count functions can be compared easily.
Consider f(n) = (n2
+ 5), g(n)= log (n2
+ 1)
Which is better f(n) or g(n)?
Complex step count functions are difficult to compare.
Therefore, these step count functions can be simplified using some mathematical methods
such as Big-O, Big-Omega etc.
For comparing, ‘rate of growth’ of step count function is easy and sufficient.
‘Rate of growth’ describes how the runtime will increase as the input size grows

Asymptotic Notations
Asymptotic notation is a way of expressing the cost of an algorithm in terms of input size.
Goal of asymptotic notation is to simplify analysis by getting rid of unnecessary
information
Big –O: Big-O of a function gives us ‘rate of growth’ of the step count function f(n), in terms
of a simple function g(n), which is easy to compare.
Definition: for the given functions f(n) and g(n)
f(n)= O(g(n)) | if there exists positive constants c and no
such that,
0<=f(n)<=cg(n) for all n>=no
}
Defines a tight upper bound for a function.
f(n)= O(g(n)) implies that
▪ g(n)grows at least as fast as f(n) or
▪ f(n) is of the order at most g(n)

Big –O Notation-Example 1
Given f(n) = 5n and g(n) = n then show that f (n)= O(g(n))
Solution:
Need to show that there exist two positive constants C and no
Such that 5n<=Cn for all n>= no
Clearly for C=5 above condition is true for all n>=1 (i.e. no
=1)
Hence it is proved that f(n) = O(g(n))
Note : We could choose a larger values of C such as 6, however to make it tighter upper
bound we usually try to find the smaller value.

Show that f(n) = 4n + 5= O(n)
Solution: To show that f(n) = 4n + 5 = O(n), we need to produce a constants c and no
such
that: f(n) <= c * n for all n>= no
If we try c = 4, this doesn't work because 4n + 5 is not less than 4n.
We need c to be at least 9 to cover all n>=1. Hence [c, no
]=[9,1]
Therefore, If no
= 1, c has to be 9, but c can be smaller for greater values of no
(if no
= 100, c
can be 5).

Big –O Notation Example
>> Eg :
1. 3n + 2 = O(n)
3n + 2 ≤ 4n for all n ≥ 2.
So C=4 and n0=2
2. 3n + 3 = O(n)
3n + 3 ≤ 4n for all n ≥ 3.
So C=4 and n0=3
3. 100n + 6 = O(n)
100n + 6 ≤ 101n for all n ≥ 6.
So C=101 and n0=6

If f(n) = n2
: prove that f(n) ≠ O(n)
• To prove this, we must show that there cannot exist the constant c and no
that
satisfies the big-O definition.
This can be proved by contradiction.
Suppose there are two positive constants c and no
that works; then, by the
definition of big-O:
Supposition: n2
≤ c * n for all value of n>= no
• Suppose n is any positive real number greater than c,
then: n * n > c * n, or n2
> c * n.
That means there exists a real number n such that n2
> c * n.
This contradicts the supposition, so the supposition is false
There is no such C that can work for all n>= no
:
f(n) ≠ O(n) where f(n) = n2

Asymptotic Notations (Cont…)
Big – Ω
Definition:
for the given functions f(n) and g(n)
f(n)= Ω (g(n)) | if there exists positive constants c and no
such that,
0<=cg(n)<=f(n) for all n>=no
}
▪ Defines tight lower bound for a function
▪ f(n)= Ω (g(n)) implies that
▪ g(n) grows at most as fast as f(n) or
▪ f(n) is of the order at least g(n)

Big –Ω Notation Example
1. 3n + 2 = Ω(n)
3n + 2 ≥ 3n for all n ≥ 1.
So c=3 and n0=1
2. 3n + 3 = Ω(n)
3n + 3 ≥ 3n for all n ≥ 1.
So c=3 and n0=1
3. 100n + 6 = Ω(n)
100n + 6 ≥ 100n for all n ≥ 1.
So c=100 and n0=1

Asymptotic Notations (Cont…)
Big – θ
Definition:
for the given functions f(n) and g(n)
f(n)=θ(g(n)) | if there exists positive constants c1 and c2 and no
such that,
0<=c1g(n)<=f(n)<=c2g(n) for all n>=no
}
▪ f(n)=θ(g(n)) iff f(n)=O(g(n)) and f(n)= Ω(g(n))
▪ Defines tight lower bound and upper bound for a function, at the same time

Big –θ Notation Example
1. 3n + 2 = Θ(n)
3n + 2 ≥ 3n for all n ≥ 2.
3n + 2 ≤ 4n for all n ≥ 2.
So, C1 = 3, C2 =4 & n0 =2.

An interesting observation
Very long

Worst Case Analysis (Usually Done)
In the worst case analysis, the upper bound on running time of an algorithm is
calculated.
For the worst case analysis, we must know the case that causes maximum
number of operations to be executed.
For Linear Search, the worst case happens when the element to be searched is
not present in the array or it is present at the last location.
Therefore, the worst case time complexity of linear search is: O(n)

Best Case Analysis (Bogus)
In the best case analysis, lower bound on running time of an algorithm is
calculated.
For the best case, we must know the case that causes minimum number of
operations to be executed.
In the linear search problem, the best case occurs when the element to be
searched is present at the first location.
The number of operations in the best case is constant (not dependent on n).
So time complexity of linear search in the best case would be O(1)

Average Case Analysis (Sometimes done)
In average case analysis, we take all possible inputs and calculate
computing time for all of the inputs.
Sum all the calculated values and divide the sum by total number
of inputs.

Basic asymptotic efficiency classes
1 Constant
loglogn double logarithmic
logn Logarithmic
Log2
n log-squared
n Linear
nlogn Log linear
n2
Quadratic
n3
Cubic
2n
Exponential
n! Factorial

Big-O Operations
Summation Rule:
O(f1(n) + f2(n)) = O(max(f1(n), f2(n)))
Example:
1. O(2n
+n3
)=
2. O(n4
+nlogn)=O(n4
)
Product Rule
Suppose T1(n) = O(f1(n)) and T2(n) = O(f2(n)). Then, we can conclude that
T1(n) * T2(n) = O(f1(n) * f2(n)).
Example:
1. O(n) * O (n2
) * O (n3
) = O (n6
)
2. O(n2
)*O(logn)=O(n2
logn)
O(2n
)

Big-O Rules
1. If f(n) is a polynomial of degree d, then f(n) is O(nd
),
i.e.,
1. Drop lower-order terms
2. Drop constant factors
Example 1: f(n)=n6
+3n2
+77 will be O(n6
)
Example 2: f(n)=5000n6
+3n2
+77+34n!+nlogn
f(n)=O( )
2. Use the smallest possible class of functions
Say “2n is O(n)” instead of “2n is O(n2
)”
n!

True or False
N2
= O(N2
)
2N = O(N2
)
N = O(N2
)
N2
= O(N)
2N = O(N)
N = O(N)
true
true
true
false
true
true

True or False
N2
= Ω(N2
)
2N = Ω(N2
)
N = Ω(N2
)
N2
= Ω(N)
2N = Ω(N)
N = Ω(N)
true
false
false
true
true
true

True or False
N2
= Θ(N2
)
2N = Θ(N2
)
N = Θ(N2
)
N2
= Θ(N)
2N = Θ(N)
N = Θ(N)
true
false
false
false
true
true

Big –O Notation Illustrations

Some tips to find out time complexity of algorithms
1) O(1): Time complexity of a function (or set of statements) is considered as O(1) if
it doesn’t contain any loop, recursion and call to any other non-constant time
function.
// set of non-recursive and non-loop statements
For example swap() function has O(1) time complexity.
A loop or recursion that runs a constant number of times is also considered as O(1).
For example the following loop is O(1).
// Here c is a constant
for (int i = 1; i <= c; i++)
{
// some O(1) expressions
}
Like
for (int i = 1; i <= 1000; i++)
{
Print i;
}
Example: Adding an element to the front of a linked list

2) O(n): Time Complexity of a loop is considered as O(n) if the loop variables is
incremented / decremented by a constant amount.
For example following functions have O(n) time complexity.
// Here c is a positive integer constant
for (int i = 1; i <= n; i += c)
{
}
for (int i = n; i > 0; i -= c)
{
}
Like
for (int i = 1; i <= n; i += 1)
{
Print i;
}
Example: linear search

3) O(nc
): Time complexity of nested loops is equal to the number of times the
innermost statement is executed.
For example the following sample loops have O(n2
) time complexity
for (int i = 1; i <=n; i += c)
{
for (int j = 1; j <=n; j += c)
{
}
}
For (i = n; i > 0; i -= c) {
for (int j = i+1; j <=n; j += c)
{ // some O(1) expressions
}
For example Selection sort and Insertion Sort have O(n2
) time complexity.

4) O(Logn) Time Complexity of a loop is considered as O(Logn) if the loop variables
is divided / multiplied by a constant amount.
for (int i = 1; i <=n; i *= c)
{
}
for (int i = n; i > 0; i /= c)
{
}
Like
for (int i = 1; i <=n; i *= 2)
{
}
For example Binary Search has O(Logn) time complexity.

5) O(LogLogn) Time Complexity of a loop is considered as O(LogLogn) if the loop
variables is reduced / increased exponentially by a constant amount.
// Here c is a constant greater than 1
for (int i = 2; i <=n; i = pow(i, c))
{
}
//Here fun is sqrt or cuberoot or any other constant root
for (int i = n; i > 0; i = fun(i))
{
}
Like
for (int i = 2; i <=n; i = i2
)
{
}

How to combine time complexities of consecutive loops?
When there are consecutive loops, we calculate time complexity as sum of time
complexities of individual loops.
for (int i = 1; i <=m; i += c)
{
}
for (int i = 1; i <=n; i += c)
{ // some O(1) expressions
}
Time complexity of above code is O(m) + O(n) which is O(m+n)
If m == n, the time complexity becomes O(2n) which is O(n).

Find the growth rate function and Time Complexity
x = x +1; //constant time
for (i=1; i<=n; i++)
{
m = m + 2; //constant time
}
for (i=1; i<=n; i++) //outer loop executed n times
{
for (j=1; j<=n; j++) //inner loop executed n times
{
k = k+1; //constant time
}
}

Homework
Q1. Rank the following function by their order of growth(increasing order).
(n+1)! , 1 , n1/lgn
, (3/2)n
, log(log n) , sqrt(2)lg n
, (lg n)lg n
, sqrt(lg n)
Q2. Analyse the following fragments of code. Formulate exact expressions for the
running time of each code fragment.
(a)
int sum = 0;
int num = 35;
for (int i=1; i<=n; i++) {
for (int j=1; j<=n; j++) {
num += j*3;
sum += num;
}
}
(b)
int count = 0;
for (int i = n; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count += 1;
(b)
for ( i=0 ; i < n ; i++ )
total = total + i;
for ( j=0 ; j < n ; j++ )
total = total + j;
(d)
int count = 0;
for (int i = 0; i < n; i++)
for (int j = i; j > 0; j--)
count = count + 1;

2-Algorithms and Complexit data structurey.pdf

More Related Content

Similar to 2-Algorithms and Complexit data structurey.pdf

Recently uploaded

2-Algorithms and Complexit data structurey.pdf