Data Structure: Algorithm and analysis

DATA STRUCTURES
ALGORITHM AND ANALYSIS
Rajdeep Chatterjee
Assistant Professor
School of Computer Engineering
KIIT University

OVERVIEW
 Algorithm
 Analysis of Algorithm
 Space Complexity
 Time Complexity
 Step Counts
 Asymptotic Notations
 Big-oh Notations
 Rate of Growth
 Types of Time Complexity
 Best Case Complexity
 Worst Case Complexity
 Average Case Complexity

ALGORITHM
 Algorithm is a finite set of well defined
computational instructions written in a proper
sequence in order to solve a problem.
 Criteria-
 Input
 Output
 Definiteness
 Finiteness
 Effectiveness

EXAMPLES
 ADD: INTEGER a, b, c
1. Read a & b
2. add a & b
3. store the sum of a & b to c
4. print c
 In C, exp(1)
void add(int a, int b){
int c;
c=a+b;
printf(“%d”,c);
}
In C, exp(2)
int add(int a, int b){
int c;
c=a+b;
return c;
}

ANALYZING AN ALGORITHM
 Why is it important?
 Predict the feasibility requirement of your code
(algorithm).
 Usual requirements
 Execution time
 Memory space
 The complexity of the algorithm is determined in
terms of space and time.
 Space complexity ← execution time
 Time complexity ← memory space

SPACE COMPLEXITY
 Amount of computer memory required during
program execution.
 Instruction space – space required to store the
code.
 Fixed space requirement – input independent
 Variable space requirement – input dependent
 S = F(I) otherwise S = C

EXAMPLES
 Exp(1)
int main(){
printf(“KIIT UNIVERSITY”);
}
 Exp(2)
void bubble(int a[], int n){
int i,j;
for(i=0;i<n-1;i++){
for(j=0;j<n-1-i;j++)
if(a[j]>a[j+1])
swap (a[j],a[j+1])
}
}

TIME COMPLEXITY
 The amount of computer time that it needs to run
to completion.
 It is determined without considering the following
information-
 The machine we are executing,
 Its machine language instruction set,
 The time required by each machine instruction,
 The translation, a compiler/interpreter will make from
the source code to machine language.

EXAMPLES
 Exp(1)
x=x+1;
 Exp(2)
for(i=1 ; i<=n ; i++)
x=x+1;
 Exp(3)
for(i=1 ; i<=n ; i++)
for(j=1 ; j<=n ; j++)
x=x+1;
Our concern should be the order of magnitude/ growth of an algorithm
for an input n.
Exp(4)
int main(){
}

STEP COUNT
Instructions / code Step Count
x=x+1 constant, c
int main(){
return 0;
}
?
for(i=1 ; i<=n ; i++)
x=x+1;
n
for(i=1 ; i<=n ; i++)
for(j=1 ; j<=n ; j++)
x=x+1;
n2

EXAMPLE - 1
1. int i, f=1, n=5; -------------- 1 time
2. for(i=1 ; i<=n ; i++) -------------- n+1 times
3. f=f*i; -------------- n times
4. printf(“%d”,f); -------------- 1 time
n i f=f*i , f=1
5 1 1*1=1
2 1*2=2
3 2*3=6
4 6*4=24
5 24*5=120
6
T(n) = 1+(n+1)+n+1 = 2n+3 ≈ O(n)
if n=0 then T(n) = 1+1 ≈ O(1)

EXAMPLE - 2
1. int a=0,b=1,c, i, n=5; ------------- 1
2. printf(“%d %d”,a,b); ------------- 1
3. for(i=1 ; i<=n-2 ; i++ ){ ------------- (n-2)+1 = (n-1)
4. c=a+b; ------------- (n-2)
5. printf(“%d”,c); ------------- (n-2)
6. a=b; ------------- (n-2)
7. b=c; } ------------- (n-2)
i a b c=a+b
1 0 1 1
2 1 1 2
3 1 2 3
4
T(n)= 1+1+(n-1)+2*(n-2) = 2+n-1+2n-4 = 3n-3 ≈ O(n)

EXAMPLE - 3
1. int i, j, n=5, a[5]={12, 2, 51, 35, 7};
2. for(i=0;i<n-1;i++){
3. for(j=0;j<n-1-i;j++)
4. if(a[j]>a[j+1]){
5. a[j] = a[j] + a[j+1];
6. a[j+1] = a[j] - a[j+1];
7. a[j] = a[j] - a[j+1]; }
8. for(i=0 ; i<n ; i++)
9. printf(“%d”, a[i]);
T(n) = ?

ASYMPTOTIC NOTATIONS
 O (Big-oh) Notation
Consider a function f(n) which is non-negative for all
integers n=0. we say that f(n)is Big-oh g(n), which we
write f(n)=O(g(n)), if there exits an integer n0 and a
constant c>0 such that for all integers n=n0, f(n)=cg(n).
 In other words,
O(g(n))={f(n): ∃ positive constants c and n0 ∋ 0 ≤ f(n) ≤cg(n) ∀ 𝑛 ≥ 𝑛0 }
 O-notation to give an upper bound on a function to within a
constant factor.

EXAMPLE
 Two students, X & Y
 Let, performance of X & Y are TX and TY respectively,
 Conclusion is for a large n, TX(n) ≻ TY(n)
 So, who is a better professional ?
 Ans. X
 When X outperforms Y and by what factor ?
 Ans. n0 and c
# Events X Y TX ∼ TY
1 10th 70% 85% TX(1) ≺ TY(1)
2 (10+2)th 65% 78% TX(2) ≺ TY(2)
3 B.Tech 8.8 cgpa 8.7 cgpa TX(3) ≻ TY(3)
4 M.Tech GATE rank 120 GATE rank 1700 TX(4) ≻ TY(4)
5 Ph.D 5 publications 2 publications TX(5) ≻ TY(5)
6 Job 20 lakhs/p.a. 10 lakhs/p.a. TX(6) ≻ TY(6)

EXAMPLE
 Suppose C=1 then, f(n)=6n+135
f(n) = cn2
 6n +135 = cn2 = n2 (since c=1)
 0 = n2 - 6n -135
 0 = (n-15)(n+9)
Since (n+9) > 0 for all values n ≥ 0, then (n-15)=0
 n0=15
 Find n0 when c=2 & c=4.

NOTATIONS
Big-oh : 0≤ f(n) ≤ cg(n)
Theta : 0≤ 𝑐1 𝑔 𝑛 ≤ f(n) ≤ c2 𝑔 𝑛 )
Omega : 0 ≤ cg(n) ≤ f(n)

RATE OF GROWTH
𝒍𝒐𝒈 𝟐 𝒏 n n𝒍𝒐𝒈 𝟐 𝒏 𝒏 𝟐 𝒏 𝟑 𝟐 𝒏
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4096 65536

TYPES OF TIME COMPLEXITY
 Best Case Time complexity
 Exp(1) – Linear search/Binary search
 Linear search, key=3
 a[0]==key, T(n)=O(1)
 Binary search, key=24
 m=(0+4)/2 = 2
 a[2]==key, T(n)=O(1)
3 5 24 45 78

 Worst Case Time complexity
 Linear search, key=78 / 80
 a[0]==key, T(n)=O(n) / O(n+1) ≈ O(n)
 Binary search, key=78 / 80
 m=(0+4)/2 = 2 …
 a[2]!=key,…
 finally T(n)=O(log2 𝑛) ….(*)
3 5 24 45 78

 Average Case Time complexity
 Linear search, key=24
 a[0]==key, T(n)=O((n+1)/2) ≈ O(n)
 There are n cases that can occur, i.e. find at the first place,
the second place, the third place and so on up to the nth
place. If found at the ith place then i comparisons are
required. Hence the average number of comparisons over
these n cases is:
 average = (1+2+3.....+n)/n = (n+1)/2
 where the result was used that 1+2+3 ...+n is equal to
n(n+1)/2.
 Binary search, key=5
 m=(0+4)/2 = 2, m=(0+1)/2=0, m=(1+1)/2=1
 a[1]==key, finally T(n) ≈ O(log2 𝑛) ….(*)
3 5 24 45 78

SORTING ALGORITHMS
Algorithm
Data
Structure
Time
Complexity:
Best
Time
Complexity:
Average
Time
Complexity:
Worst
Space
Complexity:
Worst
Quick Sort Array O(n log(n)) O(n log(n)) O(n2) O(log(n))
Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)
Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)
Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)
Bubble sort Array O(n) O(n2) O(n2) O(1)
Insertion
sort
Array O(n) O(n2) O(n2) O(1)
Selection
sort
Array O(n2) O(n2) O(n2) O(1)

Data Structure: Algorithm and analysis

More Related Content

What's hot

Viewers also liked

Similar to Data Structure: Algorithm and analysis

More from Dr. Rajdeep Chatterjee

Recently uploaded

Data Structure: Algorithm and analysis