CMSC 56 | Lecture 8: Growth of Functions

Growth of Functions
Lecture 8, CMSC 56
Allyn Joy D. Calcaben

Used to measure how well a computer algorithm scales as the
amount of data involved increases.
Big – O Notation

Used to measure how well a computer algorithm scales as the
amount of data involved increases.
Not a measure of speed but a measure of how well an
algorithm scales.
Big – O Notation

Big – O Notation is also known as Landau Notation
Named after Edmund Landau
He didn’t invent the notation, he popularized it
Ironic that he popularized Big – O as he was known to be an
exact and meticulous mathematician
Landau Notation

If n = 1: 45n3 + 20n2 + 19 =
Example

If n = 1: 45n3 + 20n2 + 19 = 84
Example

If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 =
Example

If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
Example

If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
If n = 10: 45n3 + 20n2 + 19 =
Example

If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
If n = 10: 45n3 + 20n2 + 19 = 47, 019
45,000 + 2,000 + 19
Example

If n = 100: 45n3 + 20n2 + 19
Removing Unimportant Things

If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19

If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19
= 45, 200, 019

If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19
= 45, 200, 019
= O(N3)

“Leaving away all the unnecessary things that we don’t care
about when comparing algorithms.”
Big – O Notation

2n2 + 23 vs. 2n2 + 27
Essentially they have the same running time.
This is another simplification.
Solution

Definition 1
Given A = f(n), B = g(n) grows faster than f(n), then
there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’)
for all n’’ > n, we have g(n’’) ≥ f(n’’).
Big – O Notation

Definition 1
Given A = f(n), B = g(n) grows faster than f(n), then
there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’)
for all n’’ > n, we have g(n’’) ≥ f(n’’).
Since we don’t want to care about constants, we include c so that we can
scale g(n). That is, even if we multiply g(n) with a constant c, and it still
outgrows f(n), we can still say that g(n) runs at least as fast as f(n).
Big – O Notation

If the two conditions hold true, we say that f(n) ϵ O(g(n))
f(n) is contained in Big O of g(n)
Big O means that g(n) is a function that runs at least as fast as f(n).
Big – O Notation

f(n)
g(n)h(n)
n
Comparing the Growth Rates

Get ¼ and answer. Which of the ff. is true? Write True or False
for each number.
1. h(n) ϵ O(f(n))
2. h(n) ϵ O(g(n))
3. g(n) ϵ O(f(n))
4. g(n) ϵ O(h(n))
5. f(n) ϵ O(g(n))
6. f(n) ϵ O(h(n))
Comparing the Growth Rates
f(n)
g(n)h(n)
n

Allow us to concentrate on the fastest growing part of the
function and leave out the involved constants
Importance

Simply about looking at which part of the function grows the
fastest.
A convenient way of describing the growth of the function
while ignoring the distracting and unnecessary details
Big – O Notation

f(n) = O(g(n)) is also an accepted notation
f(n) ϵ O(g(n)) is more accurate, since O(g(n)) is a set of functions.
Big – O Notation

Algorithm A RT: 3n2 – n + 10
Algorithm B RT: 2n – 50n + 256
Which algorithm is preferable in general?
Exercise

Which algorithm is preferable in general?
But, this is for general cases.
For small inputs, algorithm B might be equal to algorithm A or
better.
Solution

If n = 5: 3n2 – n + 10 = 80
2n – 50n + 256 = 38
If n = 100: 3n2 – n + 10 = 29910
2n – 50n + 256 = 1267650600228229401496703200632
Analysis

1. 3n + 1 ϵ _______
2. 18n2 – 50 ϵ _______
3. 9n4 + 18 ϵ _______
4. 30n6 + 2n + 123 ϵ _______
5. 2nn2 + 30n6 + 123 ϵ _______
More Example

1. 3n + 1 ϵ O(n)
2. 18n2 – 50 ϵ O(n2)
3. 9n4 + 18 ϵ O(n4)
4. 30n6 + 2n + 123 ϵ O(2n)
5. 2nn2 + 30n6 + 123 ϵ O(2n n2)
More Example

O(1)
O(log N)
O(N)
O(N log N)
O(N2)
O(N3)
O(2n)
Big – O Notation

3n + 1 ϵ O(n2)
True / False?
More Examples

3n + 1 ϵ O(n2)
True / False? TRUE, but NOT TIGHT
More Examples

18n2 – 50 ϵ O(n3)
True / False?
If true, tightly bound / not?
If not tightly bound, what is the tighter bound?
More Examples

18n2 – 50 ϵ O(n3)
True / False? TRUE
If true, tightly bound / not? NOT TIGHTLY BOUND
If not tightly bound, what is the tighter bound? O(n2)
More Examples

Write Correct / Incorrect, Tight / Not Tight
1. 4n2 - 300n + 12 ∈ O(n2)
2. 4n2 - 300n + 12 ∈ O(n3)
3. 3n + 5n2 - 3n ∈ O(n2)
4. 3n + 5n2 - 3n ∈ O(4n)
5. 3n + 5n2 - 3n ∈ O(3n)
6. 50•2nn2 + 5n - log(n) ∈ O(2n)
On Your Seats (1/4)

Write Correct / Incorrect, Tight / Not Tight
1. 4n2 - 300n + 12 ∈ O(n2) CORRECT, TIGHT
2. 4n2 - 300n + 12 ∈ O(n3) CORRECT, NOT TIGHT
3. 3n + 5n2 - 3n ∈ O(n2) INCORRECT, NOT TIGHT
4. 3n + 5n2 - 3n ∈ O(4n) CORRECT, NOT TIGHT
5. 3n + 5n2 - 3n ∈ O(3n) CORRECT, TIGHT
6. 50•2nn2 + 5n - log(n) ∈ O(2n) INCORRECT, NOT TIGHT
On Your Seats (1/4)

Constant
Description: Statement
Example: Adding two numbers
c = a + b
O(1)

Algorithm that will execute at the same amount of time
regardless of the amount of data.
Or simply,
Code that will execute at the same amount of time no matter
how big the array is.
O(1)

Example: Adding element to array
list.append(x)
O(1)

Logarithmic
Description: Divide in Half
Data is decreased roughly 50% each time through the algorithm
O(log N)

Example: Binary Search
O(log N)

Linear
Description: Loop
Time to complete will grow in direct proportion to the amount of
data.
Example: Factorial of N using Loops
for i in range (1, N+1):
factorial *= i
O(N)

Example: Finding the Maximum
O(N)

Example: Finding the Maximum
Look in exactly each item in the array
Big difference if it was a 10 – item array vs. a 10
thousand – item array
O(N)

Linearithmic / Loglinear
Description: Divide and Conquer
O(n log N)

Example: Merge Sort
O(n log N)

Quadratic
Description: Double Loop
Time to complete will grow proportional to the square of amount
of data
O(N2)

Cubic
Description: Triple Loop
O(N3)

Exponential
Description: Exhaustive Search (Brute Force*)
O(2n)

Given an algorithm that runs in ___ time, the computer can solve a
problem size of _____ in a matter of minutes.
Practical Implications

Given an algorithm that runs in ___ time, the computer can solve a
problem size of _____ in a matter of minutes.
Constant: any
Logarithmic: any
Linear: billions
Loglinear: hundreds of millions
Quadratic: tens of thousands
Cubic: thousands
Exponential: 30
Practical Implications

count = 0
for each character in string:
if character == ‘a’:
count += 1
Best Case, Worst Case

Best case: 2n + 1
Worst case: 3n
Running Time

We observe the ff. from the algorithm:
1. Each character in the string is considered at most once
2. For each character in the input string, a constant no. of steps
is performed (2 or 3)
Running Time

We observe the ff. from the algorithm:
1. Each character in the string is considered at most once
2. For each character in the input string, a constant no. of steps
is performed (2 or 3)
With Big – O, we say c1n + c2 ϵ O(n)
We say that the running time of the algorithm is O(n) or linear
time
Running Time

CMSC 56 | Lecture 8: Growth of Functions

In this document