Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
UNIT 6 : PROBABILITY FUNCTIONS
Gabriel Asare Okyere (PhD)
August 25, 2018
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Outline of Presentation
1 Moment Generating Functions
2 Characteristic Function
3 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
4 Special Probability Distributions
5 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
6 Useful Continuous Distributions
7 Distributions of Functions of Random Variables
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Moment Generating Functions
The Moment Generating Function (MGF) is a function that
generates moments. It is defined as follows.
Mathematical Definition of MGF
The Moment Generating Function(MGF) of a random variable X,
denoted by MX (t), is defined by
MX (t) = E (e tX ) =
(P
∞
e tx f (x) if X is discrete
f (x) = R ∞x=0 tx
−∞ e f (x)dx if X is continuous
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
A discrete random variable Y has probability mass function given
by
12
P(Y = y ) = (0.6)y (0.4)12−y , y = 0, 1, ..., 12.
y
Find the moment generating function of Y
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
MY (t) = E (e tY )
12
X 12
= e ty (0.6)y (0.4)12−y
y
y =0
12
X 12
= (0.6e t )y (0.4)12−y
y
y =0
Recognizing the sum as a binomial expansion of (0.6e t + 0.4)12 ,
we obtain
MY (t) = (0.6e t + 0.4)12
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
A continuous random variable Y has a p.d.f given by
(
4e −4y , y > 0
f (y ) =
0 elsewhere
Find the moment generating function if Y
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
Z ∞
tY
MY (t) = E (e ) = e ty f (y )dy
−∞
Z ∞
= e ty 4e −4y dy
Z0 ∞
= 4e −4y +ty dy
Z0 ∞
= 4e −(4−t)y dy
0
" #∞
e −(4−t)y 4
= −4 = ,t < 4
4−t 4−t
0
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Theorem
If Mx (t) exists, then for any positive integer k,
d k MX (t) (k)
|t=0 = MX (0) = E (X k )
dt k
d k MX (t) (k)
, where dt k
= MX is the k th derivative of MX (t) with respect
to t.
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Proof
MX (t) = E (e tX )
MX0 (t) = E (Xe tX ) ⇒ MX0 (0) = E (X )
MX00 (t) = E (X 2 e tX ) ⇒ MX00 (0) = E (X 2 )
. .
. = .
. .
MXn (t) = E (X n e tX ) ⇒ MXn (0) = E (X n )
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
A random variable Y has a moment generating function,
MY (t) = (0.6e t + 0.4)12 Find the E (Y ) and Var (Y )
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
MY (t) = (0.6e t + 0.4)12
MY0 (t) = 12(0.6e t + 0.4)11 (0.6e t )
E (Y ) = MY0 (0) = 12(0.6e 0 + 0.4)11 (0.60 )
= 7.2
MY00 (t) = 12 × 11(0.6e t + 0.4)10 (0.6e t )2
+12(0.6e t + 0.4)11 (0.6e t )
E (Y 2 ) = MY00 (0) = 12 × 11 × 0.62 + 12 × 0.6
= 54.72
Var (Y ) = E (Y 2 ) − (E (Y ))2 = 54.72 − 7.22
= 2.88
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
A continuous random variable Y has moment generating function
given by
MY (t) = 3(3 − t)−1
Find the E (Y ) and Var (Y ).
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
MY (t) = 3(3 − t)−1
MY0 (t) = 3(3 − t)−2
MY00 (t) = 6(3 − t)−3
1
E (Y ) = MY0 (0) = 3(3)−2 =
3
2 00 −3 2
E (Y ) = MY (0) = 6(3) =
9
Var (Y ) = E (Y 2 ) − (E (Y ))2
2
2 1
= −
9 3
1
=
9
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Theorem
A moment generating function always exists at t = 0 and equals to
1.
Proof
MX (0) = E (e 0X ) = E (1) = 1
Theorem
Let X be a random variable with moment generating function
MX (t). If Y = aX , where a is a constant, then MY (t) = MX (at)
Proof
MY (t) = E (e tY ) = E (E taX ) = E (e (at)X ) = MX (at)
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
4
Given that X has the moment generating function, MX (t) = 4−t
for t < 4, find the moment generating function of Y = 2X
Solution
4
MX (t) =
4−t
MY (t) = M2X (t) = MX (2t)
4
= ,t < 2
4 − (2t)
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Theorem
If Y = a + bX , and MX (t) exists, then MY (t) = e at MX (bt)
Proof
MY (t) = E (e t(a+bX ) ) = E (e at e btX ) = e at E (e btX ) = e at MX (bt)
Example
4
Given that X has the moment generating function, MX (t) = 4−t
for t < 4, find the moment generating function of Y = 4 + 3X
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
4
MX (t) =
4−t
MY (t) = M4+3X (t) = e 4t MX (3t)
4t 4 4
= e ,t <
4 − (3t) 3
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
The Uniqueness theorem
Corresponding to each moment generating function M(t), there is
a unique distribution function having that M(t) as moment
generating function.
Independence
Let X1 , X2 , ...Xn be independent random variables with moment
generating functions MXi (t)(i = 1, 2, ..., n). If
Y = X1 + X2 + ... + Xn , then MY (t) = MX1 (t)MX2 (t)...MXn (t).
Example
The random variables X and Y are independent with respective
moment generating functions MX (t) = (pe t + q)n and
MY (t) = (pe t + q)m , where p + q = 1. Find the moment
generating function of X + Y
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
Since X and Y are independent,
MX +Y (t) = MX (t)MY (t)
= (pe t + q)n (pe t + q)m
= (pe t + q)n+m
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Outline of Presentation
1 Moment Generating Functions
2 Characteristic Function
3 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
4 Special Probability Distributions
5 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
6 Useful Continuous Distributions
7 Distributions of Functions of Random Variables
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Characteristic function
Unfortunately, not all distributions have moment generating
functions and so we consider another function which exists for all
distributions and has more applications than moment generating
functions. This function is called the characteristic function.
Mathematical Definition
The characteristic function of a random variable X, denoted by
φX (t), is defined for all real t by φX (t) = E (e itX ), where i 2 = −1
Thus,
(P
∞
e itx f (x) if X is discrete
φX (t) = R ∞x=0 itx
−∞ e f (x)dx if X is continuous
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
Given
(
3e −3x , x ≥0
f (x) =
0 otherwise
Find the characteristic function of X .
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
Z ∞
φX (t) = E (e itX ) = e itx f (x)dx
−∞
Z ∞ Z ∞
−3x
= itx
e (3e )dx = 3e −(3−it)x dx
0 0
" #∞
e −(3−it)x 3
= 3 = , t < 3.
−(3 − it) 3 − it
0
Remark
It can be seen that, if MX (t) exists, then φY (t) = MY (it)
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Example
Given the discrete random variable Y as below, find the
characteristic function of Y
12
P(Y = y ) = (0.6)y (0.4)12−y , y = 0, 1, ..., 12.
y
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Solution
12
itY
X
ity 12
φY (t) = E (e )= e (0.6)y (0.4)12−y
y
y =0
12
X 12
= (0.6e it )y (0.4)12−y = (0.6e it + 0.4)12
y
y =0
Notice again that φY (t) = MY (it), if MY (t) exists.
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Theorem
φX (0) = E (e 0 ) = 1.
Theorem
If Y = a + bX , where a and b are constants, then
φY (t) = e ita φbX (t) = e ita φX (bt)
Proof
φY (t) = E (e it(a+bX ) ) = E (e ita e itbX )
= e ita E (e itbX ) = e ita φX (tb)
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables
Theorem
(n)
If the nth moment of X exists, then the nth derivation φX of
φX (t) exists, and
(k)
φX (0) = i k E (X k ), k = 1, 2, ..., n
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
