University of Benghazi

Faculty of Engineering
Electrical and Electronics Engineering Department

Probability and Random Process Course

EE277

Salma Elkawafi
Salma.elkawafi@uob.edu.ly
Goals
Understand the following:
Moments
Moments Generation function
Characteristic Function
 Moments
Moments are a set of statistical parameters to measure a distribution. The shape of any distribution can be described
by its various ‘moments’. Four moments are commonly used:
1. Mean: the average
2. Variance: Standard deviation is the square root of the variance: an indication of how closely the values are spread
about the mean.
3. Skewness: measure the asymmetry of a distribution about its peak.
a. It is often approximated by Skew = (Mean - Median) / (Std dev).
b. If skewness is positive, the mean is bigger than the median and the distribution has a large tail of high values.
c. If skewness is negative, the mean is smaller than the median and the distribution has a large tail of small values.
4. Kurtosis: measures the peakedness or flatness of a distribution.
a. Positive kurtosis indicates a thin pointed distribution.
b. Negative kurtosis indicates a broad flat distribution.
Moments
The 𝑛th moment of a random variable X is defined to be
𝜇𝑛 = 𝐸[𝑋 𝑛 ]
The 𝑛th central moment of 𝑋 is defined to be
𝑛
𝑚𝑛 = 𝐸[ 𝑋 − 𝐸 𝑋 ]

• First moment (mean):

𝜇 = 𝜇1 = 𝐸[𝑋]

• Second moment (variance)

The second moment of a random variable:
𝜇2 = 𝐸[𝑋 2 ]
while the second centered moment is its variance (scatter parameter),
𝑚2 = 𝐸[ 𝑋 − 𝜇 2 ] = 𝑉𝑎𝑟[𝑋] = 𝜎 2
• Third moment (skewness) symmetry
𝑚3
𝑆𝑘𝑒𝑤𝑋 = 3
𝜎
• Fourth moment (kurtosis) tails
𝑚
𝐾𝑢𝑟𝑡𝑋 = 44
𝜎
Moment Generating Function
The moment generating function of random variable 𝑋 evaluated at s ∈ 𝑅 is given by
𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋
The moment generating function completely determines the distribution of the random variable 𝑋.

If 𝑌 = 𝑎𝑋 + 𝑏, then 𝑀𝑌 𝑠 = 𝑒 𝑠𝑏 𝑀𝑋 𝑎𝑠

If 𝑋 and 𝑌 are independent 𝑀𝑋+𝑌 (𝑠) = 𝑀𝑋 𝑠 𝑀𝑌 (𝑠)

Moment generating function of some random variables

If 𝑋 ∼ 𝐵𝑖𝑛𝑜𝑚𝑖𝑎𝑙(𝑛, 𝑝) then 𝑀𝑋 (𝑠) = 1 − 𝑝 + 𝑝𝑒 𝑠 𝑛

𝑒 𝑠 −1
If 𝑋 ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛 𝜆 ,then 𝑀𝑋 𝑠 = 𝑒 𝜆

𝜆
If 𝑋 ∼ 𝐸𝑥𝑝(𝜆), then 𝑀𝑋 (𝑠) =
𝜆−𝑠

𝜎2 𝑠2
If 𝑋 ∼ 𝑁 𝜇, 𝜎 , then 𝑀𝑋 (𝑠) = 𝑒 2 + 𝜇𝑠
Moment Generating Function 𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋
Moment generating function and moments

The 𝑛-th derivative of the moment generating function evaluated at 0 equals the 𝑛-th moment of a
random variable:

𝑑𝑛 𝑀𝑋 𝑠
ቚ = 𝜇𝑛
𝑑𝑠 𝑛 𝑠=0
𝑀𝑋′ 𝑠 = 𝐸 𝑋𝑒 𝑠𝑋 → 𝑀𝑋′ 0 = 𝐸 𝑋

𝑀𝑋′′ 𝑠 = 𝐸 𝑋 2 𝑒 𝑠𝑋 → 𝑀𝑋′′ 0 = 𝐸 𝑋 2

(𝑛) (𝑛)
𝑀𝑋 𝑠 = 𝐸 𝑋 𝑛 𝑒 𝑠𝑋 → 𝑀𝑋 0 = 𝐸[𝑋 𝑛 ]
Moment Generating Function
Examples:
For each of the following random variables, find the MGF.
a) 𝑌 is a 𝑈𝑛𝑖𝑓𝑜𝑟𝑚 0,1 random variable.
b) 𝑋 is a discrete random variable with PMF
1
, 𝑘=1
𝑃𝑋 𝑘 = 3
2
, 𝑘=2
3

Solution
1 𝑒 𝑠 −1
a) 𝑀𝑌 𝑠 = 𝐸 𝑒 𝑠𝑌 = ‫׬‬0 𝑒 𝑠𝑋 𝑑𝑦 = 𝑠

1 2
b) 𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋 = 𝑒 𝑠 + 𝑒 2𝑠
3 3
Moment Generating Function
Examples: Find the moment-generating function of the binomial random variable 𝑋 and then use it to
verify that 𝜇 = 𝑛𝑝 and 𝜎 2 = 𝑛𝑝𝑞.
Solution

We can solve this question directly using the definition of MGF, an easier way to solve it is to use the
fact that a binomial random variable can be considered as the sum of 𝑛 independent and identically
distributed (𝑖. 𝑖. 𝑑. ) Bernoulli random variables. Thus, we can write
𝑋 = 𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 ,
where 𝑋𝑖 ∼ 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝑝). Thus,
𝑛
𝑀𝑋 𝑠 = 𝑀𝑋1 𝑠 𝑀𝑋2 𝑠 ⋯ 𝑀𝑋𝑛 𝑠 = 𝑀𝑋1 𝑠 (since the 𝑋𝑖′𝑠 are i.i.d.)
Also,
𝑀𝑋1 𝑠 = 𝐸 𝑒 𝑠𝑋1 = 𝑝𝑒 𝑠 + 𝑞.
Thus, we conclude
𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋 = 𝑝𝑒 𝑠 + 𝑞 𝑛 .
Characteristic Function
The characteristic function defined as
𝜙𝑋 𝜔 = 𝑀𝑋 (𝑗𝜔) = 𝐸[𝑒 𝑗𝜔𝑋 ],

Where 𝜔 is a real number. It is worth noting that 𝑒 𝑗𝜔𝑋 is a complex-valued random

variable.

The characteristic function has similar properties to the MGF.

From the theory of Fourier transforms, we can easily determine the density function
from the characteristic function.
1 ∞
𝑓𝑋 𝑥 = න 𝜙𝑋 (𝜔) 𝑒 −𝑗𝜔𝑥 𝑑𝜔
2𝜋 −∞
 Characteristic Function
Determine the moments from the characteristic function.

𝑛
1 𝑑𝑛
𝐸𝑋 = 𝑛 𝜙 𝜔 ቚ
𝑗 𝑑𝜔 𝑛 𝑋 𝜔=0

Determine the density function of 𝒁 from the characteristic function.

If 𝑍 = 𝑋 + 𝑌 where 𝑋, 𝑌 are independent random variables

𝜙𝑍 𝜔 = 𝜙𝑋+𝑌 (𝜔) = 𝐸 𝑒 𝑗𝜔 𝑋+𝑌

= 𝐸[𝑒 𝑗𝜔𝑋 𝑒 𝑗𝜔𝑌 ] = 𝐸[𝑒 𝑗𝜔𝑋 ]𝐸[𝑒 𝑗𝜔𝑌 ](since X and Y are independent)

𝑓𝑍 𝑧 = 𝑓𝑋 𝑥 ∗ 𝑓𝑌 𝑦 → 𝜙𝑍 𝜔 = 𝜙𝑋 𝜔 𝜙𝑌 𝜔
1 ∞
𝑓𝑍 𝑧 = න 𝜙𝑍 (𝜔) 𝑒 −𝑗𝜔𝑍 𝑑𝜔
2𝜋 −∞
 Characteristic Function
Example
If 𝑋 ∼ 𝐸𝑥𝑝(𝜆) find 𝜙𝑋 𝜔 , 𝑓𝑋 𝑥 = 𝜆𝑒 −𝜆𝑥 𝑢 𝑥 , where 𝑢 𝑥 is a step function. Also find the variance using the
characteristic function.
Solution
∞ ∞
𝑗𝜔𝑋 −𝜆𝑥 𝑗𝜔𝑥
𝜆 ∞ 𝜆
𝜙𝑋 𝜔 = 𝐸 𝑒 = න 𝜆𝑒 𝑒 𝑑𝑥 = න 𝜆 𝑒 𝑗𝜔−𝜆 𝑥 𝑑𝑥 = 𝑒 𝑗𝜔−𝜆 𝑥 ቚ =
0 0 −(𝜆 − 𝑗𝜔) 0 𝜆 − 𝑗𝜔
𝑛
1 𝑑
𝐸 𝑋𝑛 = 𝑛 𝜙 𝜔 ቚ
𝑗 𝑑𝜔 𝑛, 𝑋 𝜔=0
𝑑 𝑗𝜆 𝑑2 −(𝑗𝜆)(2 𝜆 − 𝑗𝜔 ) (−𝑗)
𝜙𝑋 𝜔 = , 𝜙𝑋 𝜔 =
𝑑𝜔 𝜆 − 𝑗𝜔 2 𝑑𝜔 2 𝜆 − 𝑗𝜔 4
1 𝑑 1 𝑗𝜆 1
𝐸𝑋 = 𝜙𝑋 𝜔 ቚ = =
𝑗 𝑑𝜔 𝜔=0 𝑗 𝜆 − 𝑗0 2 𝜆
2
1 𝑑2 1 −(𝑗𝜆)(2 𝜆 − 𝑗𝜔 ) (−𝑗) 2𝜆2 2
𝐸𝑋 = 2 𝜙 𝑋 𝜔 ቚ = ቚ = =
𝑗 𝑑𝜔 2 𝜔=0 𝑗2 𝜆 − 𝑗𝜔 4 𝜔=0 𝜆4 𝜆2

2
2 2 2 1 1
𝑉𝑎𝑟 𝑋 = 𝐸 𝑋 − 𝐸 𝑋 = 2 − =
𝜆 𝜆 𝜆2