Lecture 23: Random Variables, Expectation, and
Variance
Introduction to Mathematical Modeling, Spring 2025
Lecturer: Yijun Dong
Helpful reference: All of Statistics by Larry Wasserman §2
Random variables
Definition (Random variable)
Given a probability space (!, A, P), a random variable is a mapping
X :!→R (1)
that assigns a real number X (ω) to each outcome ω ↑ !.
• Example: Flip a fair coin twice. Each ω ↑ ! = {HH, HT , TH, TT } can
be represented as a two-bit binary number X = X1 + 2X2 where Xi = 0
if tails and Xi = 1 if heads:
X (HH) = 3, X (HT ) = 2, X (TH) = 1, X (TT ) = 0.
• Cumulative distribution function (CDF) is a function FX : R → [0, 1]:
FX (x) = Pr[X ↭ x],
1
Random variables
Definition (Random variable)
Given a probability space (!, A, P), a random variable is a mapping
X :!→R (1)
that assigns a real number X (ω) to each outcome ω ↑ !.
• Cumulative distribution function (CDF) is a function FX : R → [0, 1]:
FX (x) = Pr[X ↭ x],
• Probability density function (PDF) is a function fX : R → [0, 1] s.t.
! b ! x
Pr(a < X < b) = fX (x)dx ↓ a, b ↑ R, and FX (x) = fX (t)dt.
a →↑
d
In particular, fX (x) = dx FX (x) for all x where FX is di”erentiable .
1
Examples: discrete random variables
Discrete random variable: X : ! → Z with PDF fX : Z → [0, 1]
• Bernoulli random variable with parameter p ↑ [0, 1]:
p
if x = 1,
fX (x) = 1↔p if x = 0,
0 otherwise.
• Binomial random variable with parameters n ↑ N and p ↑ [0, 1]:
& '
n x
fX (x) = p (1 ↔ p)n→x .
x
• Poisson random variable with parameter ε > 0:
εx e →ω
fX (x) = .
x!
2
Examples: continuous random variables
Continuous random variable: X : ! → R with PDF fX : R → [0, 1]
• Uniform random variable supported over a closed interval [a, b]:
(
1
b→a if x ↑ [a, b],
fX (x) =
0 otherwise.
• Gaussian random variable with mean µ ↑ R and variance ϑ 2 (ϑ > 0):
1 →
(x→µ)2
fX (x) = ↗ e 2ω 2 .
2ϖϑ 2
• Exponential random variable with parameter ε:
(
εe →ωx if x ↘ 0,
fX (x) =
0 otherwise.
3
Independent random variables
Definition (Independent random variables)
Two random variables X and Y are independent if for all x, y ↑ R,
Pr(X = x, Y = y ) = Pr(X = x) Pr(Y = y ). (2)
Example: Let X1 , · · · , Xn be independent Bernoulli random variables with
)n
parameter p ↑ [0, 1]. Then, X = i=1 Xi is a binomial random variable with
parameters n and p.
Proof: For any x ↑ N, let C(n, x) = {S ≃ [n] : |S| = x} be the set of all
subsets of [n] with size x. Then,
fX (x) = Pr(X1 + · · · + Xn = x)
*
= Pr(Xi = 1 ↓ i ↑ S) Pr(Xi = 0 ↓ i ↑
/ S)
S↓C(n,x)
* & '
n x
= p x (1 ↔ p)n→x = p (1 ↔ p)n→x .
x
S↓C(n,x)
4
Multivariate distributions
• Random vector: X = (X1 , ..., Xn ) where X1 , ..., Xn are random variables.
• The joint distribution of (X1 , ..., Xn ) gives the PDF of X,
fX : Rn → [0, 1]:
fX (x) = f(X1 ,...,Xn ) (x1 , ..., xn ) = Pr(X1 = x1 , ..., Xn = xn ).
• If X1 , ..., Xn are independent, then
n
+
fX (x) = fXi (xi ).
i=1
• The marginal distribution of Xi is the PDF fXi : R → [0, 1] such that
!
fXi (x) = fX (x1 , · · · , xi→1 , x, xi+1 , . . . , xn ) dx1 · · · dxi→1 dxi+1 · · · dxn .
Rn→1
5
Multivariate distributions
• Random vector: X = (X1 , ..., Xn ) where X1 , ..., Xn are random variables.
• The joint distribution of (X1 , ..., Xn ) gives the PDF of X,
fX : Rn → [0, 1]:
fX (x) = f(X1 ,...,Xn ) (x1 , ..., xn ) = Pr(X1 = x1 , ..., Xn = xn ).
• If X1 , ..., Xn are independent, then
n
+
fX (x) = fXi (xi ).
i=1
• The marginal distribution of Xi is the PDF fXi : R → [0, 1] such that
!
fXi (x) = fX (x1 , · · · , xi→1 , x, xi+1 , . . . , xn ) dx1 · · · dxi→1 dxi+1 · · · dxn .
Rn→1
• Example: a Gaussian random vector X ↑ Rn with mean µ ↑ Rn and
covariance ! ↑ Rn↔n with ! ⇐ 0 is given by
& '
1 1
fX (x) = , exp ↔ (x ↔ µ)T !→1 (x ↔ µ) .
(2ϖ)n det(!) 2
5
Expectation, Variance, and Moments
• For a continuous random variable X with PDF fX , the expectation of
X is
!
E[X ] = x fX (x) dx.
R
• For a discrete random variable X with PDF fX , the expectation of X
is
*
E[X ] = x fX (x).
x↓Z
• For a continuous random variable X with PDF fX and µ = E[X ]:
• The variance of X is
!
V[X ] = E[(X → µ)2 ] = (x → µ)2 fX (x) dx.
R
• For k = 1, 2, · · · , the k-th moment of X is
!
E[X k ] = x k fX (x) dx.
R
6
Properties of expectation
• (Linearity) For any random variables X1 , ..., Xn and constants
a1 , ..., an ↑ R, - n .
* * n
E ai Xi = ai E [Xi ] (3)
i=1 i=1
• If X1 , ..., Xn are independent random variables, then
- n . n
+ +
E Xi = E [Xi ] (4)
i=1 i=1
7
Properties of variance
• V(X ) = E[X 2 ] ↔ E[X ]2
• V(aX + b) = a2 V(X )
• if X1 , ..., Xn are independent random variables
- n . n
* *
V ai Xi = ai2 V [Xi ] (5)
i=1 i=1
8
Supplementary material: moment generating function
Definition (Moment generating function (MGF))
The moment generating function of a random variable X is defined as
!
MX (t) = E[e tX ] = e tx fX (x) dx, t ↑ R.
R
Properties:
• By Taylor expansion,
* tk↑
t2
MX (t) = 1 + tE[X ] + E[X 2 ] + · · · = E[X k ].
2 k!
k=0
Assuming all moments are well-defined, for any k ↑ N,
dk
dt k M X (t) = E[X k
].
• If X1 , ..., Xn are independent random variables with MGFs MX1 , ..., MXn ,
)n
then the MGF of Y = i=1 Xi is
MY (t) = MX1 (t)MX2 (t) · · · MXn (t).
9
