CONTINUOUS RANDOM VARIABLES

DEFINITION : A continuous random variable is a function X(s)

from an uncountably infinite sample space S to the real numbers R ,

X(·) : S → R.
EXAMPLE :
Rotate a pointer about a pivot in a plane (like a hand of a clock).
The outcome is the angle where it stops : 2πθ , where θ ∈ (0, 1] .
A good sample space is all values of θ , i.e. S = (0, 1] .
A very simple example of a continuous random variable is X(θ) = θ .

Suppose any outcome, i.e., any value of θ is ”equally likely”.

What are the values of
1
P (0 < θ ≤ 2
) , P ( 13 < θ ≤ 1
2
) , P (θ = √1 )
2
?

142
The (cumulative) probability distribution function is defined as

FX (x) ≡ P (X ≤ x) .
Thus
FX (b) − FX (a) ≡ P (a < X ≤ b) .

We must have

FX (−∞) = 0 and FX (∞) = 1 ,

i.e.,
lim FX (x) = 0 ,
x→−∞
and
lim FX (x) = 1 .
x→∞

Also, FX (x) is a non-decreasing function of x . ( Why ? )

NOTE : All the above is the same as for discrete random variables !

143
EXAMPLE : In the ”pointer example ”, where X(θ) = θ , we have
the probability distribution function
F(theta)

1/2

1/3

theta
0 1/3 1/2 1
Note that
F ( 31 ) ≡ P (X ≤ 13 ) = 1
3
, F ( 21 ) ≡ P (X ≤ 12 ) = 1
2
,

P ( 31 < X ≤ 12 ) = F ( 12 ) − F ( 13 ) = 1
2
− 1
3
= 1
6
.

QUESTION : What is P ( 31 ≤ X ≤ 21 ) ?

144
The probability density function is the derivative of the probability
distribution function :
′ d
fX (x) ≡ FX (x) ≡ FX (x) .
dx

EXAMPLE : In the ”pointer example ”


 0, x≤0
FX (x) = x, 0<x≤1
1, 1<x


Thus 
 0, x≤0
fX (x) = FX′ (x) = 1, 0<x≤1
0, 1<x


NOTATION : When it is clear what X is then we also write

f (x) for fX (x) , and F (x) for FX (x) .

145
EXAMPLE : ( continued · · · )
 
 0, x≤0  0, x≤0
F (x) = x, 0<x≤1 , f (x) = 1, 0<x≤1
1, 1<x 0, 1<x
 
F(theta) f(theta)

1 1

1/2

1/3

theta theta
0 1/3 1/2 1 0 1/3 1/2 1

Distribution function Density function

NOTE :
1
1 1 1
Z
2
P( < X ≤ ) = f (x) dx = = the shaded area .
3 2 1
3
6

146
In general, from
f (x) ≡ F ′ (x) ,
with
F (−∞) = 0 and F (∞) = 1 ,

we have from Calculus the following basic identities :

Z ∞ Z ∞
f (x) dx = F ′ (x) dx = F (∞) − F (−∞) = 1 ,
−∞ −∞

Z x
f (x) dx = F (x) − F (−∞) = F (x) = P (X ≤ x) ,
−∞

Z b
f (x) dx = F (b) − F (a) = P (a < X ≤ b) ,
a

Z a
f (x) dx = F (a) − F (a) = 0 = P (X = a) .
a

147
EXERCISE : Draw graphs of the distribution and density functions
 
0, x≤0 0, x≤0
F (x) = , f (x) = ,
1 − e−x , x > 0 e−x , x > 0

and verify that

• F (−∞) = 0 , F (∞) = 1 ,
• f (x) = F ′ (x) ,
Rx
• F (x) = 0
f (x) dx , ( Why is zero as lower limit OK ? )
R∞
• 0
f (x) dx = 1 ,

• P (0 < X ≤ 1) = F (1) − F (0) = F (1) = 1 − e−1 ∼

= 0.63 ,

• P (X > 1) = 1 − F (1) = e−1 ∼

= 0.37 ,

• P (1 < X ≤ 2) = F (2) − F (1) = e−1 − e−2 ∼

= 0.23 .

148
EXERCISE : For positive integer n, consider the density functions
 n
 cx (1 − xn ) , 0 ≤ x ≤ 1
fn (x) =
0, otherwise


• Determine the value of c in terms of n .

• Draw the graph of fn (x) for n = 1, 2, 4, 8, 16 .

• Determine the distribution function Fn (x) .

• Draw the graph of Fn (x) for n = 1, 2, 3, 4, 8, 16 .

• Determine P (0 ≤ X ≤ 21 ) in terms of n .

• What happens to P (0 ≤ X ≤ 21 ) when n becomes large?

9
• Determine P ( 10 ≤ X ≤ 1) in terms of n .
9
• What happens to P ( 10 ≤ X ≤ 1) when n becomes large?

149
Joint distributions

A joint probability density function fX,Y (x, y) must satisfy

Z ∞Z ∞
fX,Y (x, y) dx dy = 1 ( “Volume” = 1 ).
−∞ −∞

The corresponding joint probability distribution function is

Z y Z x
FX,Y (x, y) = P (X ≤ x , Y ≤ y) = fX,Y (x, y) dx dy .
−∞ −∞

∂ 2 FX,Y (x, y)
By Calculus we have = fX,Y (x, y) .
∂x∂y

Also, Z d Z b
P (a < X ≤ b , c < Y ≤ d) = fX,Y (x, y) dx dy .
c a

150
EXAMPLE :

If 
 1 for x ∈ (0, 1] and y ∈ (0, 1] ,
fX,Y (x, y) =
0 otherwise ,


then, for x ∈ (0, 1] and y ∈ (0, 1] ,

Z y Z x
FX,Y (x, y) = P (X ≤ x , Y ≤ y) = 1 dx dy = xy .
0 0

Thus
FX,Y (x, y) = xy , for x ∈ (0, 1] and y ∈ (0, 1] .

For example
1 1 1 1 1
P( X ≤ , Y ≤ ) = FX,Y ( , ) = .
3 2 3 2 6

151
 0.9
1.5 0.8
0.7
1.0 0.6
f 0.5
0.5
F 0.4
0.3
0.2

0.1 0.1
0.2 0.1
0.2
0.3 0.3
0.4
y 0.4 0.5 0.3
0.2
0.1
0.1 0.2
0.3 0.4 0.7
0.6
0.5 x
0.5
0.6
0.7 0.5
0.4
x y 0.6 0.7 0.8 0.9
0.9
0.8
0.6
0.8 0.7
0.9 0.8
0.9

Also,
3 1
1 1 1 3 1
Z Z
4 2
P( ≤ X ≤ , ≤ Y ≤ ) = f (x, y) dx dy = .
3 2 4 4 1
4
1
3
12

EXERCISE : Show that we can also compute this as follows :

F ( 12 , 43 ) − F ( 13 , 34 ) − F ( 12 , 41 ) + F ( 13 , 41 ) = 1
12
.
and explain why !

152
Marginal density functions

The marginal density functions are

Z ∞ Z ∞
fX (x) = fX,Y (x, y) dy , fY (y) = fX,Y (x, y) dx .
−∞ −∞

with corresponding marginal distribution functions

Z x Z x Z ∞
FX (x) ≡ P (X ≤ x) = fX (x) dx = fX,Y (x, y) dy dx ,
−∞ −∞ −∞

Z y Z y Z ∞
FY (y) ≡ P (Y ≤ y) = fY (y) dy = fX,Y (x, y) dx dy .
−∞ −∞ −∞

By Calculus we have
dFX (x) dFY (y)
= fX (x) , = fY (y) .
dx dy

153
EXAMPLE : If

 1 for x ∈ (0, 1] and y ∈ (0, 1] ,
fX,Y (x, y) =
0 otherwise ,


then, for x ∈ (0, 1] and y ∈ (0, 1] ,

Z 1 Z 1
fX (x) = fX,Y (x, y) dy = 1 dy = 1 ,
0 0
Z 1 Z 1
fY (y) = fX,Y (x, y) dx = 1 dx = 1 ,
0 0
Z x
FX (x) = P (X ≤ x) = fX (x) dx = x ,
0
Z y
FY (y) = P (Y ≤ y) = fY (y) dy = y .
0
For example
1
P( X ≤ 3
) = FX ( 13 ) = 1
3
, P( Y ≤ 1
2
) = FY ( 21 ) = 1
2
.

154
 EXERCISE :
(1 − e−x )(1 − e−y )

for x ≥ 0 and y ≥ 0 ,
Let FX,Y (x, y) =
0 otherwise .
• Verify that
2
e−x−y

∂ F for x ≥ 0 and y ≥ 0 ,
fX,Y (x, y) = =
∂x∂y 0 otherwise .

0.9
0.8
0.7
0.9
0.6 0.8
0.5 0.7
f 0.4 0.6
0.5
0.3
F 0.4
0.2 0.3
0.1 0.2
0.1

1 1
2 1
1 2
3 2 2
y 4
3 3
4 x y
3 x
4 4

Density function fX,Y (x, y) Distribution function FX,Y (x, y)

155
EXERCISE : ( continued · · · )
FX,Y (x, y) = (1−e−x )(1−e−y ) , fX,Y (x, y) = e−x−y , for x, y ≥ 0 .

Also verify the following :

• F (0, 0) = 0 , F (∞, ∞) = 1 ,

R∞R∞
• 0 0
fX,Y (x, y) dx dy = 1 , ( Why zero lower limits ? )

R∞
• fX (x) = 0
e−x−y dy = e−x ,

R∞
• fY (y) = 0
e−x−y dx = e−y .

• fX,Y (x, y) = fX (x) · fY (y) . ( So ? )

156
EXERCISE : ( continued · · · )

FX,Y (x, y) = (1−e−x )(1−e−y ) , fX,Y (x, y) = e−x−y , for x, y ≥ 0 .

Also verify the following :

Rx R x −x
• FX (x) = 0 fX (x) dx = 0 e dx = 1 − e−x ,
Ry Ry
• FY (y) = 0
fY (y) dy = 0
e−y dy = 1 − e−y ,

• FX,Y (x, y) = FX (x) · FY (y) . ( So ? )

• P (1 < x < ∞) = FX (∞)−FX (1) = 1−(1−e−1 ) = e−1 ∼

= 0.37 ,
R1R2
• P( 1 < x ≤ 2 , 0 < y ≤ 1 ) = 0 1
e−x−y dx dy

= (e−1 − e−2 )(1 − e−1 ) ∼

= 0.15 ,

157
Independent continuous random variables

Recall that two events E and F are independent if

P (EF ) = P (E) P (F ) .

Continuous random variables X(s) and Y (s) are independent if

P (X ∈ IX , Y ∈ IY ) = P (X ∈ IX ) · P (Y ∈ IY ) ,
for all allowable sets IX and IY (typically intervals) of real numbers.

Equivalently, X(s) and Y (s) are independent if for all such sets
IX and IY the events
X −1 (IX ) and Y −1 (IY ) ,
are independent in the sample space S.

NOTE : X −1 (IX ) ≡ {s ∈ S : X(s) ∈ IX } ,

Y −1 (IY ) ≡ {s ∈ S : Y (s) ∈ IY } .

158
FACT : X(s) and Y (s) are independent if for all x and y
fX,Y (x, y) = fX (x) · fY (y) .

EXAMPLE : The random variables with density function

 −x−y
e for x ≥ 0 and y ≥ 0 ,
fX,Y (x, y) =
0 otherwise ,
are independent because (by the preceding exercise)
fX,Y (x, y) = e−x−y = e−x · e−y = fX (x) · fY (y) .

NOTE :
(1 − e−x )(1 − e−y ) for x ≥ 0 and y ≥ 0 ,

FX,Y (x, y) =
0 otherwise ,
also satisfies (by the preceding exercise)
FX,Y (x, y) = FX (x) · FY (y) .

159
PROPERTY :
For independent continuous random variables X and Y we have
FX,Y (x, y) = FX (x) · FY (y) , for all x, y .
PROOF :
FX,Y (x, y) = P (X ≤ x , Y ≤ y)
Rx Ry
= −∞ −∞
fX,Y (x, y) dy dx
Rx Ry
= −∞ −∞
fX (x) · fY (y) dy dx (by independence)
Rx Ry
= −∞
[ fX (x) · −∞
fY (y) dy ] dx
Rx Ry
= [ −∞
fX (x) dx ] · [ −∞
fY (y) dy ]

= FX (x) · FY (y) .

REMARK : Note how the proof parallels that for the discrete case !

160
Conditional distributions
Let X and Y be continuous random variables.
For given allowable sets IX and IY (typically intervals), let
Ex = X −1 (IX ) and Ey = Y −1 (IY ) ,
be their corresponding events in the sample space S .
P (Ex Ey )
We have P (Ex |Ey ) ≡ .
P (Ey )

The conditional probability density function is defined as

fX,Y (x, y)
fX|Y (x|y) ≡ .
fY (y)

When X and Y are independent then

fX,Y (x, y) fX (x) fY (y)
fX|Y (x|y) ≡ = = fX (x) ,
fY (y) fY (y)
(assuming fY (y) 6= 0 ).

161
EXAMPLE : The random variables with density function
 −x−y
 e for x ≥ 0 and y ≥ 0 ,
fX,Y (x, y) =
0 otherwise ,


have (by previous exercise) the marginal density functions

fX (x) = e−x , fY (y) = e−y ,
for x ≥ 0 and y ≥ 0 , and zero otherwise.

Thus for such x, y we have

fX,Y (x, y) e−x−y
fX|Y (x|y) = = = e−x = fX (x) ,
fY (y) e−y
i.e., information about Y does not alter the density function of X .

Indeed, we have already seen that X and Y are independent .

162
Expectation

The expected value of a continuous random variable X is

Z ∞
E[X] = x fX (x) dx ,
−∞

which represents the average value of X over many trials.

The expected value of a function of a random variable is

Z ∞
E[g(X)] = g(x) fX (x) dx .
−∞

The expected value of a function of two random variables is

Z ∞Z ∞
E[g(X, Y )] = g(x, y) fX,Y (x, y) dy dx .
−∞ −∞

163
EXAMPLE :

For the pointer experiment



 0, x≤0



fX (x) = 1, 0<x≤1




0, 1<x


we have
∞ 1
x2 1
Z Z 1
E[X] = x fX (x) dx = x dx = = ,
−∞ 0 2 0 2

and
∞ 1
x3 1
Z Z 1
2 2 2
E[X ] = x fX (x) dx = x dx = = .
−∞ 0 3 0 3

164
EXAMPLE : For the joint density function
 −x−y
 e for x > 0 and y > 0 ,
fX,Y (x, y) =
0 otherwise .


we have (by previous exercise) the marginal density functions

 −x  −y
 e for x > 0 ,  e for y > 0 ,
fX (x) = and fY (y) =
0 otherwise , 0 otherwise .
 

Z ∞ ∞
−x −x
Thus E[X] = dx = −[(x+1)e ]
xe = 1 . ( Check ! )
0 0
Z ∞
Similarly E[Y ] = y e−y dy = 1 ,
0
and Z ∞ Z ∞
E[XY ] = xy e−x−y dy dx = 1 . ( Check ! )
0 0

165
EXERCISE :
Prove the following for continuous random variables :

• E[aX] = a E[X] ,

• E[aX + b] = a E[X] + b ,

• E[X + Y ] = E[X] + E[Y ] ,

and compare the proofs to those for discrete random variables.

EXERCISE :
A stick of length 1 is split at a randomly selected point X.
( Thus X is uniformly distributed in the interval [0, 1]. )
Determine the expected length of the piece containing the point 1/3.

166
PROPERTY : If X and Y are independent then

E[XY ] = E[X] · E[Y ] .

PROOF :

R R
E[XY ] = R R
x y fX,Y (x, y) dy dx
R R
= R R
x y fX (x) fY (y) dy dx (by independence)
R R
= R
[ x fX (x) R
y fY (y) dy ] dx
R R
= [ R x fX (x) dx ] · [ R y fY (y) dy ]

= E[X] · E[Y ] .

REMARK : Note how the proof parallels that for the discrete case !

167
EXAMPLE : For  −x−y
 e for x > 0 and y > 0 ,
fX,Y (x, y) =
0 otherwise ,

we already found
fX (x) = e−x , fY (y) = e−y ,
so that
fX,Y (x, y) = fX (x) · fY (y) ,

i.e., X and Y are independent .

Indeed, we also already found that

E[X] = E[Y ] = E[XY ] = 1 ,
so that
E[XY ] = E[X] · E[Y ] .

168
Variance Z ∞
Let µ = E[X] = x fX (x) dx
−∞

Then the variance of the continuous random variable X is

Z ∞
V ar(X) ≡ E[ (X − µ)2 ] ≡ (x − µ)2 fX (x) dx ,
−∞
which is the average weighted square distance from the mean.

As in the discrete case, we have

V ar(X) = E[X 2 − 2µX + µ2 ]

= E[X 2 ] − 2µE[X] + µ2 = E[X 2 ] − µ2 .

The standard deviation of X is

p p
σ(X) ≡ V ar(X) = E[X 2 ] − µ2 .
which is the average weighted distance from the mean.

169
 e−x ,

x>0,
EXAMPLE : For f (x) =
0, x≤0,
we have
R∞
E[X] = µ = 0
x e−x dx = 1 ( already done ! ) ,
R∞ ∞
2 2 −x 2 −x
E[X ] = 0
x e dx = − [(x + 2x + 2)e ] = 2,
0

V ar(X) = E[X 2 ] − µ2 = 2 − 12 = 1 ,
p
σ(X) = V ar(X) = 1 .

NOTE : The two integrals can be done by “integration by parts ”.

EXERCISE :
Also use the Method of Moments to compute E[X] and E[X 2 ] .

170
EXERCISE : For the random variable X with density function

 0, x ≤ −1
f (x) = c , −1 < x ≤ 1
0, x>1


• Determine the value of c

• Draw the graph of f (x)
• Determine the distribution function F (x)
• Draw the graph of F (x)
• Determine E[X]
• Compute V ar(X) and σ(X)
• Determine P (X ≤ − 12 )
• Determine P (| X |≥ 12 )

171
EXERCISE : For the random variable X with density function

 x + 1 , −1 < x ≤ 0
f (x) = 1−x , 0<x≤1
0, otherwise


• Draw the graph of f (x)

R∞
• Verify that −∞ f (x) dx = 1
• Determine the distribution function F (x)
• Draw the graph of F (x)
• Determine E[X]
• Compute V ar(X) and σ(X)
• Determine P (X ≥ 13 )
• Determine P (| X |≤ 13 )

172
EXERCISE : For the random variable X with density function
3
(1 − x2 ) , −1 < x ≤ 1

f (x) = 4
0, otherwise
• Draw the graph of f (x)
R∞
• Verify that −∞ f (x) dx = 1
• Determine the distribution function F (x)
• Draw the graph of F (x)
• Determine E[X]
• Compute V ar(X) and σ(X)
• Determine P (X ≤ 0)
• Compute P (X ≥ 23 )
• Compute P (| X |≥ 23 )

173
EXERCISE : Recall the density function
 n
 cx (1 − xn ) , 0 ≤ x ≤ 1
fn (x) =
0, otherwise


considered earlier, where n is a positive integer, and where

(n+1)(2n+1)
c = n
.
• Determine E[X] .

• What happens to E[X] for large n ?

• Determine E[X 2 ]

• What happens to E[X 2 ] for large n ?

• What happens to V ar(X) for large n ?

174
Covariance
Let X and Y be continuous random variables with mean
E[X] = µX , E[Y ] = µY .

Then the covariance of X and Y is

Cov(X, Y ) ≡ E[ (X − µX ) (Y − µY ) ]
Z ∞ Z ∞
= (x − µX ) (y − µY ) fX,Y (x, y) dy dx .
−∞ −∞

As in the discrete case, we have

Cov(X, Y ) = E[ (X − µX ) (Y − µY ) ]

= E[XY − µX Y − µY X + µX µY ]

= E[XY ] − E[X] E[Y ] .

175
As in the discrete case, we also have

PROPERTY 1 :
• V ar(X + Y ) = V ar(X) + V ar(Y ) + 2 Cov(X, Y ) ,

and

PROPERTY 2 : If X and Y are independent then

• Cov(X, Y ) = 0 ,

• V ar(X + Y ) = V ar(X) + V ar(Y ) .

NOTE :
• The proofs are identical to those for the discrete case !

• As in the discrete case, if Cov(X, Y ) = 0 then X and Y are

not necessarily independent!

176
EXAMPLE : For
 −x−y
 e for x > 0 and y > 0 ,
fX,Y (x, y) =
0 otherwise ,


we already found
fX (x) = e−x , fY (y) = e−y ,
so that
fX,Y (x, y) = fX (x) · fY (y) ,

i.e., X and Y are independent .

Indeed, we also already found

E[X] = E[Y ] = E[XY ] = 1 ,
so that
Cov(X, Y ) = E[XY ] − E[X] E[Y ] = 0 .

177
EXERCISE :

Verify the following properties :

• V ar(cX + d) = c2 V ar(X) ,

• Cov(X, Y ) = Cov(Y, X) ,

• Cov(cX, Y ) = c Cov(X, Y ) ,

• Cov(X, cY ) = c Cov(X, Y ) ,

• Cov(X + Y, Z) = Cov(X, Z) + Cov(Y, Z) ,

• V ar(X + Y ) = V ar(X) + V ar(Y ) + 2 Cov(X, Y ) .

178
EXERCISE :

For the random variables X , Y with joint density function


 45xy 2 (1 − x)(1 − y 2 ) , 0 ≤ x ≤ 1 , 0 ≤ y ≤ 1
f (x, y) =
0, otherwise


R1R1
• Verify that 0 0
f (x, y) dy dx = 1 .

• Determine the marginal density functions fX (x) and fY (y) .

• Are X and Y independent ?

• What is the value of Cov(X, Y ) ?

179
The joint probability density function fXY (x, y) .

180
Markov’s inequality.
For a continuous nonnegative random variable X , and c > 0 ,
we have
E[X]
P (X ≥ c) ≤ .
c
PROOF :
Z ∞ Z c Z ∞
E[X] = xf (x) dx = xf (x) dx + xf (x) dx
0 0 c
Z ∞
≥ xf (x) dx
c
Z ∞
≥ c f (x) dx ( Why ? )
c

= c P (X ≥ c) .
EXERCISE :
Show Markov’s inequality also holds for discrete random variables.

181
Markov’s inequality : For continuous nonnegative X , c > 0 :
E[X]
P (X ≥ c) ≤ .
c
 −x
EXAMPLE : For  e for x > 0 ,
f (x) =
0 otherwise ,

we have
Z ∞
E[X] = x e−x dx = 1 ( already done ! )
0

Markov’s inequality gives

E[X] 1
c=1 : P (X ≥ 1) ≤ = = 1 (!)
1 1

E[X] 1
c = 10 : P (X ≥ 10) ≤ = = 0.1
10 10

QUESTION : Are these estimates ”sharp ” ?

182
QUESTION : Are these estimates ”sharp ” ?
Markov’s inequality gives
E[X] 1
c=1 : P (X ≥ 1) ≤ = = 1 (!)
1 1
E[X] 1
c = 10 : P (X ≥ 10) ≤ = = 0.1
10 10
The actual values are
Z ∞
P (X ≥ 1) = e−x dx = e−1 ∼
= 0.37
1
Z ∞
P (X ≥ 10) = e−x dx = e−10 ∼
= 0.000045
10

EXERCISE : Suppose the score of students taking an examination

is a random variable with mean 65 .
Give an upper bound on the probability that a student’s score is
greater than 75 .

183
Chebyshev’s inequality: For (practically) any random variable X:
1
P( | X − µ | ≥ k σ ) ≤ 2
,
k
p
where µ = E[X] is the mean, σ = V ar(X) the standard deviation.

PROOF : Let Y ≡ (X − µ)2 , which is nonnegative.

By Markov’s inequality
E[Y ]
P (Y ≥ c) ≤ .
c
Taking c = k2 σ 2 we have

P ( | X −µ | ≥ kσ ) = P ( (X −µ)2 ≥ k2 σ 2 ) = P ( Y ≥ k2 σ 2 )

E[ Y ] V ar(X) σ2 1
≤ = = 2 2 = . QED !
k2 σ2 k2 σ2 k σ k 2

NOTE : This inequality also holds for discrete random variables.

184
EXAMPLE : Suppose the value of the Canadian dollar in terms of
the US dollar over a certain period is a random variable X with

mean µ = 0.98 and standard deviation σ = 0.05 .

What can be said of the probability that the Canadian dollar is valued
between $0.88US and $1.08US ,
that is,
between µ − 2σ and µ + 2σ ?

SOLUTION : By Chebyshev’s inequality we have

1
P( | X − µ | ≥ 2 σ ) ≤ 2
= 0.25 .
2
Thus
P ( | X − µ | < 2 σ ) > 1 − 0.25 = 0.75 ,
that is,
P ( $0.88US < Can$ < $1.08US ) > 75 % .

185
EXERCISE :
The score of students taking an examination is a random variable
with mean µ = 65 and standard deviation σ = 5 .

• What is the probability a student scores between 55 and 75 ?

• How many students would have to take the examination so that
the probability that their average grade is between 60 and 70
is at least 80% ?
HINT : Defining
1
X̄ = (X1 + X2 + · · · + Xn ) , ( the average grade )
n
we have 1
µX̄ = E[X̄] = n µ = µ = 65 ,
n
and, assuming independence,
σ2 σ2 25 5
V ar(X̄) = n 2 = = , and σX̄ = √ .
n n n n

186
 SPECIAL CONTINUOUS RANDOM VARIABLES

The Uniform Random Variable

 1 
 b−a , a < x ≤ b  0, x≤a
x−a
f (x) = , F (x) = b−a
, a<x≤b
0 , otherwise 1, x>b
 
f(x) F(x)
1
___ 1
b−a

F(x1)

F(x2)

x x
a x1 x2 b a x1 x2 b

(Already introduced earlier for the special case a = 0, b = 1 .)

187
EXERCISE :

Show that the uniform density function

1

 b−a
, a<x≤b
f (x) =
0, otherwise


has mean
a+b
µ = ,
2

and standard deviation

b−a
σ = √ .
2 3

188
 A joint uniform random variable :
1 (x − a)(y − c)
f (x, y) = , F (x, y) = ,
(b − a)(d − c) (b − a)(d − c)

for x ∈ (a, b], y ∈ (c, d].

0.9
1.5 0.8
0.7
1.0 0.6
f 0.5
0.5
F 0.4
0.3
0.2

0.1 0.1
0.2 0.1
0.2
0.3 0.3
0.4
y 0.4 0.5 0.3
0.2
0.1
0.1 0.2
0.3 0.4 0.7
0.6
0.5 x
0.5
0.6
0.7 0.5
0.4
x y 0.6 0.7 0.8 0.9
0.9
0.8
0.6
0.8 0.7
0.9 0.8
0.9

Here x ∈ [0, 1], y ∈ [0, 1] .

189
EXERCISE :

Consider the joint uniform density function


 c for x2 + y 2 ≤ 4 ,
f (x, y) =
0 otherwise .


• What is the value of c ?

• What is P (X < 0) ?

• What is P (X < 0 , Y < 0) ?

• What is f ( x | y = 1 ) ?

HINT : No complicated calculations are needed !

190
The Exponential Random Variable
 −λx 
 λe , x>0  1 − e−λx , x > 0
f (x) = , F (x) =
0, x≤0 0, x≤0
 
with
R∞ −λx 1
E[X] = µ = 0
x λe dx = λ
( Check ! ) ,

2
R∞ 2
E[X ] = 0
x2 λe−λx dx = λ2
( Check ! ) ,

1
V ar(X) = E[X 2 ] − µ2 = λ2
,
p 1
σ(X) = V ar(X) = λ
.

NOTE : The two integrals can be done by “integration by parts ”.

EXERCISE : (Done earlier for λ = 1) :

Also use the Method of Moments to compute E[X] and E[X 2 ] .

191
 2.0 1.0

0.8
1.5

0.6

F (x)
f (x)

1.0

0.4

0.5
0.2

0.0 0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
x x

The Exponential density and distribution functions

f (x) = λe−λx , F (x) = 1 − e−λx ,

for λ = 0.25, 0.50, 0.75, 1.00 (blue), 1.25, 1.50, 1.75, 2.00 (red ).

192
PROPERTY : From

F (x) ≡ P (X ≤ x) = 1 − e−λx , ( for x > 0 ) ,

we have
P (X > x) = 1 − (1 − e−λx ) = e−λx .

Also, for ∆x > 0 ,

P (X > x + ∆x , X > x)
P (X > x + ∆x | X > x) =
P (X > x)
P (X > x + ∆x) e−λ(x+∆x)
= = = e−λ∆x .
P (X > x) e−λx

CONCLUSION : P ( X > x + ∆x | X > x )

only depends on ∆x (and λ), and not on x !

We say that the exponential random variable is ”memoryless ” .

193
EXAMPLE :

Let the density function f (t) model failure of a device,

f (t) = e−t , (taking λ = 1 ) ,

i.e., the probability of failure in the time-interval (0, t] is

Z t Z t
F (t) = f (t) dt = e−t dt = 1 − e−t ,
0 0

with

F (0) = 0 , ( the device works at time 0 ).

and
F (∞) = 1 , ( the device must fail at some time ).

194
EXAMPLE : ( continued · · · ) F (t) = 1 − e−t .

Let Et be the event that the device still works at time t :

P (Et ) = 1 − F (t) = e−t .

The probability it still works at time t + 1 is

P (Et+1 ) = 1 − F (t + 1) = e−(t+1) .

The probability it still works at time t + 1, given it works at time t is

P (Et+1 Et ) P (Et+1 ) e−(t+1) 1

P (Et+1 |Et ) = = = = ,
P (Et ) P (Et ) e−t e

which is independent of t !

QUESTION : Is such an exponential distribution realistic if

the “device” is your heart, and time t is measured in decades ?

195
The Standard Normal Random Variable

The standard normal random variable has density function

1 − 12 x2
f (x) = √ e , −∞ < x < ∞ ,
2π
with mean
Z ∞
µ = x f (x) dx = 0 , ( Check ! )
−∞

Since
Z ∞
E[X 2 ] = x2 f (x) dx = 1 , ( more difficult · · · )
−∞

we have

V ar(X) = E[X 2 ] − µ2 = 1 , and σ(X) = 1 .

196
 1 − 12 x2
f (x) = √ e
2π
0.40

0.35

0.30

0.25
f (x)

0.20

0.15

0.10

0.05

0.00
−4 −3 −2 −1 0 1 2 3 4
x
The standard normal density function f (x) .

197
 x
1
Z
− 12 x2
Φ(x) = F (x) = √ e dx
2π −∞

1.0

0.9

0.8

0.7

0.6
F (x)

0.5

0.4

0.3

0.2

0.1

0.0
−4 −3 −2 −1 0 1 2 3 4
x
The standard normal distribution function F (x)
( often denoted by Φ(x) ) .

198
 The Standard Normal Distribution Φ(z)
z Φ(z) z Φ(z)
0.0 .5000 -1.2 .1151
-0.1 .4602 -1.4 .0808
-0.2 .4207 -1.6 .0548
-0.3 .3821 -1.8 .0359
-0.4 .3446 -2.0 .0228
-0.5 .3085 -2.2 .0139
-0.6 .2743 -2.4 .0082
-0.7 .2420 -2.6 .0047
-0.8 .2119 -2.8 .0026
-0.9 .1841 -3.0 .0013
-1.0 .1587 -3.2 .0007

( For example, P (Z ≤ −2.0) = Φ(−2.0) = 2.28% )

QUESTION : How to get the values of Φ(z) for positive z ?

199
EXERCISE :

Suppose the random variable X has the standard normal distribution.

What are the values of

• P ( X ≤ −0.5 )

• P ( X ≤ 0.5 )

• P ( | X | ≥ 0.5 )

• P ( | X | ≤ 0.5 )

• P( − 1 ≤ X ≤ 1 )

• P ( − 1 ≤ X ≤ 0.5 )

200
The General Normal Random Variable

The general normal density function is

1 − 21 (x−µ)2 /σ 2
f (x) = √ e
2π σ

where, not surprisingly,

E[X] = µ ( Why ? )

One can also show that

V ar(X) ≡ E[(X − µ)2 ] = σ2 ,

so that σ is in fact the standard deviation .

201
The standard normal (black) and the normal density functions
with µ = −1, σ = 0.5 (red ) and µ = 1.5, σ = 2.5 (blue).

202
To compute the mean of the general normal density function
1 − 21 (x−µ)2 /σ 2
f (x) = √ e
2π σ
consider
Z ∞
E[X − µ] = (x − µ) f (x) dx
−∞

∞
1
Z
− 12 (x−µ)2 /σ 2
= √ (x − µ) e dx
2π σ −∞

−σ 2 − 12 (x−µ)2 /σ 2
∞
= √ e = 0.
2π σ −∞

Thus the mean is indeed

E[X] = µ .

203
NOTE : If X is general normal we have the very useful formula :
X −µ
P( ≤ c ) = Φ(c) ,
σ
i.e., we can use the Table of the standard normal distribution !

PROOF : For any constant c we have

µ+cσ
X −µ 1
Z
− 12 (x−µ)2 /σ 2
P( ≤ c) = P (X ≤ µ+cσ) = √ e dx .
σ 2π σ −∞

Let y ≡ (x − µ)/σ , so that x = µ + yσ .

Then the new variable y ranges from −∞ to c , and

(x − µ)2 /σ 2 = y 2 , dx = σ dy ,
so that c
X −µ 1
Z
− 12 y 2
P( ≤ c) = √ e dy = Φ(c) .
σ 2π −∞
( the standard normal distribution )

204
EXERCISE : Suppose X is normally distributed with

mean µ = 1.5 and standard deviation σ = 2.5 .

Use the standard normal Table to determine :

• P ( X ≤ −0.5 )

• P ( X ≥ 0.5 )

• P ( | X − µ | ≥ 0.5 )

• P ( | X − µ | ≤ 0.5 )

205
The Chi-Square Random Variable

Suppose X1 , X2 , · · · , Xn ,
are independent standard normal random variables.

Then χ2n ≡ X12 + X22 + · · · + Xn2 ,

is called the chi-square random variable with n degrees of freedom.

We will show that

√
E[χ2n ] = n , V ar(χ2n ) = 2n , σ(χ2n ) = 2n .

NOTE :
2
The in χ2n is part of its name , while 2
in X12 , etc. is “power 2 ” !

206
 0.5 1.0

0.4 0.8

0.3 0.6

F (x)
f (x)

0.2 0.4

0.1 0.2

0.0 0.0
0 2 4 6 8 10 0 2 4 6 8 10
x x

The Chi-Square density and distribution functions for n = 1, 2, · · · , 10.

( In the Figure for F , the value of n increases from left to right. )

207
If n = 1 then
χ21 ≡ X12 , where X ≡ X1 is standard normal .
We can compute the moment generating function of χ21 :
Z ∞
tχ21 tX 2 1 tx2 − 12 x2
E[e ] = E[e ] = √ e e dx
2π −∞
∞
1
Z
− 12 x2 (1−2t)
= √ e dx
2π −∞

Let
1 1
1 − 2t = 2 , or equivalently , σ̂ ≡ √ .
σ̂ 1 − 2t
Then
∞
1 1
Z
tχ21 − 12 x2 /σ̂ 2
E[e ] = σ̂ · √ e dx = σ̂ = √ .
2π σ̂ −∞ 1 − 2t
(integral of a normal density function)

208
Thus we have found that :

The moment generating function of χ21 is

tχ21 1
ψ(t) ≡ E[e ] = √ ,
1 − 2t

with which we can compute

E[χ21 ] = ψ ′ (0) = 1, ( Check ! )

E[(χ21 )2 ] = ψ ′′ (0) = 3, ( Check ! )

V ar(χ21 ) = E[(χ21 )2 ] − E[χ21 ]2 = 2.

209
We found that

E[χ21 ] = 1 , V ar(χ21 ) = 2 .

In the general case where

χ2n ≡ X12 + X22 + · · · + Xn2 ,
we have

E[χ2n ] = E[X12 ] + E[X22 ] + · · · + E[Xn2 ] = n ,

and since the Xi are assumed independent ,

V ar[χ2n ] = V ar[X12 ] + V ar[X22 ] + · · · + V ar[Xn2 ] = 2n ,

and √
σ(χ2n ) = 2n .

210
 0.16

0.14

0.12

0.10
f (x)

0.08

0.06

0.04

0.02

0.00
0 5 10 15 20 25
x
The Chi-Square density functions for n = 5, 6, · · · , 15 .
( For large n they look like normal density functions ! )

211
 The χ2n - Table
n α = 0.975 α = 0.95 α = 0.05 α = 0.025
5 0.83 1.15 11.07 12.83
6 1.24 1.64 12.59 14.45
7 1.69 2.17 14.07 16.01
8 2.18 2.73 15.51 17.54
9 2.70 3.33 16.92 19.02
10 3.25 3.94 18.31 20.48
11 3.82 4.58 19.68 21.92
12 4.40 5.23 21.03 23.34
13 5.01 5.89 22.36 24.74
14 5.63 6.57 23.69 26.12
15 6.26 7.26 25.00 27.49

This Table shows zα,n values such that P (χ2n ≥ zα,n ) = α .

( For example, P (χ210 ≥ 3.94) = 95% )