Continuous Random Variables
He Shuangchi
IE5004 Continuous Random Variables 1 / 36
Let’s flip a coin
Question
Flip a coin that lands heads with probability p. Flip it once every minute.
Let X be the time until the coin lands heads for the first time. Then, what
distribution does X follow?
X follows a geometric distribution with probability mass function
P[X = k] = (1 − p)k−1 p
and expected value
∞
X 1
E[X ] = k · (1 − p)k−1 p =
p
k=1
IE5004 Continuous Random Variables 2 / 36
Let’s flip the coin n times per minute
Question
Flip a coin that lands heads with probability p = λ/n. This time, we flip it
once every 1/n minute. Let X be the time until the coin lands heads for
the first time. Then, what distribution and expected value does X have?
X has probability mass function
h ki λ k−1 λ
P X = = 1−
n n n
and expected value
∞
X k 1 1
E[X ] = · (1 − p)k−1 p = =
n np λ
k=1
IE5004 Continuous Random Variables 3 / 36
Let’s speed flips up
We flip the coin more frequently in the following way
Let λ > 0 be fixed.
The number of flips per minute n increases
Set the probability that the coin lands heads to be p = λ/n.
Then, h ki λ k−1 λ
P X = = 1− →0 as n → ∞,
n n n
but
1
E[X ] =
λ
does not change with n.
IE5004 Continuous Random Variables 4 / 36
Probability that X falls in an interval when n → ∞
Put x = k/n. Then,
λ nx−1 λ
P[X = x] = 1 −
n n
Given 0 ≤ a < b,
X λ nx−1 λ
X
P[a < X ≤ b] = P [X = x] = 1−
n n
a<x≤b a<x≤b
λ nx λ −1 1
X
= λ 1− 1−
n n n
a<x≤b | {z }| {z }
→e −λx →1
Then, Z b
P[a < X ≤ b] → λe −λx dx as n → ∞
a
IE5004 Continuous Random Variables 5 / 36
From discrete to continuous
1 1
0.8 0.8
λ(1 − λ/n)nx−1
λ(1 − λ/n)nx−1
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 1 2 3 4 5 0 1 2 3 4 5
x = k/n, n = 4 x = k/n, n = 10
1 1
0.8 0.8
λ(1 − λ/n)nx−1
0.6 0.6
λe−λx
0.4 0.4
0.2 0.2
0 0
0 1 2 3 4 5 0 1 2 3 4 5
x = k/n, n = 20 x
IE5004 Continuous Random Variables 6 / 36
Continuous random variables
Definition
If a random variable X can take uncountably many values and
P[X = a] = 0 for each a ∈ R, it is a continuous random variable.
Examples of uncountable sets: R, the set of positive real numbers, any
interval of the real line.
IE5004 Continuous Random Variables 7 / 36
Probability density function
The distribution of X is described by its probability density function.
Definition
The probability density function of X is a function f (x) such that for any
two real numbers a and b with a < b,
Z b
P[a < X ≤ b] = f (x) dx
a
Because P[X = a] = P[X = b] = 0,
P[a < X ≤ b] = P[a ≤ X < b] = P[a < X < b] = P[a ≤ X ≤ b]
Z b
= f (x) dx
a
IE5004 Continuous Random Variables 8 / 36
Probability density function
f(x)
a b
x
P[a < X ≤ b] = the area under the density curve between a and b
P[X ∈ (x − δ/2, x + δ/2)] ≈ f (x) · δ
f (x) ≥ 0 for all x
R∞
−∞ f (x) dx = 1
IE5004 Continuous Random Variables 9 / 36
Cumulative distribution function
Definition
The cumulative distribution function of a random variable X is defined for
each x ∈ R by
F (x) = P[X ≤ x]
If X is a continuous random variable with probability density function
f (x), then Z x
F (x) = P[X ≤ x] = f (u) du
−∞
If X is a continuous random variable with probability density function
f (x) and cumulative distribution function F (x), then
dF (x)
f (x) =
dx
IE5004 Continuous Random Variables 10 / 36
More on cumulative distribution functions
The following properties apply to all random variables:
P[X > x] = 1 − P[X ≤ x] = 1 − F (x)
For any two numbers a < b,
P[a < X ≤ b] = P[X ≤ b] − P[X ≤ a] = F (b) − F (a)
IE5004 Continuous Random Variables 11 / 36
Distribution of a function of a random variable
Exercise
If X has cumulative distribution function FX and probability density
function fX , find the probability density function of Y = 2X .
FY (y ) = P[Y ≤ y ] = P[2X ≤ y ] = P[X ≤ y /2] = FX (y /2)
Then,
dFY (y ) dFX (y /2)
fY (y ) = =
dy dy
Write z = y /2. Then,
dFX (z) dFX (z) dz 1 1
fY (y ) = = · = fX (z) · = fX (y /2)
dy dz dy 2 2
IE5004 Continuous Random Variables 12 / 36
Distribution of a function of a random variable
Proposition
Let X be a continuous random variable with probability density function
fX . Let g (x) be a function of x that is differentiable and strictly increasing
or decreasing. Then Y = g (X ) has a probability density function
( −1
fX (g −1 (y )) dg dy(y ) if y = g (x) for some x
fY (y ) =
0 if y 6= g (x) for all x
where g −1 (y ) is defined to be the value of x such that g (x) = y .
IE5004 Continuous Random Variables 13 / 36
Proof
Consider the case g (x) is an increasing function. For y = g (x) for some x,
FY (y ) = P[g (X ) ≤ y ] = P[X ≤ g −1 (y )] = FX (g −1 (y ))
Then,
dFY (y ) dFX (g −1 (y ))
fY (y ) = =
dy dy
Write z = g −1 (y ). Then,
dFX (z) dFX (z) dz dg −1 (y )
fY (y ) = = · = fX (g −1 (y ))
dy dz dy dy
IE5004 Continuous Random Variables 14 / 36
Percentile of a continuous distribution
f(x) η(p)
Assume that the area of the blue region = p for some 0 < p < 1.
Definition
The (100p)th percentile of the distribution of a continuous random
variable X , denoted by η(p), is defined by
Z η(p)
p = P[X ≤ η(p)] = F (η(p)) = f (y ) dy
−∞
The median of X is its 50th percentile.
IE5004 Continuous Random Variables 15 / 36
Expected value
Definition
Let X be a random variable with probability density function f (x). The
expected value of X is
Z ∞
E[X ] = x · f (x) dx
−∞
Recall that if X is a discrete random variable, then
X
E[X ] = x · p(x)
x∈D
If X is a random variable with probability density function f (x) and
h(X ) is any function of X , then
Z ∞
E[h(X )] = h(x) · f (x) dx
−∞
IE5004 Continuous Random Variables 16 / 36
Example
Exercise
The time (in hours) it takes to locate and repair an electrical breakdown in
a factory is a random variable X whose probability density function is
given by (
1 if 0 < x < 1
f (x) =
0 otherwise
If the cost involved in a breakdown of duration x is x 3 , what is the
expected cost of such a breakdown?
Z 1
1
E[X 3 ] = x 3 · 1 dx =
0 4
IE5004 Continuous Random Variables 17 / 36
Variance and standard deviation
Definition
Let X be a random variable with pdf f (x) and µ = E[X ]. Then
Z ∞
2
V (X ) = E[(X − µ) ] = (x − µ)2 · f (x) dx
−∞
is called the variance of X and
p
σ= V (X )
is called the standard deviation of X .
Variance is used to measure the variability of X .
V (X ) = E[X 2 ] − µ2 .
IE5004 Continuous Random Variables 18 / 36
Continuous uniform distribution
Definition
A random variable X is a continuous uniform random variable on the
interval [a, b] with a < b if its probability density function is
(
1
if a ≤ x ≤ b
f (x) = b−a
0 otherwise
The cumulative distribution function is
0R
if x ≤ a
x 1 x−a
F (x) = a b−a dy = b−a if a < x ≤ b
1 if x > b
IE5004 Continuous Random Variables 19 / 36
Continuous uniform distribution
Exercise
If a < c < d < b, P[c < X < d] =?
d d
d −c
Z Z
1
P[c < X < d] = f (y ) dy = dy =
c c b−a b−a
Exercise
If a < c < b < d, P[c < X < d] =?
d b d
b−c
Z Z Z
1
P[c < X < d] = f (y ) dy = dy + 0 dy =
c c b−a b b−a
IE5004 Continuous Random Variables 20 / 36
Expected value of continuous uniform distribution
Z ∞ Z a Z b Z ∞
1
E[X ] = x · f (x) dx = x · 0 dx + x· dx + x · 0 dx
−∞ −∞ a b−a b
b
1 2 b
Z
1 1 1 1
= x dx = · x = · (b 2 − a2 )
b−a a b−a 2 a b−a 2
b+a
=
2
IE5004 Continuous Random Variables 21 / 36
Variance of continuous uniform distribution
Because
Z ∞ Z b
2 2 1
E[X ] = x · f (x) dx = x2 · dx
−∞ a b−a
1 3 b
1 1 1
= · x = · (b 3 − a3 )
b−a 3 a b−a 3
1 2
= (a + ab + b 2 )
3
then,
1 1
V (X ) = E[X 2 ] − (E[X ])2 = (a2 + ab + b 2 ) − (b + a)2
3 4
1 2
= (b − a)
12
IE5004 Continuous Random Variables 22 / 36
Example
Exercise
Buses arrive at a specified stop at 15-minute intervals starting at 7 A.M.
That is, they arrive at 7, 7:15, 7:30, 7:45, and so on. If a passenger arrives
at the stop at a time that is uniformly distributed between 7 and 7:30, find
the probability that he waits
(a) less than 5 minutes for a bus
(b) at least 12 minutes for a bus
Let X be the time in minutes past 7 A.M. that the passenger arrives at the
stop. For (a)
P[10 < X < 15] + P[25 < X < 30] = 5/30 + 5/30 = 1/3
For (b)
P[0 < X < 3] + P[15 < X < 18] = 3/30 + 3/30 = 1/5
IE5004 Continuous Random Variables 23 / 36
Exponential distribution
Definition
X is an exponential random variable with rate λ > 0 if its probability
density function is given by
(
λe −λx if x ≥ 0
f (x) =
0 otherwise
(
0 if x ≤ 0
F (x) =
1 − e −λx if x > 0
E[X ] = 1/λ and V (X ) = 1/λ2
IE5004 Continuous Random Variables 24 / 36
Memorylessness
The lifetime of a component is exponentially distributed with rate λ. The
probability that it works more than x hours is
P[X > x] = 1 − F (x) = e −λx
Question
Suppose it has been working for x0 hours. What is the probability that it
lasts more than additional x hours?
P[X > x0 + x, X > x0 ] P[X > x0 + x]
P[X > x0 + x|X > x0 ] = =
P[X > x0 ] P[X > x0 ]
e −λ(x0 +x)
= = e −λx
e −λx0
= P[X > x]
At any time, the distribution of remaining lifetime is exactly the same as
the distribution of original lifetime.
IE5004 Continuous Random Variables 25 / 36
Example
Exercise
Assume that the lifetime of a traffic light until a failure follows an
exponential distribution. The failure rate is specified to be one in ten
years. What is the percentage of failure by the end of the first year?
P[X ≤ 1] = F (1) = 1 − e −0.1 ≈ 0.1
IE5004 Continuous Random Variables 26 / 36
Normal distribution
Definition
X is a normal random variable with mean µ and variance σ 2 , denoted by
X ∼ N(µ, σ 2 ), if its probability density function is
1 (x−µ)2
f (x) = √ e− 2σ 2 for all x ∈ R
2πσ 2
The normal distribution is also called the Gaussian distribution.
E[X ] = µ and V (X ) = σ 2
f (x) is symmetric about µ.
f (x) has its maximum at x = µ.
(y −µ)2
F (x) = −∞ √ 1 2 e − 2σ2 dy
Rx
2πσ
IE5004 Continuous Random Variables 27 / 36
Normal probability density function
f (x)
µ−3σ µ−2σ µ−σ µ µ+σ µ+2σ µ+3σ
x
Let X ∼ N(µ, σ 2 ). Then,
P[µ − σ ≤ X ≤ µ + σ] = 68.27%
P[µ − 2σ ≤ X ≤ µ + 2σ] = 95.45%
P[µ − 3σ ≤ X ≤ µ + 3σ] = 99.73%
IE5004 Continuous Random Variables 28 / 36
Mean and variance
The location and the dispersion of a normal distribution is determined by
both mean µ and variance σ 2 .
f (x)
Question
Which one has the larger expected value, and which one has the larger
variance?
IE5004 Continuous Random Variables 29 / 36
Standard normal distribution
Definition
Z is a standard normal random variable if Z ∼ N(0, 1).
Its probability density function and cumulative distribution function
Z x
1 x2 1 y2
φ(x) = √ e − 2 and Φ(x) = √ e − 2 dy
2π −∞ 2π
One can look up Φ(x) in a standard normal table.
IE5004 Continuous Random Variables 30 / 36
Standard normal table
Tables T-3 T-2 Tables
• •
Probability
Probability
Table entry for z is the Table entry for z is
area under the the area under the
standard normal curve standard normal curve
to the left of z. z to the left of z. z
TABLE A TABLE A
Standard normal probabilities (continued) Standard normal probabilities
z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09 z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 .5000 .5040 .5080 .5120 .5160 .5199 .5239 .5279 .5319 .5359 −3.4 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0002
0.1 .5398 .5438 .5478 .5517 .5557 .5596 .5636 .5675 .5714 .5753 −3.3 .0005 .0005 .0005 .0004 .0004 .0004 .0004 .0004 .0004 .0003
0.2 .5793 .5832 .5871 .5910 .5948 .5987 .6026 .6064 .6103 .6141 −3.2 .0007 .0007 .0006 .0006 .0006 .0006 .0006 .0005 .0005 .0005
0.3 .6179 .6217 .6255 .6293 .6331 .6368 .6406 .6443 .6480 .6517 −3.1 .0010 .0009 .0009 .0009 .0008 .0008 .0008 .0008 .0007 .0007
0.4 .6554 .6591 .6628 .6664 .6700 .6736 .6772 .6808 .6844 .6879 −3.0 .0013 .0013 .0013 .0012 .0012 .0011 .0011 .0011 .0010 .0010
0.5 .6915 .6950 .6985 .7019 .7054 .7088 .7123 .7157 .7190 .7224 −2.9 .0019 .0018 .0018 .0017 .0016 .0016 .0015 .0015 .0014 .0014
0.6 .7257 .7291 .7324 .7357 .7389 .7422 .7454 .7486 .7517 .7549 −2.8 .0026 .0025 .0024 .0023 .0023 .0022 .0021 .0021 .0020 .0019
0.7 .7580 .7611 .7642 .7673 .7704 .7734 .7764 .7794 .7823 .7852 −2.7 .0035 .0034 .0033 .0032 .0031 .0030 .0029 .0028 .0027 .0026
0.8 .7881 .7910 .7939 .7967 .7995 .8023 .8051 .8078 .8106 .8133 −2.6 .0047 .0045 .0044 .0043 .0041 .0040 .0039 .0038 .0037 .0036
0.9 .8159 .8186 .8212 .8238 .8264 .8289 .8315 .8340 .8365 .8389 −2.5 .0062 .0060 .0059 .0057 .0055 .0054 .0052 .0051 .0049 .0048
1.0 .8413 .8438 .8461 .8485 .8508 .8531 .8554 .8577 .8599 .8621 −2.4 .0082 .0080 .0078 .0075 .0073 .0071 .0069 .0068 .0066 .0064
1.1 .8643 .8665 .8686 .8708 .8729 .8749 .8770 .8790 .8810 .8830 −2.3 .0107 .0104 .0102 .0099 .0096 .0094 .0091 .0089 .0087 .0084
1.2 .8849 .8869 .8888 .8907 .8925 .8944 .8962 .8980 .8997 .9015 −2.2 .0139 .0136 .0132 .0129 .0125 .0122 .0119 .0116 .0113 .0110
1.3 .9032 .9049 .9066 .9082 .9099 .9115 .9131 .9147 .9162 .9177 −2.1 .0179 .0174 .0170 .0166 .0162 .0158 .0154 .0150 .0146 .0143
1.4 .9192 .9207 .9222 .9236 .9251 .9265 .9279 .9292 .9306 .9319 −2.0 .0228 .0222 .0217 .0212 .0207 .0202 .0197 .0192 .0188 .0183
1.5 .9332 .9345 .9357 .9370 .9382 .9394 .9406 .9418 .9429 .9441 −1.9 .0287 .0281 .0274 .0268 .0262 .0256 .0250 .0244 .0239 .0233
1.6 .9452 .9463 .9474 .9484 .9495 .9505 .9515 .9525 .9535 .9545 −1.8 .0359 .0351 .0344 .0336 .0329 .0322 .0314 .0307 .0301 .0294
1.7 .9554 .9564 .9573 .9582 .9591 .9599 .9608 .9616 .9625 .9633 −1.7 .0446 .0436 .0427 .0418 .0409 .0401 .0392 .0384 .0375 .0367
1.8 .9641 .9649 .9656 .9664 .9671 .9678 .9686 .9693 .9699 .9706 −1.6 .0548 .0537 .0526 .0516 .0505 .0495 .0485 .0475 .0465 .0455
1.9 .9713 .9719 .9726 .9732 .9738 .9744 .9750 .9756 .9761 .9767 −1.5 .0668 .0655 .0643 .0630 .0618 .0606 .0594 .0582 .0571 .0559
2.0 .9772 .9778 .9783 .9788 .9793 .9798 .9803 .9808 .9812 .9817 −1.4 .0808 .0793 .0778 .0764 .0749 .0735 .0721 .0708 .0694 .0681
2.1 .9821 .9826 .9830 .9834 .9838 .9842 .9846 .9850 .9854 .9857 −1.3 .0968 .0951 .0934 .0918 .0901 .0885 .0869 .0853 .0838 .0823
2.2 .9861 .9864 .9868 .9871 .9875 .9878 .9881 .9884 .9887 .9890 −1.2 .1151 .1131 .1112 .1093 .1075 .1056 .1038 .1020 .1003 .0985
2.3 .9893 .9896 .9898 .9901 .9904 .9906 .9909 .9911 .9913 .9916 −1.1 .1357 .1335 .1314 .1292 .1271 .1251 .1230 .1210 .1190 .1170
2.4 .9918 .9920 .9922 .9925 .9927 .9929 .9931 .9932 .9934 .9936 −1.0 .1587 .1562 .1539 .1515 .1492 .1469 .1446 .1423 .1401 .1379
2.5 .9938 .9940 .9941 .9943 .9945 .9946 .9948 .9949 .9951 .9952 −0.9 .1841 .1814 .1788 .1762 .1736 .1711 .1685 .1660 .1635 .1611
2.6 .9953 .9955 .9956 .9957 .9959 .9960 .9961 .9962 .9963 .9964 −0.8 .2119 .2090 .2061 .2033 .2005 .1977 .1949 .1922 .1894 .1867
2.7 .9965 .9966 .9967 .9968 .9969 .9970 .9971 .9972 .9973 .9974 −0.7 .2420 .2389 .2358 .2327 .2296 .2266 .2236 .2206 .2177 .2148
2.8 .9974 .9975 .9976 .9977 .9977 .9978 .9979 .9979 .9980 .9981 −0.6 .2743 .2709 .2676 .2643 .2611 .2578 .2546 .2514 .2483 .2451
2.9 .9981 .9982 .9982 .9983 .9984 .9984 .9985 .9985 .9986 .9986 −0.5 .3085 .3050 .3015 .2981 .2946 .2912 .2877 .2843 .2810 .2776
3.0 .9987 .9987 .9987 .9988 .9988 .9989 .9989 .9989 .9990 .9990 −0.4 .3446 .3409 .3372 .3336 .3300 .3264 .3228 .3192 .3156 .3121
3.1 .9990 .9991 .9991 .9991 .9992 .9992 .9992 .9992 .9993 .9993 −0.3 .3821 .3783 .3745 .3707 .3669 .3632 .3594 .3557 .3520 .3483
3.2 .9993 .9993 .9994 .9994 .9994 .9994 .9994 .9995 .9995 .9995 −0.2 .4207 .4168 .4129 .4090 .4052 .4013 .3974 .3936 .3897 .3859
3.3 .9995 .9995 .9995 .9996 .9996 .9996 .9996 .9996 .9996 .9997 −0.1 .4602 .4562 .4522 .4483 .4443 .4404 .4364 .4325 .4286 .4247
3.4 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9998 0.0 .5000 .4960 .4920 .4880 .4840 .4801 .4761 .4721 .4681 .4641
Figure: Table of Φ(z)
IE5004 Continuous Random Variables 31 / 36
Standard normal distribution
Proposition
Let X ∼ N(µ, σ 2 ). Then (X − µ)/σ is a standard normal random variable.
Let Z = (X − µ)/σ. For g (x) = (x − µ)/σ,
dg −1 (z)
g −1 (z) = σz + µ and =σ
dz
Then,
dg −1 (z)
fZ (z) = fX (g −1 (z)) ·
dz
1 (g −1 (z)−µ)2 dg −1 (z)
=√ e − 2σ2 ·
2πσ 2 dz
1 − z2
=√ e 2
2π
IE5004 Continuous Random Variables 32 / 36
Example
Exercise
If X is a normal random variable with µ = 3 and σ 2 = 16, find
(a) P[X < 11]
(b) P[X > −1]
(c) P[2 < X < 7]
Let Z = (X − 3)/4. Then Z is a standard normal random variable.
P[X < 11] = P[(X − 3)/4 < (11 − 3)/4] = P[Z < 2]
= Φ(2) = 0.9772
P[X > −1] = P[(X − 3)/4 > (−1 − 3)/4] = P[Z > −1]
= 1 − Φ(−1) = 0.8413
P[2 < X < 7] = P[(2 − 3)/4 < (X − 3)/4 < (7 − 3)/4] = P[−1/4 < Z < 1]
= P[Z < 1] − P[Z < −1/4] = Φ(1) − Φ(−1/4) = 0.4400
IE5004 Continuous Random Variables 33 / 36
Central limit theorem (CLT)
Theorem
Let X1 , X2 , . . . be iid random variables with mean µ and variance σ 2 , and
X̄n = (X1 + · · · + Xn )/n. Then,
h √n(X̄ − µ) i
n
lim P ≤ x = Φ(x)
n→∞ σ
√
When n is large, n(X̄n − µ)/σ approximately follows a standard
normal distribution.
X̄n is approximately normally distributed with mean µ and variance
σ 2 /n.
IE5004 Continuous Random Variables 34 / 36
An approximate normal distribution
Let X1 , . . . , X20 be iid Bernoulli random variables with p = 0.5.
0.4
0.35
0.3
0.25
f (x)
0.2
0.15
0.1
0.05
0
−5 −4 −3 −2 −1 0 1 2 3 4 5
x
√
Figure: The probability histogram of 20(X̄20 − µ)/σ and the standard normal
probability density function.
IE5004 Continuous Random Variables 35 / 36
Homework
Reading assignment
Read Chapters 5.1–5.5 and 5.7
Study Chapter 5.5.1 on your own
Exercise problems:
5.4, 5.7, 5.15, 5.19, 5.34, 5.40
IE5004 Continuous Random Variables 36 / 36
