Addis Ababa Science & Technology University

Department of Electrical & Computer Engineering

Probability and Random Process (EEEg-2114)

Chapter 2: Random Variables

 Random Variables
Outline
 Introduction
 The Cumulative Distribution Function (CDF)
 Probability Density and Mass Functions
 Expected Value, Variance and Moments
 Some Special Distributions
 Functions of One Random Variable

2
Introduction
 A random variable X is a function that assigns a real number
X(ω) to each outcome ω in the sample space Ω of a random
experiment.
 The sample space Ω is the domain of the random variable and
the set RX of all values taken on by X is the range of the random
variable.
 Thus, RX is the subset of all real numbers.


A
X ( )  x
Re al Line
x B

3
Random Variable (RV):
 Function of elements of the sample space Ω.
 Function that maps all elements of the sample space into
points on the real line or some parts there of.
 Represented by capital letters (like X,Y or W) & any its
particular value by lowercase letter such as x, y or w.
Conditions for a function to be a RV.
A function X to be a RV:
1. It should not be multi-valued. i.e every point in the Ω must
correspond to only one value of the RV.(one-to-one OR many-to-one)
2. The set {X≤x} shall be an event for any real number x. This set
corresponds to those points ω in Ω for which the RV X(ω) doesn’t
exceed the no x. The probability of this event, P{X≤x}, is equal to
sum of probability of all the elementary events corresponding to
{X≤x}.
4
3. P{X=∞}=0 and P{X=-∞}=0 .
Introduction Cont’d……
 If X is a random variable, then {ω: X(ω)≤ x}={X≤ x} is an event
for every X in RX.
Example: Consider a random experiment of tossing a fair coin three
times. The sequence of heads and tails is noted and the sample
space Ω is given by:
  {HHH , HHT , HTH , THH , THT , HTT , TTH , TTT}

Let X be the number of heads in three coin tosses.

X assigns each possible outcome ω in the sample space Ω a
number from the set RX={0, 1, 2, 3}.
 : HHH HHT HTH THH THT HTT TTH TTT
X ( ) : 3 2 2 2 1 1 1 0

5
The Cumulative Distribution Function

 The cumulative distribution function (cdf) of a random variable

X is defined as the probability of the event {X≤ x}.
FX ( x)  P( X  x)
Properties of the cdf, FX(x):
 The cdf has the following properties.

i. FX ( x) is a non - negative function, i.e.,

0  FX ( x)  1
ii. lim FX ( x)  1
x 

iii. lim FX ( x)  0
x  

6
The Cumulative Distribution Function Cont’d…..

iv. FX ( x) is a non - decreasing function of X , i.e.,

If x1  x2 , then FX ( x1 )  FX ( x2 )

v. P ( x1  X  x2 )  FX ( x2 )  FX ( x1 )
vi. P ( X  x)  1  FX ( x )

Example:
Find the cdf of the random variable X which is defined as the
number of heads in three tosses of a fair coin.

7
The Cumulative Distribution Function
Solution:
 We know that X takes on only the values 0, 1, 2 and 3 with
probabilities 1/8, 3/8, 3/8 and 1/8 respectively.
 Thus, FX(x) is simply the sum of the probabilities of the
outcomes from the set {0, 1, 2, 3} that are less than or equal to x.
0, x  0
1 / 8, 0  x  1

 FX ( x)  1 / 2, 1  x  2
7 / 8, 2  x  3

1, x  3

8
Types of Random Variables
 There are two basic types of random variables.
i. Continuous Random Variable
 A continuous random variable is defined as a random variable
whose cdf, FX(x), is continuous every where and can be written as
an integral of some non-negative function f(x), i.e.,

FX ( x)   f (u )du


ii. Discrete Random Variable

 A discrete random variable is defined as a random variable whose
cdf, FX(x), is a right continuous, staircase function of X with
jumps at a countable set of points x0, x1, x2,……
9
The Probability Density Function
 The probability density function (pdf) of a continuous random
variable X is defined as the derivative of the cdf, FX(x), i.e.,
dFX ( x)
f X ( x) 
dx
Properties of the pdf, fX(x):
i. For all values of X , f X ( x)  0

ii. 

f X ( x)dx  1
x2
iii. P( x1  X  x2 )   f X ( x)dx
x1

10
The Probability Mass Function
 The probability mass function (pmf) of a discrete random
variable X is defined as:

PX ( X  xi )  PX ( xi )  FX ( xi )  FX ( xi 1 )

Properties of the pmf, PX (xi ):

i. 0  PX ( xi )  1, k  1, 2, .....

ii. PX ( x)  0, if x  xk , k  1, 2, .....

iii. Pk
X ( xk )  1

11
Calculating the Cumulative Distribution Function

 The cdf of a continuous random variable X can be obtained by

integrating the pdf, i.e.,
x
FX ( x)   f X (u )du


 Similarly, the cdf of a discrete random variable X can be obtained by

using the formula:

FX ( x)  P
xk  x
X ( xk )U ( x  xk )

12
 Expected Value, Variance and Moments
i. Expected Value (Mean)
 The expected value (mean) of a continuous random variable X,
denoted by μX or E(X), is defined as:


 X  E ( X )   xf X ( x )dx


 Similarly, the expected value of a discrete random variable X is

given by:

 X  E ( X )   xk PX ( xk )
k

 Mean represents the average value of the random variable in a

very large number of trials.
13
Expected Value, Variance and Moments Cont’d…..
ii. Variance
 The variance of a continuous random variable X, denoted by
σ2X or VAR(X), is defined as:
 2 X  Var ( X )  E[( X   X ) 2 ]

 2
X  Var ( X )   ( X   X ) 2 f X ( x)dx


 Expanding (X-μX )2 in the above equation and simplifying the

resulting equation, we will get:
 2
X  Var ( X )  E ( X )  [ E ( X )]
2 2

14
 Expected Value, Variance and Moments Cont’d…..
 The variance of a discrete random variable X is given by:

 2
X  Var ( X )   ( xk   X ) PX ( xk )
2

 The standard deviation of a random variable X, denoted by σX, is

simply the square root of the variance, i.e.,
 X  E ( X   X ) 2  Var ( X )

iii.Moments
 The nth moment of a continuous random variable X is defined as:
E ( X n )   x n f X ( x)dx , n 1

15
Expected Value, Variance and Moments Cont’d…..

 Similarly, the nth moment of a discrete random variable X is

given by:

E ( X )   xk PX ( xk ) ,
n
n 1
k

 Mean of X is the first moment of the random variable X.

16
 Functions That Give Moments
 Two functions can be defined that allow moments to be
calculated for a random variable X.
 They are the characteristic function and the moment
generating function.
Characteristic Function
The characteristic function of a random variable X is defined by

Where it is a function of the real number - If (3.11) is

written in terms of the density function , is seen to be the
Fourier transform (with the sign of w revers
ed of .)

17
Because of this fact, if is known, can be found from the
inverse Fourier transform (with sign of x reversed).

By formal differentiation of (3.12) n times with respect to w

and setting w=0 in the derivative, we may show that the
nth moments of X is given by

A major advantage of using to find moments is that

always exists, so the moments can always be found if is
known, provided, of course, the derivatives of exists.
18
It can be shown that the maximum magnitude of a
characteristic function is unity and occurs at w=0; that is,

Example :consider the random variable with the exponential

density given below and find its characteristic function
and first moment.

By substituting the density function into (3.12), we get

19
The derivative of is

So the first moment becomes

20
 Moment Generating Function
 Another statistical average closely related to the
characteristic function is the moment generating function,
defined by

 The main advantage of the moment generating function

derives from its ability to give the moments. Moments are
related to by the expression
:

21
Example :To illustrate the calculation and use of the
moment generating function, let us reconsider the
exponential density of the earlier example.

By differentiation we have the first moment

22
Some Special Distributions of with their Special application

23
24
25
26
27
28
29
30
31
Some Special Distributions

i. Continuous Probability Distributions

1. Normal (Gaussian) Distribution
 The random variable X is said to be normal or Gaussian
random variable if its pdf is given by:
1  ( x   ) 2 / 2 2
f X ( x)  e .
2 2
 The corresponding distribution function is given by:
x 1  x 
FX ( x )   e  ( y   ) 2 / 2 2
dy  G 

2 2
  
x 1  y2 / 2
where G ( x)  

2
e dy
32
Some Special Distributions Cont’d……
 The normal or Gaussian distribution is the most common continuous probability distribution.

f X (x )

x

Fig. Normal or Gaussian Distribution

33
 Some Special Distributions Cont’d……
2. Uniform Distribution f X (x )
1
 1 ba
 , a xb x
f X ( x)   b  a a b

3. Exponential Distribution
 0, otherwise. Fig. Uniform Distribution

f X (x )

 e  x , x  0,
f X ( x)   x
 0, otherwise.
Fig. Exponential Distribution

34
 Some Special Distributions Cont’d……
4. Gamma Distribution

5. Beta Distribution
 x 1 x / 
 e , x  0,
where
f X ( x )   ( )  


 0, otherwise.

 1
 x a 1 (1  x) b 1 , 0  x  1,
f X ( x )    ( a , b)

 0, otherwise.

1
 ( a , b)  0
u a 1 (1  u ) b1 du.

35
 Some Special Distributions Cont’d……
6. Rayleigh Distribution

 x  x 2 / 2 2
7. Cauchy Distribution

 2e , x  0,
f X ( x )  
8. Laplace Distribution


 0, otherwise.

 /
f X ( x)  ,    x  .
  (x  )
2 2

1 |x|/ 
f X ( x)  e ,    x  .
2
36
Some Special Distributions Cont’d….

I. Discrete Probability Distributions

1. Bernoulli Distribution
P ( X  0)  q, P ( X  1)  p.
2. Binomial Distribution
 n  k n k
P( X  k )   k p q , k  0,1,2,  , n.
 

3. Poisson Distribution
k
P ( X  k )  e  , k  0,1,2,  , .
k!

37
Some Special Distributions Cont’d….

4. Hypergeometric Distribution
m  N m 
   
k   n k 
P( X  k )    
N

, max(0, m  n  N )  k  min( m, n )
 
n 
 

5. Geometric Distribution

P ( X  k )  pqk , k  0,1,2,, , q  1  p.

6. Negative Binomial Distribution

 k  1 r k  r
P( X  k )    p q , k  r , r  1,  .
 r 1
38
Random Variable Examples

Example-1:
The pdf of a continuous random variable is given by:
kx , 0  x 1
f X ( x)  
0 , otherwise

where k is a constant.
a. Determine the value of k .
b. Find the corresponding cdf of X .
c. Find P(1 / 4  X  1)
d . Evaluate the mean and variance of X .
39
Random Variable Examples Cont’d……

Solution:
 1
a. 
f X ( x ) dx  1  
0
kxdx  1
 x2 1
 k    1
 2 0
k
 1
2
k  2

2 x , 0  x 1
 f X ( x)  
0, otherwise
40
Random Variable Examples Cont’d……

Solution:
b. The cdf of X is given by :
x
FX ( x )   
f X (u ) du
Case 1 : for x  0
FX ( x )  0, since f X ( x )  0, for x  0
Case 2 : for 0  x  1
x x x
FX ( x )   f X (u ) du   2udu  u  x2
2
0 0 0

41
Random Variable Examples Cont’d……

Solution:
Case 3 : for x  1
1 1 1
FX ( x )   f X (u ) du   2udu  u 1
2
0 0 0
 The cdf is given by
0, x0
 2
FX ( x )   x , 0  x 1
1, x 1


42
Random Variable Examples Cont’d……

Solution:
c. P (1 / 4  X  1)
i. Using the pdf
1 1
P (1 / 4  X  1)   f X ( x) dx   2 xdx
1/ 4 1/ 4

1
 P (1 / 4  X  1)  x 2
 15 / 16
1/ 4
 P (1 / 4  X  1)  15 / 16
ii. Using the cdf
P (1 / 4  X  1)  FX (1)  FX (1 / 4)
 P (1 / 4  X  1)  1  (1 / 4) 2  15 / 16
 P (1 / 4  X  1)  15 / 16
43
Random Variable Examples Cont’d……

Solution:
d. Mean and Variance
i. Mean
1 1
 X  E ( X )   xf X ( x) dx   2 x 2 dx
0 0

2 x3 1
 X   2/3
3 0
ii. Variance
 X  Var ( X )  E ( X 2 )  [ E ( X )]2
2

1 1
E ( X )   x f X ( x )dx   2 x 3 dx  1 / 2
2 2
0 0

  X  Var ( x)  1 / 2  (2 / 3) 2  1 / 18
2

44
Random Variable Examples Cont’d……..

Example-2:
Consider a discrete random variable X whose pmf is given by:
1 / 3 , xk  1, 0, 1
PX ( xk )  
0 , otherwise

Find the mean and variance of X .

45
Random Variable Examples Cont’d……

Solution:
i. Mean
1
 X  E( X )  x
k  1
k PX ( xk )  1 / 3(1  0  1)  0

ii. Variance
 X 2  Var ( X )  E ( X 2 )  [ E ( X )]2
1

 k X k
2
E( X ) 2
x P ( x )  1 / 3[( 1) 2
 ( 0) 2
 (1) 2
]  2/3
k  1

  X  Var ( x)  2 / 3  (0) 2  2 / 3
2

46
 Functions of One Random Variable
 Let X be a continuous random variable with pdf fX(x) and suppose g(x) is a function of the
random variable X defined as:

 gpdf
 We can determine the cdfYand ( X of
) Y in terms of that of X.

 Consider some of the following functions.

aX  b
sin X X2

1
Y  g( X ) |X |
X
X
log X
eX | X | U ( x)

47
 Functions of a Random Variable Cont’d…..
 Steps to determine fY(y) from fX(x):
Method I:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. Determine the cdf of Y using the following basic approach.

3. Obtain fY(y) from FY(y) by using direct differentiation, i.e.,

FY ( y )  P ( g ( X )  y )  P (Y  y )

dFY ( y )
fY ( y ) 
dy

48
 Functions of a Random Variable Cont’d…..
Method II:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. If Y=g(X) is one to one function and has an inverse transformation x=g-1(y)=h(y), then the pdf of Y is given by:

3. Obtain Y=g(x) is not one-to-one function, then the pdf of Y can be obtained as follows.
i. Find the real roots of the function Y=g(x) and denote them by xi

dx dh( y )
fY ( y )  f X ( x)  f X [h( y )]
dy dy

49
 Functions of a Random Variable Cont’d…..

ii. Determine the derivative of function g(xi ) at every real root xi , i.e. ,

iii. Find the pdf of Y by using the following formula.

dxi
g ( xi ) 
dy

dxi
f Y ( y)   f X ( xi )   g ( xi ) f X ( xi )
i dy i

50
Examples on Functions of One Random Variable

Examples:
a. Let Y  aX  b. Find f Y ( y ).

b. Let Y  X 2 . Find f Y ( y ).

1
c. Let Y  . Find f Y ( y ).
X
 
d . The random variable X is uniform in the interval [ , ].
2 2
If Y  tan X , determine the pdf of Y .

51
Examples on Functions of One Random Variable…..

Solutions:
a. Y  aX  b

i. Using Method  I

Suppose that a  0

 y b
Fy ( y )  P (Y  y )  P (aX  b  y )  P X  
 a 

 y b
FY ( y )  FX  
 a 
dFY ( y ) 1  y b
 f Y ( y)   fX   (i )
dy a  a 

52
Examples on Functions of One Random Variable…..

Solutions:

a. Y  aX  b

i. Using Method  I

On the other if a  0, then

 y b
Fy ( y )  P (Y  y )  P (aX  b  y )  P X  
 a 

 y b
FY ( y )  1  FX  
 a 
dFY ( y ) 1  yb
 f Y ( y)    fX   (ii )
dy a  a 

53
Examples on Functions of One Random Variable…..

Solutions:

a. Y  aX  b
i. Using Method  I

From equations (i ) and (ii ) , we obtain :

1  y b
f Y ( y)  fX  , for all a
a  a 

54
 Examples on Functions of One Random Variable…..

Solutions:
a. Y  aX  b
ii. Using Method  II
The function Y  aX  b is one - to - one and the range
space of Y is IR
y b
For any y, x   h( y ) is the principal solution
a
dx dh( y ) 1 dx 1
   
dy dy a dy a
dx dh y  1  y b
f Y ( y)  f X ( x)  f X  h( y )   f Y ( y)  fX  
dy dy a  a 

55
Examples on Functions of One Random Variable…..

Solutions:

b. The function Y  X 2 is not one - to - one and the range

space of Y is y  0
For each y  0, there are two solutions given by
x1   y and x 2  y

56
Examples on Functions of One Random Variable…..

Solutions:
dx1 1 dx1 1
b.    and
dy 2 y dy 2 y

dx2 1 dx2 1
  
dy 2 y dy 2 y

dxi dx1 dx2

f Y ( y)   f X ( xi )  f Y ( y )  f X ( x1 )  f X ( x2 )
i dy dy dy

 1
2 y f X  y   f  y  ,
X y0
 f Y ( y)  

 0, otherwise
57
Examples on Functions of One Random Variable…..

Solutions:
1
c. The function Y  is one - to - one and the range
X
space of Y is IR / 0
1
For any y, x   h( y ) is the principal solution
y
dx dh( y ) 1
  2
dy dy y

dx dh y  1 1
f Y ( y)  f X ( x)  f X  h( y )   f Y ( y )  2 f X  
dy dy y  y

1 1
 fY ( y)  2
f X 
  , IR / 0
y  y
58
Examples on Functions of One Random Variable…..

Solutions:
d . The function Y  tan X is one - to - one and the range

space of Y is (, )

For any y, x  tan 1 y  h( y ) is the principal solution

dx dh( y ) 1
 
dy dy 1 y2

dx dh y  1/ 
f Y ( y)  f X ( x)  f X  h( y )   f Y ( y) 
dy dy 1 y2

1
 f Y ( y)  ,   y  
 (1  y )
2

59
Examples on Functions of One Random Variable…..

Solutions:

60
Assignment-II

1. The continuous random variable X has the pdf given by:

k (2 x  x 2 ) , 0  x  2
f X ( x)  
0 , otherwise

where k is a constant.

Find :
a. the value of k .
b. the cdf of X .
c. P ( X  1)
d . the mean and variance of X .
61
Assignment-II Cont’d…..

2. The cdf of continuous random variable X is given by:

0 , x0


F ( x )  k x , 0  x  1
X


1, x 1

where k is a constant.
Determine :
a. the value of k .
b. the pdf of X .

c. the mean and variance of X .

62
Assignment-II Cont’d…..

3. The random variable X is uniform in the interval [0, 1]. Find

the pdf of the random variable Y if Y=-lnX.

63