Chapter 5.

Probability and
Random Process

Review of Probability

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 1

 Introduction

⚫ Deterministic signals: the class of signals that may be modeled

as completely specified functions of time.
⚫ A signal is “random” if it is not possible to predict its precise
value in advance.
⚫ A random process consists of an ensemble (family) of sample
functions, each of which varies randomly with time.
⚫ A random variable is obtained by observing a random process at
a fixed instant of time.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 2

 Probability
⚫ The probability theory is rooted in the phenomenon, that can be modeled
by an experiment whose outcome cannot be predicted with certainty.
⚫ Some of the random experiments includes, tossing a coin, throwing a die
and drawing a card from a deck.
⚫ Sample space : The set of all possible outcomes is called the sample
space denoted by Ω.
⚫ The outcomes are denoted by ω’s and each ω lies in Ω. Events are
subset of the sample space.
Example: Experiment of tossing a coin, the possible out comes are head
and a tail
Ω= [Head Tail ] where ω1= head and ω2= tail
⚫ A sample space is discrete if the number of its elements are finite or
countably infinite, otherwise it is a non-discrete sample space.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 3
 Probability Conti…
⚫ Sample space corresponding to a random experiment whose out
comes are set of all values which is infinite and uncountable is
non-discrete or continuous sample space.
Example: choosing a real number from 0 to 1
⚫ Disjoint events or Mutually exclusive events: Two events are
mutually exclusive if occurrence of one event precludes the
occurrence of the other event or their intersection is empty.
Example: In the experiment of throwing a die, the event of getting
an out come of odd number and the event of getting an out come
which is divisible by 2.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 4

 Probability Conti…
⚫ Probability P can be defined as set of function assigning
non negative values to all events E such that the following
conditions are satisfied

⚫ Some important
properties of the
probability are

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 5

 Conditional Probability
⚫ Let P[E1|E2] denote the probability of event E1, given that event E2
has occurred. The probability P[E1|E2] is called the conditional
probability of E1 given E2 which is given by

⚫ If it happens that P (E1| E2) = P (E1), then the knowledge of E2 does

not change the probability of E1 . In this case, the events E1 and E2 are
said to be independent. For independent events,
P (E1 Ո E2) = P (E1)P (E2).
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 6
 Example 5.1.1
In an experiment of throwing a fair die, find the probability of
getting outcome
a) greater than 3

b) an even number

c) Also find P(A\B)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 7

 Example 5.1.1

a) event greater than 3

b) getting an even number

c) P(A|B)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 8

 Total Probability Theorem and Bay’s Theorem
⚫ Total Probability Theorem
If the events {Ei} [i=1, 2, n ] are disjoint and their union is the entire sample
space, then they make a partition of the sample space Ω. Then, if for an event A,
we have the conditional probabilities
{P (A | Ei)} [i=1, 2, n ] , P (A) can be obtained by applying the total probability
theorem stated as

⚫ Bay’s Theorem
Bayes's rule gives the conditional probabilities P (Ei | A) by the relation

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 9

 Example 5.1.2
In a certain city, 50% of the population drive to work, 30%
take the subway, and 20% take the bus. The probability of
being late for those who drive is 10%, for those who take the
subway is 3%, and for those who take the bus is 5%.
1. What is the probability that an individual in this city will be
late for work?
2. If an individual is late for work, what is the probability that
he drove to work?

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 10

 Example 5.1.2
Solution: Let D, S, and B denote the events of an individual driving, taking
the subway, or taking the bus. Then P (D) = 0.5, P(S) = 0.3, and
P (B) = 0.2. If L denotes the event of being late, then, from the
assumptions, we have
P(L / D) = 0. 1 ; P(L/S) = 0.03; P(L /B) = 0.05.
1. From the total probability theorem,
P (L) = P(D)P(L/D) + P(S)P(L / S) + P (B)P(L / B)
= 0.5 x 0. 1 + 0.3 x 0.03 + 0.2 x 0.05
= 0.069.
2. Applying Bay’s rule we have

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 11

 Example 5.1.3
In a binary communication system, the input bits transmitted over the
channel are either 0 or 1with probabilities 0.3 and 0.7, respectively.
When a bit is transmitted over the channel, it can be either received
correctly or incorrectly (due to channel noise). Let us assume that if a
0 is transmitted, the probability of it being received in error (i.e.,
being received as 1) is 0.01, and if a 1 is transmitted, the probability
of it being received in error (i.e., being received as 0) is 0. 1 .
1 . What is the probability that the output of this channel is 1 ?
2 . Assuming we have observed a 1 at the output o f this channel, what
is the probability that the input to the channel was a 1 ?

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 12

 Example 5.1.3
Solution: Let X denote the input and Y denote the output. From the
problem assumptions, we have

P(X = 0) = 0.3; P(X = 1) = 0.7;

P (Y = 0/X = 0) = 0.99; P(Y = 0/X = 1) = 0. 1 ;
P(Y = l / X = 0) = 0.0 1 ; P(Y = l / X = 1) = 0.9.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 13

1. From total probability theorem
P(Y = 1) = P (Y = 1 , X = 0) + P(Y = 1 , X = 1)
= P (X = 0)P(Y = l / X = 0) + P(X = l)P(Y = l / X = 1)
= 0.3 x 0.01 + 0.7 x 0.9
= 0.003 + 0.63
= 0.633.

2. From Bay’s rule we have

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 14

 Random Variables

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 15

 5.1.3 Random Variable
⚫ A random variable is a mapping from the sample space Ω to the set of
real numbers. OR
⚫ A random variable is an assignment of real numbers to the outcomes of a
random experiment. A schematic diagram representing a random
variable is given in figure 5.1 below

Figure 5.1 A random variable as a mapping from Ω to R

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 16
 Random Variable cont…
⚫ Example: Throw a die once
Random Variable X = "The score shown on the top face".
X could be 1, 2, 3, 4, 5 or 6
So the Sample Space Ω = {1, 2, 3, 4, 5, 6}
⚫ Example: Tossing a fair coin

Sample Space Ω = { Head, Tail }

X is the random variable which takes the
value 1 when head shows up and takes Head 1
-1 when tail shows up X
-1
Tail

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 17

 Random Variable Cont…
⚫ Random variables are denoted by capital letters X, Y, and so on.
⚫ Individual values of the random variable X are X( ω).
⚫ A random variable is discrete if the range of its values is either finite or
countably infinite. This range is usually denoted by {xi}.
⚫ A continuous random variable is one in which the range of values is a
continuum.
⚫ The cumulative distribution function or CDF : It is defined as

Which can be simply written as FX(x) = P( X ≤ x )

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 18

 Properties of CDF

⚫ For discrete random variables, Fx(x) is a staircase function.

⚫ A random variable is continuous if Fx (x) is a continuous function.
⚫ A random variable is mixed if it is neither discrete nor continuous.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 19

 Figure 5.2 CDF for a discrete
random variable

Figure 5.3 CDF for a continuous

random variable

Figure 5.4 CDF for a mixed

random variable

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 20

 Probability density function (PDF)
If the distribution function is continuously differentiable, then

fX(x) is called as probability density function(PDF) of the

random variable X. Probability density function is always a
non negative function with a total area of one unit.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 21

 Properties of PDF

For discrete random variables, we generally define the probability mass

function, or PMF which is defined as {pi}, where pi= P(X = xi).
Obviously for all i we have pi ≥ 0 and

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 22

 Example: Coin Flips
When a coin is flipped 3 times the sample space will be
Ω = { HHH, HHT, HTH, THH, HTT, THT, TTH, TTT }

If X is the number of heads, then X is a random variable whose

probability distribution is as follows

Possible Events X P(X)

TTT 0 1/8
HTT, THT, TTH 1 3/8
HHT, HTH, THH 2 3/8
HHH 3 1/8
Total 1
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 23
 Important Random Variables
⚫ Bernoulli random variable.
⚫ Binomial random variable.
⚫ Uniform random variable
⚫ Gaussian or normal random variable.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 24

 Bernoulli random variable
⚫ This is a discrete random variable taking two values 1 and 0, with
probabilities p and 1 - p.
⚫ A Bernoulli random variable is a good model for a binary-data generator.
⚫ When binary data is transmitted over a communication channel some bits
are received in error.
⚫ We can model an error by modulo-2 addition of a 1 to the input bit, thus,
we change a 0 into a 1 and a 1 into a 0.
⚫ Therefore, a Bernoulli random variable can also be employed to model the
channel errors

Figure 5.5 The PMF for the

Bernoulli random variable

25

2/4/2021
Latha, Dept. of ECE, ASE, Bengaluru
 Binomial random variable
⚫ This is a discrete random variable giving the number of 1 's in a
sequence of n-independent Bernoulli trials. The PMF is given by

⚫ Example : it is used to model the total number of bits received in error

when a sequence of n bits is transmitted over a channel with a bit-
error probability of p.

Figure 5.6 The PMF for the binomial

random variable
2/4/2021 26
Latha, Dept. of ECE, ASE, Bengaluru
 Example 5.1.5
Assume 10,000 bits are transmitted over a channel
in which the error probability is 10-3. What is the
probability that the total number of errors is less
than 3?

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 27

 Example 5.1.5
Assume 10,000 bits are transmitted over a channel in which the error
probability is 10-3. What is the probability that the total number of
errors is less than 3?
Solution : In this example, n = 10,000, p = 0.001, and we are looking
for P (X < 3). We have

2/4/2021 28

Latha, Dept. of ECE, ASE, Bengaluru

 Uniform random variable
This is a continuous random variable taking values between a and b
with equal probabilities for intervals of equal length. The density
function is given by

Figure 5.7 The PDF for the uniform random

variable.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 29

 Gaussian or normal random variable
⚫ The Gaussian, or normal, random variable is a continuous random
variable described by the density function.

⚫ There are two parameters involved in the definition of the Gaussian

random variable.
⚫ The parameter m is called the mean and can assume any finite value.
⚫ The parameter σ is called the standard deviation and can assume any
finite and positive value.
⚫ The square of the standard deviation, i.e., σ 2, is called the variance.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 30
A Gaussian random variable with mean and variance is denoted
by N(m, σ 2). The random variable N(0, 1) is usually called standard
normal.

Figure 5.8 The PDF for the Gaussian random

variable.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 31

 Q-Function
⚫ Assuming that X is a standard normal random (N(0, 1)) variable, we
define the function Q (x) as P (X > x). The Q-function is given by the
relation

⚫ This function represents the area under the tail of a standard normal
random variable
⚫ Q-function is a decreasing function
⚫ This function is well tabulated and
frequently used in analyzing the
performance of communication systems
Q(x) using calculator(fx-991ES)
Mode 3→AC
Shift(1) →5 →3: R(x)= Q(x) Figure 5.9 The Q-function as the area under32 the
2/4/2021 tail of a standard normal random variable
Latha, Dept. of ECE, ASE, Bengaluru
 33
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru
Q(x) satisfies the relations

Two important upper bounds on the Q-function are widely used to find
bounds on the error probability of various communication systems.

These bounds are given as

A frequently used lower bound is

 x−m
For N(m, σ 2) random variable Q ( x ) = P ( X  x ) = Q 
  
2/4/2021 34
Latha, Dept. of ECE, ASE, Bengaluru
 Example 5.1.6
⚫ X is a Gaussian random variable with mean 1 and variance 4. Find the
probability that X is between 5 and 7

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 35

 5.1.4 Function of a Random Variable
⚫ A linear function of a Gaussian random variable is itself a Gaussian
random variable.
⚫ If X is N(m, σ 2), and random variable Y is a linear function of X
such that Y = aX + b, then we can show that Y is also a Gaussian
random variable of the form N(am + b, a2σ 2)
Example 5.1.8
Assume X is a N(3, 6) random variable. Find the density function of
Y = -2X + 3.
a=-2, b=3, m=3, σ 2=6
Mean of Y mY=am +b=-6+3=-3,
Variance of Y = σ 2Y= a*2σ 2=4*6=24
Hence Y is N(-3, 24)
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 36
 Statistical Averages
The mean, expected value, or expectation of a continuous random
variable X is defined as

If Y=g(X), then

The mean, expected value, or expectation of a discrete random

variable X is defined as

If Y=g(X), then

In general, the nth moment of a random variable X is defined as

2/4/2021 37
Latha, Dept. of ECE, ASE, Bengaluru
 Variance of X
⚫ In the special case where g(X) = (X - E(X))2
⚫ E(g(X)) is called the variance of X
⚫ E(g(X))=E( X – E (X) )2
= E [ X2 + (E(X))2 - 2X E(X)]
= E(X2) + (E(X))2 -2 E(X) E(X)
= E(X2) + (E(X))2 – 2(E(X))2
E(g(X)) = E(X2) – (E(X))2
⚫ The variance is denoted by σ2 and its square root σ, is called
the standard deviation
σ2 = E(X2) – (E(X))2

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 38

⚫ Variance is a measure of the spread of the density function of X.
⚫ If the variance is small, it indicates that the random variable is very
concentrated around its mean, and in a sense is "less random."
However, if the variance is large, then the random variable is highly
spread hence, it is less predictable.

For any constant c, these relations hold

Variance has these properties

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 39

 Mean and variance of the important
random variables

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 40

 Multiple Random Variable
Let X and Y represent two random variables defined on the same sample
space Ω. For these two random variables, we can define the joint CDF as

Or simply as

The joint PDF denoted as fX,Y(x, y) is defined by

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 41

The basic properties of the joint and marginal CDFs and PDFs can be
summarized with the following relations:

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 42

 The conditional probability density function of the random variable Y,
given that the value of the random variable X is equal to x, is denoted
by fY|X(y|x) and defined as

If the density function after the knowledge of X is the same as the

density function before the knowledge of X, then the random
variables are said to be statistically independent. For statistically
independent random variables, f X,Y(x, y) = fX(x) fY(y)

If g(X, Y) is a function of X and Y then the expected value of g(X, Y) is

obtained from

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 43

 Example 5.1.9

Let X be N(3, 4) and Y be N(-2, 6) . Assuming X and Y are

independent, determine fX,Y(x, y).
Solution: We have , for independent random variables X and Y
f X,Y(x, y) = fX(x) fY(y)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 44

 Correlation and Covariance
⚫ E (XY) is called the correlation of X and Y which is given by
 
E ( XY ) =  XY f
− − 
X ,Y ( x, y ) dx dy

⚫ If X and Y are independent, then their correlation

E(XY) = E(X) E(Y)

⚫ The covariance of X and Y is defined as

COV(X, Y) = E(X Y) - E(X) E(Y)

⚫ If COV(X, Y) = 0, i.e., if E(XY) = E(X) E(Y), then X and Y are
called uncorrelated random variables.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 45
 Correlation coefficient is the normalized version of the covariance
and it is denoted by ρX,Y and is defined as

⚫ It is obvious that if X and Y are independent, then COV(X, Y) = 0

⚫ Independence implies lack of correlation but lack of correlation does
not generally imply independence.
⚫ Covariance might be zero, but the random variables may still be
statistically dependent.
⚫ Properties of the expected value and variance applied to multiple random
variables (In these relations, the ci 's are constants.)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 46

 Example 5.1.10
Assuming that X is N(3, 4), Y is N(- 1 , 2), and X and Y are
independent, determine the covariance of the two random variables
Z = X - Y and W = 2X + 3Y.

COV(W, Z) = E(WZ) - E(W)E(Z)

WZ= (X-Y) (2X+3Y)= 2X2 + XY - 3Y2

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 47

 Example 5.1.10
Assuming that X is N(3, 4), Y is N(- 1, 2), and X and Y are
independent, determine the covariance of the two random variables
Z = X - Y and W = 2X + 3Y.
COV(W, Z) = E(WZ) - E(W)E(Z)
WZ= (X-Y) (2X+3Y)= 2X2 + XY - 3Y2
E (Z) = E (X) - E (Y) = 3 + 1 = 4,
E(W) = 2E(X) + 3E(Y) = 6 - 3 = 3,
E (X2) = VAR(X) + (E (X))2 = 4 + 9 = 13,
E (Y2) = VAR(Y) + (E(Y))2 = 2 + 1 = 3, and E (XY) = E(X)E(Y) = -3.
COV(W, Z) = E(WZ) - E(W)E(Z)
= E (2X2 - 3Y2 + XY) - E(Z)E(W)
= 2E(X2) – 3E(Y2) + E(XY) - E(Z)E(W)
= (2 x 13) – (3 x 3) - 3 – (4 x 3)
2/4/2021 =2 Latha, Dept. of ECE, ASE, Bengaluru 48
 Jointly Gaussian Random Variables
Jointly Gaussian or binormal random variables X and Y are
distributed according to a joint PDF of the form

where m1 , m2, σ12 and σ22 are the mean and variances of X and Y,
respectively, and ρ is their correlation coefficient.
When two random variables X and Y are distributed according to a
binormal distribution, then X and Y are normal random variables and
the conditional densities f(x|y) and f(y|x) are also Gaussian
The definition of two jointly Gaussian random variables can be extended
to more random variablesLatha, Dept. of ECE, ASE, Bengaluru
2/4/2021 49
 Properties of Jointly Gaussian Random Variables
⚫ If n random variables are jointly Gaussian, any subset of them is also
distributed according to a jointly Gaussian distribution of the appropriate
size. In particular, all individual random variables are Gaussian.
⚫ Jointly Gaussian random variables are completely characterized by the
means of all random variables m1 , m2,, . . . , mn and the set of all
covariance COV(Xi , Xj) for all 1 ≤ i ≤ n and 1 ≤ j ≤ n. These so-called
second-order properties completely describe the random variables.
⚫ Any set of linear combinations of (X 1 , X2, . . . , Xn) are themselves
jointly Gaussian. In particular, any linear combination of Xi 's is a
Gaussian random variable.
⚫ Two uncorrelated jointly Gaussian random variables are independent.
Therefore, for jointly Gaussian random variables, independence and
uncorrelatedness are equivalent. As previously stated, this is not true for
general random variables.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 50
 5.1 .6 Sums of Random Variables
Law of large numbers and central limit theorem
⚫ Law of large numbers: It states that if the sequence of random
variables X1 , X2, . . . , Xn are uncorrelated with the same mean mx
and variance σ2 < ∞ or Xi 's are i.i.d. (independent and identically
distributed) random variables then for any

where

This means that the average converges (in probability) to the expected
value.

2/4/2021 51
Latha, Dept. of ECE, ASE, Bengaluru
⚫ Central limit theorem : This theorem states that if
Xi 's are i.i.d. (independent and identically distributed)
random variables which each have a mean m and variance σ2,
Then converges to a N( m, σ2/n ). Central limit
theorem states that the sum of many i.i.d. random variable
converges to a Gaussian random variable. This theorem
explains why thermal noise follows a Gaussian distribution.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 52

Example : A random variable X takes the values 0 and 1 with
probability α and β= 1- α respectively. Find the mean and
variance of X.
Solution : It is given that P(X=0) = α, and P(X=1) = 1- α
We know that the mean of X is
E(X) =Σ xi P(xi)
= 0. α + 1. β = β

E(X2) = Σ xi2 P(xi)

= 0. α + 12. β = β
The variance of X is σx2 = E(X2) – (E(X))2
= β – β2 = β (1 – β )
σx2 = β α
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 53
Example : Binary data are transmitted over a noisy communication channel
in a block of 16 binary digits. The probability that the
received bit is in error is 0.01. Assume that the error occuring in
various digit positions in a block are independent.
(a) Find the mean or average error per block

(b) Find the variance of the number of errors per block

(c) Find the probability that the number of errors per block is greater than or
equal to 4.
Solution:
(a) Let X representing the number of errors per block. X has binomial
distribution with n=16 and p=0.01. Average number of errors per block
is E(X) = np = 0.16
(b) Variance σx2 = np (1-p) = 0.158
(c) P(X ≥ 4 )= 1- P(X ≤ 3)
= 1-[ P(X=0) + P(X = 1) + P(X = 2) + P(X = 3)]
2/4/2021 54

= 0.014 Latha, Dept. of ECE, ASE, Bengaluru

Example : The pdf of a random variable X is given by
k a  x  b
f X ( x) = 
0 otherwise
(a) Determine the value of k
(b) Let a =-1 and b = 2. Calculate P(|X| ≤ c) for c=0.5

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 55

Example : The pdf of a random variable X is given by
k a  x  b
f X ( x) = 
0 otherwise where k is a constant
(a) Determine the value of k
(b) Let a =-1 and b = 2. Calculate P(|X| ≤ c) for c=0.5

Solution  b
1
(a) We know that
 f ( x ) dx =  k dx = 1 Or k =
−
X
a
b−a

 1
 a  xb
Hence f X ( x) =  b − a
0 otherwise

X is an uniform random variable

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 56

(b) With a=-1 and b=2 we have
1
 −1  x  2
f X ( x) =  3

0 otherwise

 1  1 1
1/ 2 1/ 2
1 1
P | X |   = P  −  X   =
 2  2 2 
−1 / 2
f X ( x) dx =  dx =
−1 / 2
3 3

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 57

Example 5.10 : X is a Gaussian random variable with a
mean 4 and a variance 9, i.e., N(4, 9).
Determine the following probabilities:
1. P(X > 7).
2. P (0 < X < 9).

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 58

Example 5.10 : X is a Gaussian random variable with a mean 4
and a variance 9, i.e., N(4, 9).
Determine the following probabilities:
1. P(X > 7).
2. P (0 < X < 9).

7−4
1. P ( X  7) = Q  = Q(1) = 0.158
 3 
0−4 9−4  4  5
2. P(0  X  9) = Q  −Q  = Q −  − Q 
 3   3   3  3
4 5
= 1 − Q  − Q 
3 3
= 0.858

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 59

Example 5.11:The noise voltage in an electric circuit can be
modeled as a Gaussian random variable with a mean equal to
zero and a variance equal to 10-8 .What is the probability that the
value of the noise exceeds 10-4?What is the probability that it exceeds
4 x 10-4? What is the probability that the noise value is between
-2 x 10-4 and 10-4?
Solution: X is a Gaussian random variable with mean zero and variance
σ2 = 10-8 . Hence x
P ( X  x ) = Q 
 

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 60

Example 5.22 : Two random variables X and Y are distributed according to

1. Find the value of the constant K.

2. Find the marginal probability density functions of X and Y.
3. Are X and Y independent?
4. Find fX|Y(x|y).
5. Find E(X|Y = y).
6. Find COV(X, Y) and ρX,Y
1. We know that  

f
− − 
X ,Y ( x, y ) dx dy =1

  


− − 
f X ,Y ( x, y ) dx dy =   K e − x − y dx dy
0 y
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 61
    


− − 
f X ,Y ( x, y ) dx dy = K  e − y  e − x dx dy
0 y
  
e−x
f ( x, y ) dx dy = K  e −y 
dy
−1
X ,Y y
− −  0
   


− − 
f X ,Y ( x, y ) dx dy = K  e − y e − y dy = K  e − 2 y dy
0 0

 
e −2 y
−− f X ,Y ( x, y) dx dy = K − 2

0

K
= =1
2
 K =2

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 62

2. The marginal density functions are

3. Two random variables are said to be independent if

f X,Y (x, y) = fX (x) fY (y)

LHS= f X,Y (x, y) = 2e-x-y

RHS= fX (x) fY (y) = 2e-x(1-e-x) 2 e-2y

LHS ≠ RHS
Hence X and Y are not independent
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 63
4. Conditional density function
If x < y then fX | Y(x|y) =0

If x ≥ y then

5.

6. Covariance of X and Y
COV(X, Y) = E(X Y) - E(X) E(Y)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 64

 Here we are using

2/4/2021 65
Latha, Dept. of ECE, ASE, Bengaluru
 Hence

and

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 66

 Random Process

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 67

 Introduction
⚫ A random process is the natural extension of random variables
when dealing with signals.
⚫ In many situations, the deterministic assumption on time-varying
signals is not a valid assumption, and it is more appropriate to model
signals as random rather than deterministic functions.
⚫ One such example is the case of thermal noise in electronic circuits.
⚫ Thermal noise is due to the random movement of electrons as a result
of thermal agitation, therefore, the resulting current and voltage can
only be described statistically.
⚫ Another example is information source, such as a speech source,
generates time-varying signals whose contents are not known in
advance. Otherwise there would be no need to transmit them.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 68

⚫ A random process, or a random signal, can be viewed as a
set of possible realizations of signal waveforms.
⚫ The realization of one from the set of possible signals is governed by
some probabilistic law.
⚫ Example 5.2.1: Assume that we have a signal generator that can
generate one of the six possible sinusoidal signals. The amplitude of
all sinusoids is one, and the phase for all of them is zero, but the
frequencies can be 100, 200, . . . , 600 Hz. We throw a die, and
depending on its outcome, which we denote by F, we generate a
sinusoid whose frequency is 100 times what the die shows (l00F).
This means that each of the six possible signals will be realized with
equal probability . This is an example of a random process. This
random process can be defined as
X (t) = cos (2π x100Ft). (x(t) =A cos (2π f t+ϕ))

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 69

Example 5.2.2: Assume that we uniformly choose a phase Θ between 0 and 2π
and generate a sinusoid with a fixed amplitude and frequency but with a
random phase Θ. In this case, the random process is X (t) = A cos(2πf0t + Θ),
where A and f0 denote the fixed amplitude and frequency and Θ denotes the random
phase. Some sample functions for this random process are shown in

Figure 5.11 Sample functions of the

random process given in Example 5.2.2.

2/4/2021 70
Latha, Dept. of ECE, ASE, Bengaluru
 The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. In this case, an analytic description of the
random process is given. For this random process, each sample is a constant signal

Figure 5.12 Sample functions of the random

process given in Example 5.2.3

2/4/2021 71
Latha, Dept. of ECE, ASE, Bengaluru
⚫ we see that corresponding to each outcome ωi in a sample space Ω,
there exists a signal x ( t ; ωi).
⚫ This description is similar to the description of random variables in
which a real number is assigned to each outcome ωi.
⚫ For each ωi, there exists a deterministic time function x ( t ; ωi) , which
is called a sample function or a realization of the random process
⚫ At each time instant t0 and for each, ωi € Ω, we have the number

x(t0; ωi)
⚫ For the different outcomes (ωi 's) at a fixed time t0, the numbers

x ( t0; ωi) constitute a random variable denoted by X (t0).

⚫ As, a random variable is nothing but an assignment of real numbers to
the outcomes of a random experiment, any time instant, the value of a
random process is a random variable.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 72

 Figure 5.13 Sample functions of a random process.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 73
⚫ This is a bridge that connects the concept of a random process to
concept of a random variable
⚫ At any time instant, the value of a random process is a random
variable

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 74

 Example 5.2.4:Consider X (t) = cos (2π x100Ft). Determine the
values of random variable X (0.001).
Solution: Possible values of F are 1, 2, …6.
Possible values of X (0.001) are cos (0.2π ), cos (0.4π ), ……
cos (1.2π ).

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 75

 5.2.1 Statistical averages
⚫ At any given time, the random process defines a random variable
⚫ At time instant t0, the random process X(t0) is an ordinary random
variable, which has a density function, mean and variance at that time.
⚫ Even though mean and variance are deterministic numbers ,they
depend on the time t0.
⚫ Two random variables X(t1) and X(t2) defined at two different time
instants t1 and t2 will have different density functions, expectations and
variances.
⚫ Statistical properties of X(t1) and X(t2) are different

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 76

Mean or expectation of a random process
Definition: The mean, or expectation, of the random process X ( t) is a
Deterministic function of time denoted by mx(t) that at each time
instant t0 equals the mean of the random variable X (t0).
mx(t) = E[X (t)] for all t.
Since at any t0 the random variable X (t0) is well defined with a
probability density function fX(t0)(x), we have

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 77

 Pictorial representation of the mean of a random
process

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 78

 Autocorrelation function
Definition: The autocorrelation function of the random process
X (t), Denoted by RX (t1, t2), is defined by
RX (t1, t2) = E[X (t1) X (t2)].
From this definition, it is clear that RX(t1 , t2), is a deterministic function
of two variables t1 and t2 given by

The autocorrelation function is especially important because it

completely describes the power spectral density and the power content of
a large class of random processes.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 79
 Wide sense stationary process (WSS)
⚫ A random process observed at any given time is just a random
variable and the properties of this random variable depend on the time
at which the random process is observed.
⚫ It can happen that some of the properties of this random variable are

independent of time.
⚫ Depending on what properties are independent of time, different notions

of stationarity can be defined.

⚫ One of the most useful notions of stationarity is the notion of wide-sense

stationary (WSS) random processes.

⚫ A process is WSS if its mean and autocorrelation do not depend on
the choice of the time origin.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 80
Definition 5.2.3. A process X (t) is WSS if the following conditions
are satisfied:

1. mx(t) = E[X (t)] is independent of t.

2. RX (t1 , t2) depends only on the time difference τ = t 1- t2 not on t1 and
t2 individually.

Mean and autocorrelation of a WSS process is denoted by mx and RX (τ).

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 81

 Example 5.2.10
Consider a random process X (t) = A cos(2πf0t + Θ),
where A and f0 denote the fixed amplitude and frequency, Θ denotes
the random phase which is assumed to be uniformly distributed between
0 and 2π. Verify whether the random process X(t) is WSS.

RX (t1, t2) = E[X (t1) X (t2)]

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 82

 Example 5.2.10
Consider a random process X (t) = A cos(2πf0t + Θ),
where A and f0 denote the fixed amplitude and frequency, Θ denotes
the random phase which is assumed to be uniformly distributed between
0 and 2π. Verify whether the random process X(t) is WSS.
Solution : The mean of the random process
As Θ is uniformly distributed from 0 to 2π, It’s pdf is

We observe that, mx (t) is independent of t.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 83
 The autocorrelation function of the random process is
RX (t1, t2) = E[X (t1) X (t2)]

Here we have used

As the mean mx(t) is zero and autocorrelation is depending on (t1- t2), the
process X(t) is WSS.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 84
 Example 5.2.10

Consider a random process Y (t) = A cos(2πf0t + Θ),

where A and f0 denote the fixed amplitude and frequency and Θ denotes
the random phase which is assumed to be uniformly distributed
between 0 and π. Verify whether the random process X(t) is WSS.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 85

 Example 5.2.10
Consider a random process Y (t) = A cos(2πf0t + Θ),
where A and f0 denote the fixed amplitude and frequency and Θ denotes the random
phase which is assumed to be uniformly distributed between 0 and π. Verify
whether the random process X(t) is WSS.

Since mY (t) is not independent of t, the process Y(t) is not stationary

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 86

 5.2.3 Multiple random Process
⚫ Multiple random processes arise naturally when dealing with two or
more random processes.
⚫ For example, take the case where we are dealing with a random
process X (t) and we pass it through a linear time-invariant (LTI)
system.
⚫ For each sample function input x ( t ; ωi) , we have a sample function
output y ( t ; ωi) = x ( t ; ωi) ⁎ h(t) where h(t) denotes the impulse
response of the system.
⚫ We can see that for each ωi € Ω, we have the two signals x ( t ; ωi)
and y ( t ; ωi).
⚫ Therefore, we are dealing with two random processes, X (t) and Y (t).

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 87

 Dependency between the random processes
⚫ Independent and uncorrelated random processes
Two random processes X (t) and Y(t) are independent if for all
positive integers m, n and for all t1 , t2, . . . , tn and τ1, , τ 2, . . . , τ m,
the random vectors [X(t1), X (t2), …… X (tn)] and
[Y(τ 1), Y(τ 2) , . . Y(τ m)] are independent. Similarly, X (t)and Y (t) are
uncorrelated if the two random vectors are uncorrelated.

We know that the independence of random processes implies that they are
uncorrelated, whereas uncorrelatedness generally does not imply
independence, except for the important class of Gaussian processes for
which the two properties are equivalent.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 88

⚫ Cross correlation
The cross correlation between two random processes X (t) and
Y(t) is defined as

In general

⚫ Jointly stationary Random processes

Two random processes X (t) and Y(t) are jointly wide-sense stationary,
or simply jointly stationary, if both X (t) and Y(t) are individually
stationary and the cross-correlation RXY (t1, t2) depends only on
τ = t1 – t2.
For jointly stationary random processes
RXY(τ )= RYX(-τ )
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 89
 Example 5.2.15

Assuming that the two random processes X (t) and Y (t) are jointly
stationary, determine the autocorrelation of the process
Z(t) = X (t) + Y(t) .

Here we have used

E[ X (t+ τ) X (t)] + E[Y (t+ τ) Y (t)] + E(X (t+ τ) Y (t)] + E[Y (t+ τ) X (t)]
= RX (τ) + RY (τ) + RXY (τ) + RYX (τ)

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 90

 5.2.4 Random Processes and Linear Systems
⚫ We know that, when a random process passes through an LTI system,
the output is also a random process .
⚫ In this section our focus is on the properties of the output process
based on the knowledge of the input process.
⚫ Assuming that a stationary process X (t) is the input to an LTI system
with the impulse response h(t) and the output process is denoted by
Y(t),we would try the find the answers for the following questions.
⚫ Under what conditions will the output process be stationary?

⚫ Under what conditions will the input and output processes be

jointly stationary?
⚫ How can we obtain the mean and the autocorrelation of the output
process, as well as the cross correlation between the input and
output processes?
8/6/2020 Latha, Dept. of ECE, ASE, Bengaluru 91
 Random Process Passing through an LTI system

X(t) Y(t)
h(t)

We next demonstrate that the input and output processes X (t) and Y (t)
will be jointly stationary with

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 92

Mean of the output process mY
The output process is the convolution of the input process and the
impulse response of the system given by
Y(t) =
We have

here mY is independent of t
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 93
Cross correlation between X(t) and Y(t)
The cross-correlation function between the output and the input is

This shows that RXY (t1, t2) depends only on τ= t1 - t2.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 94
Autocorrelation function of the output process
The autocorrelation of the output is

This shows that both RY (τ) and RXY (τ) depend only on τ = t1 -t2 and,
hence, the output process is stationary. Therefore, the input and output
processes are jointly stationary.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 95
 5.2.5 Power Spectral Density of the
Stationary Processes
⚫ A random process is a collection of signals
⚫ spectral characteristics of these signals determine the spectral
characteristics of the random process
⚫ If the signals of the random process are slowly varying, then the
random process will mainly contain low frequencies and its power
will be mostly concentrated at low frequencies
⚫ On the other hand, if the signals change very fast, then most of the
power in the random process will be at the high-frequency
components
⚫ power spectral density or power spectrum of the random process is
a function that determines the distribution of the power of the random
process at different frequencies
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 96
⚫ The power spectral density of a random process X(t) is denoted by
SX (f).
⚫ Meaning of Power Spectral Density SX (f) : The value of SX (f)is the
power concentrated in a frequency interval [f ; f +df].
⚫ The unit for power spectral density is W /Hz
⚫ Wiener-Khinchin theorem: For a stationary random process X (t), the
power spectral density is the Fourier transform of the autocorrelation
function
SX(f )=FT{RX (τ)}
⚫ For a cyclostationary process, the power spectral density is the Fourier
transform of the average autocorrelation function
SX(f )=FT{Avg(RX (τ))}

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 97

 Example 5.2.16
The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1].
(a) Verify whether X(t) is WSS process. If found WSS answer part b
(b) Find the power spectral density of X(t).

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 99

 Example 5.2.16
The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. Find the power spectral density of X(t).
Solution : Here X(t) is a stationary process and autocorrelation function
is given by

Hence power spectral density

Sx(f) = FT{Rx(τ)}
= FT{⅓}
Sx(f) = ⅓ δ(f)
In this case, for each realization of the process, we have a different power
spectrum.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 100
 Power of a Random Process
⚫ Power, of a random process is the sum of the
powers at all frequencies in that random process
⚫ Total power can be obtained by integrating power spectral density
over all frequencies
⚫ Power in the random process which is denoted by PX is given by

⚫ Since SX ( f ) is the Fourier transform of RX (τ), then RX (τ), will be the

inverse Fourier transform of SX ( f ) Therefore

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 101

⚫ Substituting τ = 0 into this relation yields

⚫ Which is same as PX Therefore

PX = RX (0)

⚫ Power in a stationary random process can be found either by

integrating its power spectral density (adding all power components)
or substituting τ = 0 in the autocorrelation function of the process.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 102

⚫ Example 5.2.17:Assume that we uniformly choose a phase
Ѳ between 0 and 2π and generate a sinusoid with a fixed
amplitude and frequency but with a random phase Ѳ. In this
case, the random process is X (t) = A cos(2πf0t + Ѳ), where
A and f0 denote the fixed amplitude and frequency and Ѳ
denotes the random phase. Find the power in the process.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 103

 Example 5.2.17:Assume that we uniformly choose a phase Ѳ between 0 and
2π and generate a sinusoid with a fixed amplitude and frequency but with a
random phase Ѳ. In this case, the random process is X (t) = A cos(2πf0t + Ѳ),
where A and f0 denote the fixed amplitude and frequency and Ѳ denotes the
random phase. Find the power in the process.
Solution:
Using power spectral density Using autocorrelation function

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 104

 Power spectra in LTI system
When a stationary random process with mean mx and autocorrelation
function RX (τ) passes through an LTI system with the impulse response
h(t), the output process will be also stationary with
Mean of the output process

We know that the Fourier Transform of h(t) is

 
When f =0
H ( f ) =  h(t ) e − j 2 ft
dt H (0) =  h(t ) dt
− −
Hence mY = mX H(0)
The mean value of the response of the system only depends on the value of
H ( f ) at f = 0 (DC response), Since the mean of a random process is basically
its DC value
105
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru
Power Spectral density of the output process
Autocorrelation of the output process is

Taking FT on both the sides, we get

FT { RY (τ) }=FT { RX (τ) } FT { h(τ) } FT {h(-τ) }
SY ( f ) = SX( f ) H( f ) H*( f )
SY ( f ) = SX ( f ) | H( f ) |2

In power spectrum, the phase of H (f) is irrelevant, only the magnitude

of H (f) affects the output power spectrum. This is also intuitive since
power depends on the amplitude and not on the phase of the signal.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 106

Cross-spectral density SXY ( f )
X(t) and Y(t) are jointly stationary with cross-correlation function
RXY(τ) = RX (τ) * h(- τ). The cross spectral density is
SX Y ( f ) = FT{RXY(τ) }
FT{RXY (τ)}=FT{RX(τ)} . FT{h(-τ)}
SX Y ( f ) = SX ( f ) H*( f )
and since RYX (τ) = RXY (-τ) , we have
SY X ( f ) = S*X Y ( f )= SX ( f ) H( f )
Although SX ( f ) and SY ( f ) are real nonnegative functions, SX Y ( f ) and
SY X ( f ) can generally be complex functions.

2/4/2021 107

Latha, Dept. of ECE, ASE, Bengaluru

Example 5.2.18:Consider a random process X (t) = A cos(2πf0t + Θ),
where A and f0 denote the fixed amplitude and frequency, Θ denotes
the random phase which is assumed to be uniformly distributed
between 0 and 2π. X(t) is passed through a differentiator with
frequency response H( f ) = j2πf . Find the power spectral density of
the output process Y(t) and cross spectral density.
Example 5.2.19 :The process X (t) is defined by X (t) = X, where X is
a random variable uniformly distributed on [- 1 , 1]. Fid the power
spectral density of the process. X(t)is passed through a differentiator
with frequency response H( f ) = j2πf. Find the Power spectral density
of the output process Y(t) and cross spectral density.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 108

 Example 5.2.18
Consider a random process X (t) = A cos(2πf0t + Θ), where A and f0 denote the
fixed amplitude and frequency, Θ denotes the random phase which is assumed to be
uniformly distributed between 0 and 2π. X(t) is passed through a differentiator with
frequency response H( f ) = j2πf. Find the power spectral density of the output
process Y(t) and cross spectral density.
The power spectral density f X(t) is found to be

Output power spectral density is Cross spectral density is

SY( f ) = SX( f ) |H( f )|2 or SX Y ( f ) = SX ( f ) H*( f )

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 109

 Example 5.2.19
The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. Fid the power spectral density of the process. X(t)
is passed through a differentiator with frequency response H( f ) = j2πf. Find the
Power spectral density of the output process Y(t) and cross spectral density.
Solution: The autocorrelation of X(t) is
or

Hence power spectral density of X(t) is

Output power spectral density is Cross spectral density is

SY( f ) = SX( f ) |H( f )|2 or SX Y ( f ) = SX ( f ) H*( f )

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 110

 5.3 Gaussian and White Processes
Gaussian processes play an important role in communication systems
as thermal noise in electronic devices can be closely modeled by
Gaussian process.
Why thermal noise is a Gaussian process?
⚫ Consider a resistor. The free electrons in a resistor move as a result of
thermal agitation.
⚫ The movement of the electrons is quite random and can be in any
direction.
⚫ Their velocity is a function of the ambient temperature. The higher the
temperature, the higher the velocity of the electrons.
⚫ The movement of these electrons generates a current with a random
value.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 111
 ⚫ We can consider each electron in motion as a tiny current source,
whose current is a random variable that can be positive or negative,
depending on the direction of the movement of the electron.
⚫ The total current generated by all electrons, which is the generated
thermal noise, is the sum of the currents of all these current sources.
⚫ We can assume that at least a majority of these sources behave
independently and, therefore, the total current is the sum of a large
number of independent and identically distributed random variables.
⚫ Now by applying the central limit theorem, we conclude that this total
current has a Gaussian distribution. This is the reason that thermal
noise can be very well modeled by a Gaussian random process.
⚫ Gaussian processes provide rather good models for some information
sources as well.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 112

 5.3.1 Gaussian Processes
Definition 5.3.1: A random process X (t) is a Gaussian process if for
all n and all (t1 , t2, . . . , tn), the random variables {X(ti)} ni=1 have a
jointly Gaussian density function.

It follows that at any time instant t0, the random variable X (t0) is
Gaussian and at any two points t1 , t2, random variables
(X(t1), X (t2)) are distributed according to a two dimensional jointly
Gaussian distribution.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 113

 Example 5.3.1
Determine the probability density function of the random variable X(3), where
X(t) be a zero-mean stationary Gaussian random process with the power spectral
density
Solution:
Since X (t) is a Gaussian random process, the probability density function of
random variable X(t) at any value of t is Gaussian. Therefore, X(3) is N(m, σ2).
Now we need to find m and σ2 , Since the process is zero mean, at any time
instance t, we have E[X(t)] = 0 which means
mean m = E[X(3)] = 0.
To find the variance

Since for a stationary process E[X(t1) X(t2)] = RX(t1 - t2)= RX (3 - 3)= RX (0)
2/4/2021 114
Latha, Dept. of ECE, ASE, Bengaluru
 We know that
PX = RX (0) and

Therefore, X(3) is N(0, 5000), or the density function for X(3) is

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 115

 Jointly Gaussian Random Processes
Definition 5.3.2: The random processes X (t) and Y(t) are jointly
Gaussian if for all n, m and all (t1 , t2, . . . , tn ) and (τ1, , τ 2, . . . , τ m), the
random vector [X(t1), X (t2), …… X (tn)] and [Y(τ 1), Y(τ 2) , . . Y(τ m)]
is distributed according to an n + m dimensional jointly Gaussian
distribution.

From this definition, it is obvious that if X (t) and Y(t) are jointly
Gaussian, then each of them is individually Gaussian two individually
Gaussian random processes are not always jointly Gaussian.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 116

Properties of the jointly Gaussian Random Processes

Property 1. If the Gaussian process X (t) is passed through an LTI

system, then the output process Y(t) will also be a Gaussian process.
Moreover, X (t) and Y(t) will be jointly Gaussian processes.

Property 2. For jointly Gaussian processes, uncorrelatedness and

Independence are equivalent.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 117

 5.3.2 White Processes
⚫ The term white process is used to denote processes in which all
frequency components appear with equal power.
⚫ The power spectral density is a constant for all frequencies or
independent of operating frequency.
⚫ This parallels the notion of "white light," which contain equal amount
of all frequencies in the visible spectrum.
Definition 5.3.3: A process X (t) is called a white process if it has a flat
power spectral density, i.e., if SX ( f ) is a constant for all f.

Figure 5.19 Power spectrum

of a white process

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 118

⚫ If we find the power content of a white process using
SX ( f ) = C, a constant, we will have

⚫ A white process is not a meaningful physical process, as no real

physical process can have infinite power.
⚫ Thermal noise can be closely modeled as white noise.
⚫ Quantum mechanical analysis of the thermal noise shows that it has a
power spectral density given by

⚫ Where h denotes Planck's constant (equal to 6.6 x 10-34 J x sec) and k

is Boltzmann's constant (equal to 1 .38 x 10-23 J/K). T denotes the
temperature in degrees Kelvin.
2/4/2021 119
Latha, Dept. of ECE, ASE, Bengaluru
 Figure 5.20 Power spectrum
of thermal noise

⚫ Spectrum achieves its maximum at f = 0, and the value of this

maximum is kT/2.
⚫ The spectrum goes to zero as f goes to infinity, but the rate of
convergence to zero is very slow.
⚫ For instance, at room temperature (T' = 300° K) Sn ( f ) drops to 90%
of its maximum at about f ≈ 2 x 1012 Hz, which is beyond the
frequencies employed in conventional communication systems.
⚫ Thus, we conclude that thermal noise, though not precisely white, for
all practical purposes can be modeled as a white process with power
spectrum equal to kT/2.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 120
⚫ The value kT is usually denoted by N0
⚫ The power spectral density of thermal noise is usually given as
Sn ( f ) = N0/2.
⚫ Looking at the autocorrelation function for a white process, we see
that

⚫ This shows that for all τ ≠ 0, we have Rn (τ) = 0.

⚫ Thus, if we sample a white process at two points t1 and t2 (t1 ≠ t2), the
resulting random variables will be uncorrelated.
⚫ If the random process is white and also Gaussian, any pair of random
variables X (t1 ), X (t2), where t1 ≠ t2, will also be independent.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 121

 Properties of the Thermal Noise

1. Thermal noise is a stationary process.

2. Thermal noise is a zero-mean process.
3. Thermal noise is a Gaussian process.
4. Thermal noise is a white process with a power spectral density
kT N 0
Sn ( f ) = =
2 2
It is clear that the power spectral density of thermal noise increases with
increasing the ambient temperature. Therefore, keeping electric
circuits cool makes their noise level low.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 122

 Lowpass and Bandpass signals
⚫ A low pass signal is a signal in which the spectrum (frequency content)
of the signal is located around the zero frequency.
⚫ A band pass signal is a signal with a spectrum far from the zero
frequency.
⚫ Consider a sinusoidal signal
⚫ This is a sinusoidal signal that can be represented by a phasor

⚫ The phasor has a magnitude of A and

a phase of θ.
⚫ We can expand the signal x (t) as

Figure 2.50 Phasor corresponding to

a sinusoid signal.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 123

 x(t) is a lowpass signal
Band width=W Hz
(W-0)

g(t) is a bandpass signal

Band width=2W

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 124

 ⚫ Sinusoidal signal x(t) is a single frequency band pass signal.
⚫ It has two components.
⚫ The first component is xc = A cos(θ), which is in the direction of
cos(2πfct). This is called the in-phase component.
⚫ The other component is xs = A sin(θ), which is in the direction of -
sin(2πfct). This is called the quadrature component.
⚫ We can also write this as

⚫ If we have a phasor with slowly varying functions then

where A (t) and θ (t) are slowly varying signals

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 125

⚫ In this case, the in-phase and quadrature components are

and we have
⚫ Here both the in-phase and quadrature components of a bandpass
signal are slowly varying signals, therefore, they are both lowpass
signals
⚫ Basically a bandpass signal can be represented in terms of two
lowpass signals, namely, its in-phase and quadrature components.
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 126
⚫ In this case, the complex lowpass signal

is called the lowpass equivalent of the bandpass signal x(t).

⚫ The envelope and the phase of the bandpass signal are

⚫ We can also express xl (t) as

⚫ We can express a bandpass signal in terms of two lowpass signals
either by in-phase and quadrature components or by envelope and
phase of the bandpass signal

127
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru
 5.3.3 Filtered Noise Processes

⚫ In many cases, the white noise generated in one stage of the

system is filtered by the next stage.
⚫ In the next stage we encounter filtered noise that is a bandpass
process.
⚫ In a bandpass process, sample functions are bandpass signals.
⚫ They can be expressed in terms of the in-phase and quadrature
components.
⚫ We study the main properties of bandpass noise processes and
⚫ particularly study the main properties of the in-phase and
quadrature processes.

2/4/2021 128
Latha, Dept. of ECE, ASE, Bengaluru
⚫ Let X (t) is the output of an ideal bandpass filter of bandwidth
W which is located at frequencies around fc
⚫ Since thermal noise is white and Gaussian, the filtered thermal noise
will be Gaussian but not white.
⚫ The power spectral density of the filtered noise will be

Since for ideal filters |H( f )|2= H( f )

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 129

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 130
 The power in the filtered noise or bandpass noise in this case is

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 131

 The power in the filtered noise or bandpass noise in this case is

2/4/2021 132
Latha, Dept. of ECE, ASE, Bengaluru
⚫ All bandpass filtered noise signals have an in-phase and
quadrature phase components that are lowpass signals.
⚫ Bandpass random process X (t) can be expressed

⚫ Xc(t) is in phase component and Xs(t) is quadrature component.

⚫ Xc(t) and Xs(t) are low pass processes.
⚫ We can also represent the filtered noise in terms of its envelope and
phase as

where A (t) and θ(t) are lowpass random processes

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 133

 Properties of the In-Phase and Quadrature
Processes
Filtered noise has the following properties for Xc(t) and Xs(t)
1. Xc(t) and Xs(t) are zero-mean, lowpass, jointly stationary, and jointly
Gaussian random processes.
2. If the power in process X (t) is PX, then the power in each of the
processes Xc(t) and Xs(t) is also PX. In other words,

3. Processes Xc(t) and Xs(t) have a common power spectral density. This
power spectral density is obtained by shifting the positive frequencies
in SX ( f ) to the left by fc shifting the negative frequencies of SX ( f ) to
the right by fc, and adding the two shifted spectra.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 134

The power in the in-phase and quadrature components in this case is
2N0W which is same as P1
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 135
 The power in the in-phase and quadrature components in this case is
N0W which is same as P2
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 136
4. If +fc and -fc are the axis of symmetry of the positive and negative
frequencies, respectively in the spectrum of H ( f ), then Xc(t) and
Xs(t) will be independent processes.
Example: H1( f )

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 137

 Noise Equivalent Bandwidth
⚫ When a white Gaussian noise passes through a filter, the output
process, although still Gaussian, will not be white anymore.
⚫ The filter characteristic shapes the spectral properties of the output
process, and we have

⚫ Now if we want to find the power content of the output process, we

have to integrate SY ( f ).
⚫ Thus

⚫ Therefore, to determine the output power, we have to evaluate the

integral

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 138

⚫ To do this calculation, we define Bneq ,the noise equivalent
bandwidth of a filter with the frequency response H ( f ), as

where Hmax is the maximum of |H( f ) | in the passband of the filter.

⚫ Now the output power is

Figure 5.23 Noise equivalent bandwidth of a typical filter.

⚫ Noise equivalent bandwidth Bneq , of filters and amplifiers is usually

provided by the manufacturer, which makes the power calculation
simpler
2/4/2021 139
Latha, Dept. of ECE, ASE, Bengaluru
 Example 5.3.4
Find the noise equivalent bandwidth of a lowpass RC filter.
Solution: For this filter,

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 140

 Here Hmax=1

Hence

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 141

Example 5.33: Let the random process X (t) be defined by X (t) = A + Bt
where A and B art independent random variables, each uniformly distributed
on [-1, l]. Find mX(t) and RX(t1 , t2).
Solution: A, B are uniformly distributed over [−1 1] so that
E[A] = E[B] = (1 - 1) / 2 =0.
mX(t) = E[A + Bt] = E[A] + E[B] t = 0

RX ( t1, t2) = E [ X( t1 ) X( t2 ) ] = E [ ( A + B t1 ) ( A + B t2 ) ]
= E [A2 + AB t2 + BA t1 + B2 t1t2 ]
= E [A2] + E [AB ] t2 + E [ BA ] t1 + E[B2] t1t2
The random variables A, B are independent therefore
E[AB] = E[A]E[B] = 0.
Furthermore

Therefore

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 142

Example 5.38 The random process X (t) is defined by
X (t) = X cos 2πf0t + Y sin 2πf0t, where X and Y are two zero-mean
independent Gaussian random variables each with the variance σ2.
1. Find mx(t).
2. Find RX (t + τ, t). Is X (t) stationary?
3. Find the power spectral density of X (t).
4. Answer Parts 1 and 2 for the case where σ2X ≠ σ2Y.
Solution:
1. mX(t) = E[X(t)] = E[X cos 2πf0t + E[Y sin 2πf0t)]
= E[X] cos 2πf0t + E[Y ] sin 2πf0t)
=0 since E[X] = E[Y] = 0.

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 143

2. RX (t + τ, t) = E [ X (t+ τ) X(t) ]
= E[(X cos(2πf0(t + τ)) + Y sin(2πf0(t + τ )))
(X cos (2πf0t) + Y sin(2πf0t))]
= E[X 2 cos (2πf0(t + τ)) cos (2πf0t)] +
E[XY cos (2πf0(t + τ)) sin(2πf0t)] +
E[YX sin(2πf0(t + τ)) cos (2πf0t)] +
E[Y 2 sin(2πf0(t + τ)) sin(2πf0t)]
= E[X 2] [ cos (2πf0(t + τ)) cos (2πf0t)] +
E[XY] [ cos (2πf0(t + τ)) sin(2πf0t)] +
E[YX] [sin(2πf0(t + τ)) cos (2πf0t)] +
E[Y 2 ] [sin(2πf0(t + τ)) sin(2πf0t)]

E[XY]= E[YX] = E[X] E[Y] =0, E[X 2] = E[Y 2 ] = σ2

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 144
 = σ2[ cos (2πf0(t + τ)) cos (2πf0t)] +
σ2 [sin(2πf0(t + τ)) sin(2πf0t)]
= σ2 cos(2πf0τ)
Since mX (t) =0 and RX (t + τ, t) is depending on τ, the process X(t) is a
stationary process.
3. The power spectral density is the Fourier transform of the
autocorrelation function, hence
SX ( f ) = σ2/2 [δ(f − f0) + δ(f + f0)] .
4. If σ2X ≠ σ2Y , then
mX(t) = E[X] cos(2πf0t) + E[Y ] sin(2πf0t) = 0
RX (t + τ, t) = E [ X (t+ τ) X(t) ]
= E[(X cos(2πf0(t + τ)) + Y sin(2πf0(t + τ )))
(X cos (2πf0t) + Y sin(2πf0t))]
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 145
 = E[X 2 cos (2πf0(t + τ)) cos (2πf0t)] +
E[XY cos (2πf0(t + τ)) sin(2πf0t)] +
E[YX sin(2πf0(t + τ)) cos (2πf0t)] +
E[Y 2 sin(2πf0(t + τ)) sin(2πf0t)]
= E[X 2] [ cos (2πf0(t + τ)) cos (2πf0t)] +
E[XY] [ cos (2πf0(t + τ)) sin(2πf0t)] +
E[YX] [sin(2πf0(t + τ)) cos (2πf0t)] +
E[Y 2 ] [sin(2πf0(t + τ)) sin(2πf0t)]
E[XY]= E[YX] = E[X] E[Y] =0, E[X 2] = σ2X E[Y 2 ] = σ2Y
RX (t + τ, t) = σ2X /2 [cos(2πf0(2t + τ)) + cos(2πf0τ)] +
σ2Y /2 [cos(2πf0τ) − cos(2πf0(2t + τ))]
= [(σ2X− σ2Y ) / 2] [cos(2πf0 (2t + τ))] +[ (σ2X+σ2Y ) / 2] [cos(2πf0τ)]
The process is not stationary as autocorrelation depends on t
2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 146
Example 5.44: Assume X (t) is a stationary process with the power spectral
density SX ( f ). This process passes through the system shown in
Figure P-5.44.
1. Is Y(t) stationary? Why?
2. What is the power spectral density of Y(t)?
3. What frequency components cannot be present in the output process and why?

Impulse response h (t ) =
d
(  (t ) +  (t − T ) ) = ( (t ) ) + (  ( t − T ) )
d d
dt dt dt
h (t ) =  ' (t ) +  ' ( t − T )
Frequency response H ( f ) = j 2  f + j 2  f e − j 2 f T
2/4/2021 147
Latha, Dept. of ECE, ASE, Bengaluru
1. The system with impulse response h (t ) =  ' (t ) +  ' ( t − T ) is an LTI system.
If input to the LTI system is a stationary process, then output is also stationary
process. Hence Y(t) is a stationary process.

2. Power spectral density of Y ( t ) is

SY ( f ) = SX ( f ) |H( f )|2
H(f ) = j2πf + j2πf e−j2πf T = j2πf [ 1+ e−j2πf T ], and
|H( f )|2 = 4π2f 2 | 1+ e−j2πf T |2
SY ( f ) = SX ( f ) 4π2f 2 | 1+ e−j2πf T |2
SY ( f ) = SX ( f ) 4π2f 2 |(1 + cos (2πf T ) - j sin(2πf T )|2
SY ( f ) = SX ( f ) 4π2f 2 [(1 + cos (2πf T ))2 + sin2(2πf T )]
SY ( f ) = SX ( f ) 4π2f 2 [(1 + cos2(2πf T ) + 2 cos (2πf T )) + sin2(2πf T )]
SY ( f ) = SX ( f ) 8π2f 2 (1 + cos (2πf T ))

2/4/2021 148
Latha, Dept. of ECE, ASE, Bengaluru
3. The power spectral density of the out put process is
SY ( f ) = SX ( f ) 8π2f 2 (1 + cos (2πf T ))
The following frequencies do not present at the out put process.
1. f =0, since there is f 2 in SY ( f ) expression, SY ( f ) = 0 when f 2 =0
k 1
2. f = +
T 2T

since SY ( f )=0 when cos (2  f T ) = − 1 or

2  f T = (2 k + 1 )  or
k 1
f = +
T 2T

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 149

 Example 5.45 :The stationary random process X (t) has a power
spectral density denoted by SX ( f ).
1. What is the power spectral density of Y (t) = X (t) - X (t - T) ?
2. What is the power spectral density of Z(t) = X'(t) - X (t) ?
3. What is the power spectral density of W(t) = Y(t) + Z(t )?
Solution:
1. Y (t) = X (t) - X (t - T) =X(t) * h(t)

h(t) = δ(t) – δ( t – T )
H(f)= 1 – e -j2πfT
|H ( f )|2 = | 1 – e -j2πfT |2 = |1 – ( cos(2πfT) –j sin(2πfT))|2
= 2 (1 - ( cos(2πfT) )
SY ( f ) = SX ( f ) |H ( f )|2
= SX ( f ) 2 (1 - cos(2πfT))

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 150

 2. Y (t) = Xꞌ(t) - X (t) =X(t) * h(t)
h(t) = δꞌ (t) – δ( t )
H(f)= j2πf – 1
|H( f )|2 = 4π2f 2 + 1
SY ( f ) = SX ( f ) |H ( f )|2
= SX ( f ) (4π2f 2 + 1 )

3. W(t) = Y(t) + Z(t ) = X (t) - X (t - T) + X'(t) - X (t)

= X'(t) - X (t - T)
W(t) = X(t) * h(t)
h(t) = δꞌ (t) - δ( t –T )
H(f)= j2πf – e-j2πfT = j2πf – ( cos(2πfT) –j sin(2πfT))
= – ( cos(2πfT) + j(2πf + sin(2πfT))
| H(f) |2= 1+ 4π2f 2 + 4π f sin(2πfT)
SZ( f ) = SX ( f ) (1+ 4π2f 2 + 4π f sin(2πfT))

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 151

 Example 5.4 :Under what conditions can two disjoint events A and B be
independent?
Solution: If two events are mutually exclusive (disjoint) then
P(A∪B) = P(A)∪P(B) which implies that
P(A∩B) =0. If the events are independent, then
P(A∩B) = P(A)∩P(B). Combining these two conditions we obtain that two
disjoint events are independent if
P(A ∩ B) = P(A)P (B) = 0
Thus, at least one of the events should be of zero probability

2/4/2021 Latha, Dept. of ECE, ASE, Bengaluru 152